Intelligent video analysis system with adjustable AI computing power

Abstract

The invention belongs to the technical field of computer vision, and discloses an intelligent video analysis system with adjustable AI computing power. A content complexity matrix is constructed through texture gradient change rate analysis, a high-density aggregation region and a low-density sparse region are identified, and a mapping relation between computing power requirements and video contents is established. A computing power granularity segmentation model is innovatively introduced, computing resources are accurately allocated to all blocks, a closed-loop optimization mechanism is formed through confidence monitoring, and dynamic adjustment of the computing power resources is achieved. The system guarantees the analysis quality of a key area, remarkably improves the resource utilization efficiency, and solves the problem of traditional balance between precision and efficiency. The technology is suitable for various resource-limited scenes such as smart city monitoring, automatic driving perception and industrial quality inspection, limited computing power can be intelligently distributed according to video content characteristics, and a stable and reliable analysis effect is provided.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to an intelligent video analysis system with adjustable AI computing power. Background Technology

[0002] Existing video analytics systems suffer from deficiencies in computing resource allocation, primarily due to their uniform resource allocation strategies. These strategies often overlook the spatial heterogeneity and temporal variations of video content, leading to insufficient analysis quality in complex areas and wasted resources in simpler areas. For example, in smart city surveillance systems, densely populated intersections receive the same computing power as static background areas, increasing the missed detection rate of key targets. In industrial quality inspection systems, treating product details the same as large, uniform areas makes accurate defect identification difficult. Current technologies also generally lack the ability to deeply quantify content complexity, failing to accurately distinguish multi-dimensional characteristics such as texture gradients, motion intensity, and information entropy, resulting in a lack of scientific basis for resource allocation. Furthermore, most systems employ statically preset resource scheduling strategies, which lag behind in responding to sudden changes in scenarios (such as severe weather changes in autonomous driving or sudden changes in lighting in security monitoring), failing to adjust computing power allocation in a timely manner to cope with sudden increases in complexity. Most seriously, existing systems lack closed-loop quality monitoring mechanisms. When edge computing devices experience a decrease in analysis confidence due to insufficient computing power, the system cannot detect and compensate for this, leading to unstable analysis results. In high-concurrency video stream processing scenarios, coarse-grained resource management further exacerbates computational bottlenecks, making it impossible to perform fine-grained scheduling of computing power based on content importance. This leads to an overall performance decline in resource-constrained environments. Particularly in streaming media analytics platforms that process multiple video streams simultaneously, the lack of a dynamic cross-stream computing power balancing mechanism based on content complexity means that a single complex video stream may consume excessive resources, compressing the processing quality of other streams. This results in an uneven overall system service quality, failing to meet the differentiated analysis needs of practical applications.

[0003] In view of this, the present invention proposes an intelligent video analysis system with adjustable AI computing power to solve the above problems. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: an intelligent video analysis system with adjustable AI computing power, comprising: The video frame acquisition module is used to acquire a continuous frame sequence of the video stream, divide each frame into multiple local blocks, and extract the texture gradient change rate between corresponding blocks of adjacent frames. The complexity analysis module is used to calculate the spatial complexity and temporal complexity of each block based on the texture gradient change rate, and to construct a content complexity matrix containing the spatial complexity and temporal complexity values. The feature point recognition module is used to extract feature points from the content complexity matrix, calculate the distribution density of feature points in each block, and identify high-density clustered areas and low-density sparse areas. The computing power demand calculation module is used to obtain computing time data and analysis accuracy data of historical frames under different computing power configurations. Based on the area ratio of high-density clustered regions and low-density sparse regions, it calculates the computing power demand coefficient of each block in the current frame. The computing power partitioning and scheduling module is used to construct a computing power granular partitioning model based on the computing power demand coefficient, divide the available computing power resources into multiple computing power granular units, and allocate a corresponding number of computing power granular units to each block. The execution control module is used to generate a computing power scheduling sequence for video analysis tasks based on the allocation results of computing power granular units, and control the AI ​​inference engine to perform video content analysis according to the computing power scheduling sequence; The confidence monitoring module is used to collect and analyze the confidence values ​​of the output results in real time, calculate the decay gradient of the confidence values ​​within a preset time window, and identify the confidence fluctuation range caused by insufficient computing power. The adaptive optimization module is used to dynamically adjust the allocation ratio of computing power granular units according to the fluctuation amplitude and duration of the confidence fluctuation range, so as to realize the adaptive optimization configuration of computing power resources during video analysis. The modules are connected via wired and / or wireless means to enable data transmission between them.

[0005] The technical effects and advantages of the AI ​​computing power adjustable intelligent video analysis system of the present invention are as follows: This invention accurately identifies and prioritizes the analysis quality of critical content areas, ensuring that the detection accuracy of important targets remains unaffected even in resource-constrained edge computing environments. This intelligent resource allocation strategy enables the system to perform exceptionally well in highly dynamic scenarios, effectively avoiding the performance crashes that plague traditional systems in complex environments. The adaptive optimization mechanism of this invention ensures consistently stable analysis quality throughout long-term operation, maintaining reliable analysis results even under external interference factors such as day-night cycles and weather changes, significantly improving the practicality and reliability of various video surveillance and analysis applications. Most importantly, this invention overcomes the traditional trade-off between resources and quality, enabling the simultaneous consideration of global monitoring scope and detailed local analysis even in environments with limited computing power. Attached Figure Description

[0006] Figure 1 This is a schematic diagram of an intelligent video analysis system with adjustable AI computing power according to the present invention. Detailed Implementation

[0007] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0008] This application provides an AI-powerable intelligent video analysis system. The system's execution entities include, but are not limited to, video surveillance platforms, intelligent analysis centers, edge computing devices, and multimedia processing systems, which can be considered general computing nodes in this application. The computing power scheduling system includes, but is not limited to, at least one cloud-based computing resource scheduling engine, a distributed computing power allocation system, and an intelligent resource optimizer.

[0009] Please see Figure 1 In this embodiment of the invention, an intelligent video analysis system with adjustable AI computing power includes: The video frame acquisition module acquires continuous frame sequences from the video stream, dividing each frame into multiple local blocks and extracting the texture gradient change rate between corresponding blocks in adjacent frames. This module acquires high-quality video frames in real-time through the video stream decoding interface, employs an adaptive grid algorithm to divide each frame into multiple blocks of variable size, and calculates the change characteristics of corresponding blocks between adjacent frames through multi-directional texture gradient analysis. The texture gradient change rate, as a fundamental measure of the dynamic characteristics of the blocks, reflects the spatiotemporal variation pattern of the video content, providing crucial input for subsequent complexity analysis.

[0010] The complexity analysis module calculates the spatial and temporal complexity values ​​of each block based on the texture gradient change rate, constructing a content complexity matrix that includes both spatial and temporal complexity values. This module performs spatial analysis on the gradient relationships between blocks within a single frame and temporal correlation analysis on the gradient sequence over time to comprehensively evaluate the complexity of the video content. Spatial complexity reflects the degree of texture difference between a block and its neighbors, while temporal complexity quantifies the regularity of block changes over time. Together, they constitute the content complexity matrix, providing a two-dimensional evaluation basis for feature point extraction and computational demand calculation.

[0011] The feature point recognition module extracts feature points from the content complexity matrix, calculates the distribution density of feature points in each block, and identifies high-density clustered regions and low-density sparse regions. This module employs an improved feature detection algorithm to extract salient local extrema from the complexity matrix and analyzes the spatial distribution patterns of these feature points using density clustering. High-density clustered regions typically correspond to important, information-rich, and complex content in the video, while low-density sparse regions reflect relatively static or simple background content. This region division provides a crucial basis for subsequent differentiated computing power allocation.

[0012] The computing power requirement calculation module acquires computation time and analysis accuracy data for historical frames under different computing power configurations. Based on the area ratio of high-density clustered regions and low-density sparse regions, it calculates the computing power requirement coefficient for each block in the current frame. This module establishes a computing power-accuracy mapping relationship through historical data analysis, thereby predicting the computing power requirement for the current frame. The unevenness of complexity distribution, as a key indicator, reflects the complexity distribution characteristics of video content and directly affects the overall computing power allocation strategy. By combining the content complexity matrix and the accuracy sensitivity curve, a reasonable computing power requirement coefficient is allocated to each block, ensuring the analysis quality of important areas while avoiding resource waste.

[0013] The computing power partitioning and scheduling module is used to construct a granular partitioning model based on computing power demand coefficients. This model divides available computing power resources into multiple granular units and allocates a corresponding number of these units to each block. This module divides the overall computing power resources according to the smallest granularity baseline, forming a flexibly deployable pool of computing power units. Through priority ranking and proportional allocation mechanisms, it ensures that high-density clustered areas receive sufficient computing resources while rationally allocating remaining computing power. The computing power allocation mapping table, as the output of the partitioning model, directly guides the resource scheduling of the AI ​​inference engine, achieving fine-grained management of computing power.

[0014] The execution control module generates a computing power scheduling sequence for video analysis tasks based on the allocation results of computing power granular units, and controls the AI ​​inference engine to execute video content analysis according to the computing power scheduling sequence. This module converts the computing power allocation mapping table into specific hardware resource scheduling instructions, including computing core allocation, memory bandwidth adjustment, and inference precision settings, forming a complete execution plan. Through deep integration with the AI ​​inference engine, dynamic allocation of computing power resources is achieved, enabling the analysis process to be executed efficiently according to preset strategies while maintaining the system's real-time responsiveness.

[0015] The confidence level monitoring module collects and analyzes the confidence level values ​​of the output results in real time, calculates the decay gradient of the confidence level values ​​within a preset time window, and identifies confidence level fluctuation ranges caused by insufficient computing power. This module continuously tracks the quality indicators of AI inference results and identifies patterns of abnormal confidence level fluctuations through time-series analysis. Through sliding window analysis and derivative calculation, it accurately captures the downward trend of confidence level and, combined with backtracking analysis of computing power utilization, determines the fluctuation range caused by insufficient computing power. This information directly reflects the effectiveness of the current computing power allocation strategy, providing feedback for subsequent dynamic adjustments.

[0016] The adaptive optimization module dynamically adjusts the allocation ratio of computing power granular units based on the fluctuation amplitude and duration of the confidence level fluctuation range, achieving adaptive optimization of computing power resources during video analytics. This module establishes a fluctuation assessment coordinate system, classifies adjustment strategy levels according to fluctuation characteristics, and executes corresponding computing power reallocation operations. The three-level adjustment strategy (mild, moderate, and severe) addresses different degrees of insufficient computing power, employing progressive intervention measures from fine-tuning to reconstruction to ensure the system can quickly respond to and correct unreasonable resource allocation, forming a closed-loop adaptive optimization mechanism.

[0017] The modules are connected via wired and / or wireless means to enable data transmission between them.

[0018] In this embodiment of the invention, the detailed implementation steps for dividing each frame into multiple local blocks and extracting the texture gradient change rate between corresponding blocks in adjacent frames include: An adaptive grid partitioning algorithm is employed to divide video frames into multiple variable-sized local blocks, recording the spatial coordinates of each block. Adaptive grid partitioning is fundamental for content-aware analysis, dynamically adjusting block sizes to adapt to the characteristics of the video content. The partitioning process first performs an initial uniform grid partitioning of the video frame, then adaptively adjusts it based on a content complexity evaluation function. For regions with rich textures and dense edges, a finer-grained partitioning is used to preserve detailed information; for regions with simple textures and straightforward content, larger blocks are used to improve computational efficiency. Spatial coordinates are represented using normalized methods to ensure consistent processing across different video resolutions and to provide positional references for block matching in subsequent adjacent frames.

[0019] For each local block, multi-directional texture gradients are extracted, and the weighted vector sum of these gradients is calculated, denoted as the block texture feature vector. Texture gradient extraction is a crucial step in capturing visual features, revealing content structure by analyzing spatial variations in pixel intensity. The extraction process employs an improved Sobel operator to compute gradient responses in multiple directions (typically eight principal directions), covering horizontal, vertical, and diagonal directions. For the gradient response in each direction, its magnitude and direction are calculated, and then weighted fusion is used to generate a comprehensive feature. The weight allocation is adaptively determined based on the saliency of the gradient response, ensuring that the principal texture directions contribute more significantly. The final feature vector is a high-dimensional representation that encodes the texture structure information of the block, providing a feature basis for comparing adjacent frames.

[0020] Obtain the texture feature vectors of blocks with the same spatial coordinates in adjacent frames, calculate the Euclidean distance between them, and denote it as the texture dissimilarity between blocks. Texture dissimilarity calculation is a core metric for quantifying temporal change, reflecting content evolution by comparing feature vectors. The calculation process first matches corresponding blocks in adjacent frames based on spatial coordinates, and then calculates the Euclidean distance between their texture feature vectors. The Euclidean distance intuitively reflects the geometric distance in feature space, and the formula is: ; in, For blocks and Texture difference and These are the texture feature vectors of the corresponding blocks. and For the vector of the first Each component represents a differentiability value. A higher dissimilarity value indicates more significant changes in the content of a block, directly reflecting the dynamism and complexity of the video in that area.

[0021] The temporal derivative of the texture dissimilarity is calculated to obtain the rate of change of the texture dissimilarity over time, denoted as the texture gradient rate of change. The texture gradient rate of change is a high-order feature describing the dynamic characteristics of content changes, and its temporal derivative reflects the acceleration of the change. The calculation process uses the central difference method, employing the texture dissimilarity of three consecutive frames to calculate the rate of change. ; in, For time The rate of change of texture gradient, , These represent the texture difference between the previous and next frames, respectively. This represents the inter-frame time interval. A positive rate of change indicates increasing difference and accelerated change; a negative rate of change indicates decreasing difference and slowed change. This metric effectively captures abrupt changes and trend shifts in video content, providing a dynamic basis for complexity assessment.

[0022] Outlier removal is performed on the texture gradient change rate, retaining valid gradient change rates within a preset percentile interval, and updating the texture gradient change rate of each block. Outlier removal is a key process for improving feature stability, filtering out unreliable extreme values ​​through statistical analysis. The removal process first calculates the distribution statistics of the gradient change rate, including the median and interquartile range; then, it determines a reasonable percentile interval (usually 10%-90% or 5%-95%, adjusted according to application requirements); finally, values ​​outside the interval are marked as outliers and replaced using interval boundary values ​​or local interpolation. This statistical filtering method can effectively handle data anomalies caused by noise, coding artifacts, or tracking errors, improving the reliability of subsequent complexity analysis. The updated texture gradient change rate more accurately reflects the true change characteristics of the video content, providing high-quality input for spatial and temporal complexity calculations.

[0023] In this embodiment of the invention, the detailed implementation steps for calculating the spatial complexity and temporal complexity values ​​of each block based on the texture gradient change rate, and constructing a content complexity matrix containing the spatial complexity and temporal complexity values, include: Spatial variance analysis (SAVA) is performed on the texture gradient change rate of all blocks within a single frame to calculate the gradient dispersion of each block and its neighboring blocks, denoted as the spatial complexity value. SAVA is an effective method for assessing the heterogeneity of local content, quantifying spatial complexity by calculating the degree of difference between a block and its surrounding environment. The analysis process first defines the neighborhood of each block, typically using a 3×3 or 5×5 spatial window; then, the difference in gradient change rate between the central block and its neighboring blocks is calculated to assess their dispersion. The formula for calculating the spatial complexity value is: ; in, For blocks The space complexity value, The texture gradient change rate of the central block. For neighboring blocks The rate of change of texture gradient, For blocks The neighborhood set, The size is the neighborhood size. Regions with high spatial complexity values ​​typically have complex textures or distinct boundaries, requiring more computational resources for accurate analysis.

[0024] A temporal sequence of texture gradient change rates for the same block across multiple consecutive frames is established, and the autocorrelation coefficient of the temporal sequence is calculated, denoted as the temporal correlation degree. Temporal autocorrelation analysis is an important method for evaluating temporal coherence, revealing temporal patterns by calculating the correlation between the sequence and its time-shifted versions. The analysis process first constructs a gradient change rate temporal sequence containing N frames (usually 10-30 frames) for each block; then, the autocorrelation coefficient of this sequence at different time delays is calculated to evaluate the periodicity and regularity of the sequence. The autocorrelation coefficient is calculated using a standardized formula to ensure that the result is within the range of [-1, 1], with values ​​closer to 1 indicating a more regular temporal pattern. The temporal correlation degree is the average absolute value of all autocorrelation coefficients, reflecting the overall predictability of the block's change pattern.

[0025] The temporal complexity value of each block is calculated based on the reciprocal of the temporal correlation degree and the peak fluctuation range of the texture gradient change rate. The temporal complexity value is a comprehensive indicator for evaluating the difficulty of changes in the time dimension, combining two key factors: predictability and the magnitude of change. The calculation process first takes the reciprocal of the temporal correlation degree, reflecting temporal uncertainty; then considers the peak fluctuation range of the gradient change rate to quantify the severity of the change; finally, a weighted combination yields the final temporal complexity value. The calculation formula is:

[0026] in, For blocks The time complexity value, For temporal correlation degree, This represents the peak fluctuation range of the gradient rate of change. This is the weighting coefficient, typically set to 0.5-1.0. Regions with high temporal complexity usually contain rapid changes, irregular movements, or scene transitions, requiring higher time sampling rates and processing power.

[0027] The spatial complexity and temporal complexity values ​​are weighted and fused, with the fusion weights adaptively determined based on the motion intensity of the video scene. Complexity fusion is a crucial step in balancing the spatial and temporal dimensions, adapting to different scene characteristics through dynamic weights. The fusion process first assesses the overall motion intensity of the video, quantifying the motion level through global motion vector analysis or background separation techniques; then, it dynamically adjusts the weight ratio of spatial and temporal complexity based on the motion intensity; finally, it calculates the fusion complexity through weighted summation. For high-motion scenes, the weight of temporal complexity increases; for static scenes, the weight of spatial complexity dominates. This adaptive weighting mechanism ensures that the complexity assessment accurately reflects the main challenging factors of the video content, providing a reasonable basis for subsequent computing power allocation.

[0028] The weighted fusion complexity values ​​are arranged according to the spatial coordinates of each block to generate a content complexity matrix. Each element of the content complexity matrix corresponds one-to-one with the spatial location of a block. The content complexity matrix is ​​a two-dimensional mapping of the video content characteristics, intuitively reflecting the spatial distribution of complexity. The construction process first determines the matrix dimension to correspond to the block division of the video frame; then, the fusion complexity value of each block is filled into the corresponding matrix position, maintaining the spatial relationship; finally, optional smoothing is applied to reduce noise and enhance structural features. The complexity matrix can be visualized using a heatmap, intuitively showing the complexity distribution of the content. High-value areas typically correspond to important content or difficult areas in the video. The matrix serves as input for feature point extraction, providing a structured data foundation for identifying key regions.

[0029] In this embodiment of the invention, the detailed implementation steps for calculating the computing power requirement coefficient of each block in the current frame based on the area ratio of high-density clustered regions to low-density sparse regions include: The pixel areas of high-density clustered regions and low-density sparse regions in the current frame are statistically analyzed, and the ratio of their areas is denoted as the complexity distribution imbalance. Complexity distribution imbalance is a key indicator for evaluating video content characteristics, reflecting the proportional relationship between complex and simple regions. The statistical process first determines the boundaries between high-density and low-density regions based on feature point density thresholds; then, it calculates the number of pixels occupied by each type of region; finally, it calculates the ratio between the two to obtain the imbalance. A high imbalance indicates that the video content has a significant complexity gradient, suitable for differentiated computing power allocation strategies; a low imbalance indicates that the content complexity is relatively uniform, suitable for balanced resource allocation. This indicator directly affects the overall computing power allocation ratio, providing macro-level guidance for resource scheduling.

[0030] This method extracts historical samples with similar complexity imbalances from historical frame computation time data, establishing a mapping relationship between complexity imbalance and total computing power consumption. Historical data analysis forms the empirical basis for predicting computing power requirements, and similarity matching improves prediction accuracy. The analysis process first establishes a historical frame database, recording the relationship between imbalance and actual computing power consumption in different scenarios. Then, based on the current frame's imbalance, the most similar historical samples are found through k-nearest neighbor search. Finally, a mapping function from imbalance to computing power consumption is established using a weighted average or regression model. This historical experience-based prediction method can adapt to different types of video content, considers the actual impact of scene characteristics on computing power requirements, and provides a reliable basis for the overall computing power budget of the current frame.

[0031] Based on the mapping relationship, the total computing power consumption of the current frame is predicted. Combining this with the complexity values ​​of each block in the content complexity matrix, the weight of each block in the total computing power consumption is calculated. The block's computing power share is the basis for micro-resource allocation, and the importance of each block is determined by its complexity contribution. The calculation process first estimates the overall computing power requirement of the current frame based on the mapping relationship; then, it calculates the proportion of each block's complexity to the total complexity of the entire frame; finally, it preliminarily allocates the total computing power according to the complexity proportion, deriving the theoretical weight of each block. This proportional allocation method ensures that computing power resources are proportional to content complexity, with complex areas receiving more resources and simple areas using fewer resources, achieving preliminary optimization of resource utilization.

[0032] This process involves acquiring historical frames with varying computing power configurations to analyze accuracy data and identifying the sensitivity curve between accuracy loss rate and computing power reduction rate. Sensitivity analysis is a key method for balancing resource efficiency and analysis quality, guiding precise allocation through a quantified relationship curve. The analysis process first collects accuracy test data under different computing power configurations, covering multiple configuration points from high accuracy and high computing power to low accuracy and low computing power. Then, curve fitting is used to establish a functional relationship between the accuracy loss rate and the computing power reduction rate. Finally, the curve characteristics are analyzed to identify critical points and sensitive intervals. The sensitivity curve typically exhibits a non-linear relationship; accuracy decreases slowly before a certain threshold, but deteriorates rapidly after exceeding the threshold. This relationship characteristic provides a theoretical basis for the reasonable reduction of computing power resources, enabling the system to maximize resource savings within an acceptable range of accuracy loss.

[0033] Based on sensitivity curves, a precision guarantee threshold is set for each block. The computing power requirement coefficient for each block is determined by multiplying the precision guarantee threshold by its weighted proportion. This coefficient is the direct basis for the final computing power allocation, comprehensively considering both complexity proportion and precision requirements. The determination process first sets different precision guarantee thresholds based on the importance of the block's content and application needs, setting higher thresholds for important areas and accepting lower precision for less important areas. Then, the precision threshold is multiplied by the previously calculated weighted proportion to obtain the adjusted requirement coefficient. Finally, normalization is performed so that the sum of the requirement coefficients for all blocks is 1, facilitating subsequent resource allocation. This precision-sensitive adjustment mechanism ensures the optimal return on investment for computing power resources, guaranteeing the analysis quality of key areas while avoiding excessive resource allocation to non-critical areas, providing a scientific basis for resource optimization in actual deployment.

[0034] In this embodiment of the invention, a granular computing power partitioning model is constructed based on the computing power demand coefficient. Available computing power resources are divided into multiple granular computing power units, and a corresponding number of granular computing power units are allocated to each block. The detailed implementation steps include: The process involves obtaining the total available computing resources of the AI ​​inference engine and dividing it by a preset minimum granularity benchmark to obtain the total number of divisible computing power units. Computing resource quantization is a fundamental step in achieving fine-grained scheduling, providing a manageable resource pool through standard unit partitioning. The quantization process first obtains the currently available computing resources, including the number of CPU / GPU cores, memory bandwidth, and dedicated accelerators, through the system API or resource monitoring module. Then, based on application characteristics and hardware architecture, a suitable minimum granularity benchmark is determined, representing the smallest resource unit for effective scheduling. Finally, a simple division is used to calculate the total number of currently available computing power units. The choice of granularity needs to balance flexibility and management overhead; too small a granularity increases scheduling complexity, while too large a granularity reduces allocation accuracy. A reasonably set granularity unit can meet the needs of fine-grained scheduling while maintaining system operating efficiency.

[0035] Based on the computing power demand coefficient of each block, the theoretical number of computing power granular units required for each block is calculated, and this number is then normalized. Theoretical demand calculation is the process of transforming abstract demands into specific resource requests, quantifying the demand through mathematical mapping. The calculation process first multiplies the computing power demand coefficient by the total number of computing power granular units to obtain the ideal allocation for each block; then, a rounding function (usually rounding up) is applied to ensure that each block receives at least one basic computing power unit; finally, normalization adjustments are performed to ensure that the total allocation for all blocks does not exceed the available total. This proportion-based allocation strategy directly reflects the relative importance of blocks, with more complex blocks receiving more computing power units and simpler blocks receiving fewer resources, achieving initial optimization of resource utilization.

[0036] The system determines whether the total number of computing power units meets the theoretical requirements of all blocks. If not, it prioritizes the theoretical computing power units. Resource constraint handling is a necessary step in a real-world system, resolving resource contention through a priority mechanism. The process first compares the total theoretical demand with the total available resources. When demand exceeds supply, a priority ranking mechanism is activated, calculating a comprehensive priority score based on factors such as block importance, complexity, and location. Then, all blocks are arranged in descending order of their scores, forming a priority queue for resource allocation. Priority calculation typically considers factors such as complexity, block location (central areas are usually more important), and historical performance analysis to ensure that critical blocks receive sufficient support even with limited resources. This dynamic priority mechanism is a key strategy for the system to cope with resource constraints, ensuring that limited resources are used to their maximum potential.

[0037] Based on priority ranking, computing power units are allocated first to blocks within high-density clustered areas, with remaining units distributed proportionally to other blocks. Differentiated allocation is the core strategy for resource optimization, ensuring key areas are prioritized through skewed allocation. The allocation process first identifies sets of blocks belonging to high-density clustered areas, which typically contain crucial information from the video. Then, according to priority, the required computing power units are allocated to blocks within high-density areas until their theoretical requirements are met or a preset resource limit is reached (usually 60%-80% of the total resources). Finally, the remaining computing power units are distributed proportionally to blocks in low-density sparse areas. This "prioritized allocation with overall consideration" strategy ensures the analysis quality of critical areas while avoiding the extreme case of completely unresourced peripheral areas, achieving a balance between overall performance and resource efficiency.

[0038] A computing power allocation mapping table is generated, containing the correspondence between each block and the number of computing power granular units allocated to it. This mapping table serves as the output of the computing power granularity partitioning model. The generation of the mapping table is the final output of the computing power allocation, providing a clear mapping relationship from blocks to resources. The generation process first creates a data structure recording the unique identifier, spatial coordinates, complexity value, priority score, and number of allocated computing power units for each block; then, it sorts the blocks by identifier or spatial location for easy subsequent lookup and access; finally, metadata is added, including generation time, total resource quantity, and allocation strategy parameters, enhancing traceability and interpretability. As a direct output of the computing power partitioning model, the mapping table is passed to the execution control module and transformed into specific hardware resource scheduling instructions. This explicit mapping relationship enables the system to achieve fine-grained resource management, providing appropriate computational support for each block, ultimately improving the overall performance and efficiency of video analytics.

[0039] In this embodiment of the invention, the detailed implementation steps for real-time acquisition and analysis of the confidence level of the output results, calculation of the decay gradient of the confidence level within a preset time window, and identification of confidence level fluctuation ranges caused by insufficient computing power include: The confidence score for each detected target is extracted from the video analysis results output by the AI ​​inference engine, and these scores are recorded as a time series. Confidence score data collection is a fundamental step in quality monitoring, reflecting changes in analytical performance through continuous tracking. The collection process first extracts confidence metrics from the AI ​​inference engine's output, including confidence scores for object detection, IoU values ​​for segmentation, and probability distributions for classification. Then, this data is organized by timestamp to form a continuous time series. Simultaneously, each confidence score is associated with its corresponding block ID and computing power allocation record for subsequent analysis. As a direct indicator of AI analysis quality, the trend of confidence score changes effectively reflects the sufficiency of computing resources, providing real-time feedback for dynamic adjustments. The system typically maintains historical data within a sliding time window, with the window size set according to application requirements, typically ranging from 30 to 300 frames, capturing both short-term fluctuations and identifying long-term trends.

[0040] Within a preset time window, a moving average filter is applied to the confidence score numerical sequence to obtain a smoothed confidence score curve. Signal filtering is a crucial process for reducing noise interference, extracting the main trend of confidence scores through smoothing. The filtering process employs a weighted moving average algorithm, performing convolution operations on the original confidence score sequence to eliminate the influence of high-frequency noise and random fluctuations. The filtering window size is dynamically adjusted according to data characteristics, typically ranging from 5 to 15 frames. A window that is too small cannot effectively filter out noise, while a window that is too large may mask important short-term changes. The smoothed curve after filtering retains the main trend of confidence score changes and filters out random fluctuations, providing a high-quality input signal for subsequent gradient analysis. This allows the system to more accurately identify the true performance degradation caused by insufficient computing power, rather than temporary fluctuations.

[0041] The first derivative of the confidence level smoothing curve is calculated and denoted as the instantaneous confidence level decay rate. The percentage of time periods with a negative instantaneous confidence level decay rate is then statistically analyzed. Decay rate analysis is a core technique for detecting performance degradation, quantifying the rate of change through derivative calculation. The analysis process first calculates the discrete first derivative of the smoothing curve, using the central difference method to improve accuracy; then, the derivative value at each time point is evaluated, with negative values ​​indicating a decrease in confidence level; finally, the percentage of negative points within the entire time window is statistically analyzed. Under normal circumstances, the confidence level should be relatively stable or fluctuate slightly, with the percentage of negative values ​​typically not exceeding 30%; however, when computing power is insufficient, the confidence level will continuously decrease, leading to a significant increase in the percentage of negative values. This statistically based method can effectively distinguish between normal fluctuations and system performance degradation, providing a reliable quantitative standard for identifying fluctuation ranges.

[0042] When the time percentage exceeds a preset fluctuation threshold, the corresponding time period is marked as a candidate confidence fluctuation interval. The fluctuation threshold is typically set between 40% and 60%, and is dynamically adjusted based on the application's stability requirements and fault tolerance. A low threshold makes the system more sensitive and can detect problems earlier, but may increase false alarms; a high threshold is more conservative, reducing false alarms, but may delay intervention. Marking candidate intervals provides time range positioning for subsequent computing power correlation analysis, narrowing the search space for problem investigation.

[0043] A retrospective analysis of computing power allocation records within the candidate confidence fluctuation range is performed to determine whether the utilization rate of computing power granular units has reached saturation during this period. Computing power correlation analysis is a key verification for determining the cause of performance degradation, establishing causal relationships through historical data backtracking. The analysis process first retrieves computing power allocation records and resource utilization status within the fluctuation range; then, it calculates the computing power utilization rate of each block, typically assessed using indicators such as processor utilization, memory bandwidth utilization, or dedicated accelerator load; finally, it determines whether saturation has been reached, i.e., resource utilization consistently approaches its upper limit (usually above 90%). Saturation indicates that the currently allocated computing power is insufficient to support high-quality analysis, and is the direct cause of the confidence decline. This causal analysis based on historical data accurately identifies problems caused by insufficient computing power, avoiding misjudging other factors (such as changes in the video content itself) as resource issues, thus improving the accuracy of system diagnosis.

[0044] Candidate confidence fluctuation intervals where utilization reaches saturation are identified as confidence fluctuation intervals caused by insufficient computing power, and the fluctuation amplitude and duration of these intervals are recorded. The final confirmation of the fluctuation intervals is the result of multiple verifications, comprehensively considering both confidence changes and resource status. The confirmation process first filters out periods where computing power utilization reaches saturation from candidate intervals; these intervals meet the two necessary conditions of fluctuation and resource constraints. Then, the key characteristics of the intervals are calculated, including fluctuation amplitude (maximum decrease in confidence) and duration (length of continuous fluctuation). Finally, the confirmed fluctuation intervals and their characteristics are recorded and stored as the basis for subsequent adjustments. Fluctuation amplitude directly reflects the severity of performance loss, while duration indicates the persistence of the problem; both together determine the urgency and strength of subsequent adjustments. This detailed feature recording enables the system to implement precise, targeted adjustments, rather than a simple one-size-fits-all response.

[0045] In this embodiment of the invention, the detailed implementation steps for dynamically adjusting the allocation ratio of computing power granular units based on the fluctuation amplitude and duration of the confidence level fluctuation range include: A two-dimensional assessment coordinate system is established, mapping confidence level fluctuation intervals to their corresponding positions within the coordinate system. This two-dimensional coordinate system serves as a visualization tool for problem severity grading, enabling comprehensive multi-factor assessment through spatial mapping. The construction process first determines the coordinate axis range: the X-axis represents duration, typically normalized to the [0,1] interval, and the Y-axis represents fluctuation amplitude, also normalized to the [0,1] interval. Then, the characteristic values ​​of each fluctuation interval are mapped to a point in the coordinate system. Finally, regions are divided based on the point's location, forming decision boundaries. This two-dimensional assessment method considers both the severity (fluctuation amplitude) and the persistence (duration) of the problem, providing a more comprehensive assessment perspective than a single indicator and offering a theoretical basis for subsequent grading adjustments.

[0046] The computing power adjustment strategy is categorized into three levels based on the location of the fluctuation: mild, moderate, and severe. This categorization forms the basis for differentiated responses, enabling precise intervention through a hierarchical mechanism. The process begins by defining decision boundaries in the evaluation coordinate system, typically using diagonal lines or curves to separate different regions. Then, the level is determined based on the location of the fluctuation point: regions closer to the origin (low amplitude, short duration) correspond to mild adjustment, intermediate regions to moderate adjustment, and regions farther from the origin (high amplitude, long duration) to severe adjustment. This severity-based hierarchical strategy allows the system to take intervention measures of appropriate intensity according to the urgency of the problem, avoiding overreaction or underreaction, and improving the targeting and efficiency of the adjustment.

[0047] For mild adjustments, the proportion of computing power allocated to high-density clustered areas is increased by a factor of one multiple of the preset baseline value. Mild adjustments are the gentlest intervention, mitigating minor issues through small resource allocation adjustments. The adjustment process first identifies current high-density clustered areas, which are typically the key points of computing power demand; then, the computing power allocation proportion in these areas is increased, typically by 1.2-1.5 times the baseline value; simultaneously, the allocation in low-density areas remains unchanged, meeting the increased demand through minor adjustments to the overall total computing power. This gentle adjustment is suitable for short-term performance fluctuations, quickly responding to and mitigating problems without causing drastic changes to the overall resource allocation, maintaining system stability and predictability.

[0048] For moderate adjustment levels, while increasing computing power in high-density clustered areas, the allocation of computing power in low-density sparse areas is reduced, with the adjustment amount being twice the preset baseline value. Moderate adjustment is a balancing intervention measure that addresses obvious problems through resource reallocation. The adjustment process first significantly increases the proportion of computing power in high-density areas, typically by 1.5-2.0 times the baseline value; simultaneously, it appropriately reduces resource allocation in low-density areas, typically by 10%-30%; finally, it rebalances the resource proportions of each area to ensure that the total allocation does not exceed available resources. This adjustment method achieves optimized resource reallocation under the constraint of fixed total resources, improving the analysis quality of critical areas through a "one for all" strategy while controlling performance loss in edge areas within acceptable limits, making it suitable for persistent moderate performance problems.

[0049] For severe adjustments, the construction process of the granularity-based computing power partitioning model is re-executed, and the minimum granularity benchmark value is reduced to obtain finer-grained computing power allocation capabilities. Severe adjustments are the most powerful intervention, addressing serious problems through systematic restructuring. The adjustment process first reduces the minimum granularity benchmark value, typically to 50%-70% of its original value, improving the granularity of resource allocation; then, the entire granularity-based computing power partitioning model construction process is re-executed, reassessing the complexity and importance of each block; finally, a completely new allocation scheme is generated, potentially involving block re-partitioning, priority recalculation, and global resource reallocation. This thorough adjustment approach is suitable for long-term, severe performance problems, fundamentally reconstructing resource allocation strategies, but it incurs significant computational overhead and is therefore triggered only when necessary. The strategy of reducing granularity is particularly helpful in handling scenarios with extremely uneven complexity distribution, enabling the system to more accurately concentrate resources in the most critical small areas, avoiding resource waste in coarse-grained allocation.

[0050] The adjusted computing power allocation scheme is applied to the analysis of subsequent video frames, and the changing trend of confidence scores is continuously monitored, forming a closed-loop adaptive optimization mechanism for computing power. Closed-loop optimization is a key mechanism for the long-term stable operation of the system, achieving continuous improvement through feedback adjustment. The implementation process first translates the adjusted allocation scheme into specific execution instructions, applying them to the resource scheduling of the AI ​​inference engine; then, it continues to collect and analyze confidence data from subsequent frames to evaluate the effectiveness of the adjustment measures; finally, based on new monitoring results, it determines whether further adjustments are needed or whether the previous configuration needs to be restored. This feedback-based closed-loop control enables the system to continuously learn and adapt to changes in video content, optimizing resource utilization while ensuring analysis quality, forming a truly adaptive system. The core advantage of the closed-loop mechanism lies in its dynamic adaptability, capable of handling various changing factors such as video content, hardware performance, and environmental conditions, maintaining stable system performance under various conditions.

[0051] In another embodiment, the system further includes: The computing resource prediction module is used to build a predictive model of content complexity and computing power requirements based on historical video analysis data, and to estimate the computing power requirements of future frames in advance. This module uses a temporal prediction algorithm, combined with content characteristic analysis, to predict the complexity change trend of the video in the next few seconds or minutes, and adjust the system resource allocation strategy in advance. This proactive resource management mechanism significantly improves the system's adaptability to sudden scene changes and reduces latency and performance degradation caused by insufficient computing power.

[0052] The cross-device computing power collaboration module is used to achieve dynamic scheduling of computing resources across devices in a multi-device deployment environment. This module manages the overall computing resource pool of the device cluster through a distributed protocol, dynamically determining the device allocation scheme for analysis tasks based on the load status of each device and the characteristics of the video content. This collaboration mechanism effectively solves the problem of limited computing power of a single device in an edge computing environment, improving the scalability and robustness of the overall system.

[0053] The content-adaptive coding module dynamically adjusts video coding parameters based on the computing power allocation of each block. This module uses high-quality coding to preserve details in high-computing-power blocks, while employing high compression ratios to save bandwidth in low-computing-power blocks. This content-aware coding strategy optimizes resource utilization across the entire processing pipeline, achieving end-to-end optimization from video acquisition to analysis and processing.

[0054] This invention achieves intelligent dynamic allocation of computing resources in video analysis through texture gradient analysis, complexity matrix construction, feature point density identification, computing power requirement calculation, granular segmentation, and confidence monitoring. The differentiated computing power allocation method of this invention can accurately identify high-value content regions and rationally allocate limited computing power resources, improving overall system efficiency while ensuring the quality of key content analysis.

[0055] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0056] It should be noted that all formulas in this manual are calculated by removing dimensions and taking their numerical values. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0057] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. An intelligent video analysis system with adjustable AI computing power, characterized in that, include: The video frame acquisition module is used to acquire a continuous frame sequence of the video stream, divide each frame into multiple local blocks, and extract the texture gradient change rate between corresponding blocks of adjacent frames. The complexity analysis module is used to calculate the spatial complexity value and temporal complexity value of each block based on the texture gradient change rate, and to construct a content complexity matrix containing the spatial complexity value and the temporal complexity value. The feature point recognition module is used to extract feature points from the content complexity matrix, count the distribution density of feature points in each block, and identify high-density clustered areas and low-density sparse areas. The computing power demand calculation module is used to obtain the computing time data and analysis accuracy data of historical frames under different computing power configurations, and calculate the computing power demand coefficient of each block of the current frame based on the area ratio of the high-density clustered region and the low-density sparse region. The computing power partitioning and scheduling module is used to construct a computing power granularity partitioning model based on the computing power demand coefficient, divide the available computing power resources into multiple computing power granular units, and allocate a corresponding number of computing power granular units to each block. The execution control module is used to generate a computing power scheduling sequence for video analysis tasks based on the allocation results of the computing power granular units, and control the AI ​​inference engine to perform video content analysis according to the computing power scheduling sequence. The confidence monitoring module is used to collect and analyze the confidence values ​​of the output results in real time, calculate the decay gradient of the confidence values ​​within a preset time window, and identify the confidence fluctuation range caused by insufficient computing power. The adaptive optimization module is used to dynamically adjust the allocation ratio of the computing power granular units according to the fluctuation amplitude and duration of the confidence fluctuation range, so as to realize the adaptive optimization configuration of computing power resources during video analysis.

2. The AI ​​computing power adjustable intelligent video analysis system according to claim 1, characterized in that, The step of dividing each frame into multiple local blocks and extracting the texture gradient change rate between corresponding blocks in adjacent frames includes: An adaptive grid partitioning algorithm is used to divide the video frame into multiple local blocks of variable size, and the spatial coordinates of each block are recorded. For each local block, extract multi-directional texture gradients, calculate the weighted vector sum of the multi-directional texture gradients, and denote it as the block texture feature vector; Obtain the texture feature vectors of blocks with the same spatial coordinates in adjacent frames, calculate the Euclidean distance between them, and denote it as the texture difference between blocks; The time dimension derivative of the texture difference is calculated to obtain the rate of change of the texture difference with time, which is denoted as the texture gradient change rate. Outlier values ​​are removed from the texture gradient change rate, and valid gradient change rates within a preset percentile range are retained. The texture gradient change rate of each block is then updated.

3. The AI ​​computing power adjustable intelligent video analysis system according to claim 1, characterized in that, The step of calculating the spatial complexity and temporal complexity of each block based on the texture gradient change rate, and constructing a content complexity matrix containing the spatial complexity and temporal complexity values, includes: Spatial variance analysis is performed on the texture gradient change rate of all blocks in a single frame to calculate the gradient dispersion of each block and its neighboring blocks, which is denoted as the spatial complexity value. A temporal sequence of the texture gradient change rate of the same block in multiple consecutive frames is established, and the autocorrelation coefficient of the temporal sequence is calculated and denoted as the temporal correlation degree. The temporal complexity value of each block is calculated based on the reciprocal of the temporal correlation degree and the peak fluctuation range of the texture gradient change rate. The space complexity value and the temporal complexity value are weighted and fused together, and the fusion weight is adaptively determined according to the motion intensity of the video scene. The weighted and fused complexity values ​​are arranged according to the spatial coordinates of each block to generate the content complexity matrix, and the matrix elements of the content complexity matrix correspond one-to-one with the spatial location of the block.

4. The AI ​​computing power adjustable intelligent video analysis system according to claim 1, characterized in that, The calculation of the computing power requirement coefficient for each block in the current frame based on the area ratio of the high-density clustered region to the low-density sparse region includes: The pixel areas of the high-density clustered region and the low-density sparse region in the current frame are counted, and the ratio of their areas is calculated and denoted as the complexity distribution imbalance degree. Extract historical samples that are similar to the current complexity distribution imbalance from the computation time data of the historical frames, and establish a mapping relationship between the complexity distribution imbalance and the total computing power consumption. Based on the mapping relationship, predict the total computing power consumption of the current frame, and combine the complexity values ​​of each block in the content complexity matrix to calculate the weight of each block in the total computing power consumption. Obtain the analysis accuracy data of the historical frames under different computing power configurations, and identify the sensitivity curve between the accuracy loss rate and the computing power reduction rate; Based on the sensitivity curve, a precision guarantee threshold is set for each block, and the computing power requirement coefficient for each block is determined by multiplying the precision guarantee threshold by the proportion weight.

5. The AI ​​computing power adjustable intelligent video analysis system according to claim 1, characterized in that, The step of constructing a granular computing power segmentation model based on the computing power demand coefficient, dividing available computing power resources into multiple granular computing power units, and allocating a corresponding number of these granular computing power units to each block includes: Obtain the total available computing power resources of the AI ​​inference engine, and divide the total available computing power resources by the preset minimum computing power granularity benchmark value to obtain the total number of divisible computing power granular units. Based on the computing power demand coefficient of each block, calculate the number of theoretical computing power granular units required for each block, and normalize the number of theoretical computing power granular units. Determine whether the total number of computing power granular units meets the theoretical requirements of all blocks. If not, prioritize the number of theoretical computing power granular units. Based on the priority ranking, the computing power granular units are allocated to blocks within the high-density clustering area first, and the remaining computing power granular units are allocated to other blocks proportionally. A computing power allocation mapping table is generated, which contains the correspondence between each block and the number of computing power granular units allocated to it. The computing power allocation mapping table is the output result of the computing power granularity segmentation model.

6. The AI ​​computing power adjustable intelligent video analysis system according to claim 1, characterized in that, The confidence level of the real-time acquisition and analysis output results is calculated, and the decay gradient of the confidence level within a preset time window is determined to identify confidence level fluctuation ranges caused by insufficient computing power, including: The confidence score of each detected target is extracted from the video analysis results output by the AI ​​inference engine, and the confidence score is recorded in time series. Within the preset time window, the confidence score numerical sequence is subjected to a moving average filter to obtain a smoothed confidence score curve after filtering. Calculate the first derivative of the confidence level smoothing curve, denoted as the instantaneous decay rate of the confidence level, and count the percentage of time when the instantaneous decay rate of the confidence level is negative; When the time percentage exceeds a preset fluctuation threshold, the corresponding time period is marked as a candidate confidence fluctuation range. A retrospective analysis is performed on the computing power allocation records within the candidate confidence fluctuation range to determine whether the utilization rate of the computing power granular unit has reached saturation during this time period. The candidate confidence fluctuation range where the utilization rate reaches saturation is determined as the confidence fluctuation range caused by insufficient computing power, and the fluctuation amplitude and duration of this range are recorded.

7. The AI ​​computing power adjustable intelligent video analysis system according to claim 1, characterized in that, The step of dynamically adjusting the allocation ratio of computing power granular units based on the fluctuation amplitude and duration of the confidence level fluctuation range includes: Establish a two-dimensional evaluation coordinate system for the fluctuation amplitude and the duration, and map the confidence fluctuation range to the corresponding position in the evaluation coordinate system; The computing power adjustment strategy levels are divided according to the corresponding location, and the computing power adjustment strategy levels include mild adjustment, moderate adjustment and severe adjustment; For the mild adjustment level, the proportion of computing power particle units allocated in the high-density clustered area is increased by a factor of one of the preset benchmark values. For the moderate adjustment level, while increasing the computing power of the high-density clustered area, the computing power allocation of the low-density sparse area is reduced, and the adjustment range is the second multiple of the preset benchmark value. For severe adjustment levels, the construction process of the computing power granularity partitioning model is re-executed, and the minimum computing power granularity benchmark value is reduced to obtain a finer-grained computing power allocation capability. The adjusted computing power allocation scheme is applied to the analysis process of subsequent video frames, and the changing trend of the confidence value is continuously monitored to form a closed-loop computing power adaptive optimization mechanism.

Images