Dynamic frame rate regulation and control multi-target tracking method based on reinforcement learning
Through the dynamic frame rate control method based on reinforcement learning, the adaptability problem of the multi-target tracking method on resource-constrained devices is solved, the adaptive optimization of the frame rate is achieved, the stability and efficiency of the system are improved, and it is suitable for multi-target tracking tasks in edge computing and mobile terminals.
Patent Information
- Application Number
- CN202510913268.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-03
- Publication Date
- 2025-10-21
AI Technical Summary
Existing multi-target tracking methods lack adaptability on resource-constrained terminal devices, resulting in rigid frame rate strategies, serious resource waste, and difficulty in coping with dynamic changes in the number of targets and motion states, affecting tracking accuracy and system stability.
A dynamic frame rate control method based on reinforcement learning is adopted. By initializing the video tracking system, collecting video state features, and using the reinforcement learning strategy model to predict the frame rate adjustment action, the adaptive optimization of the frame rate is achieved by combining proportional frame rate control, frame rate smooth transition and cooling period mechanism.
It realizes real-time perception and dynamic adjustment of multi-target tracking on resource-constrained devices, improves the stability and robustness of the system, reduces computing resource consumption, adapts to the complexity changes of different scenarios, and ensures tracking accuracy and efficiency.
Smart Images

Figure CN120823235A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent video analysis and computer vision technology, and in particular to a multi-target tracking method with dynamic frame rate control based on reinforcement learning. Background Art
[0002] In video multi-object tracking (MOT), maintaining the identities and trajectory coherence of multiple moving targets across consecutive frames typically requires high-frame-rate video input, combined with computationally complex target detection and data association algorithms. This approach can achieve high tracking accuracy in scenarios with densely populated targets or intense motion, but it places high demands on system computing resources, data bandwidth, and power management, making it particularly unsuitable for deployment on resource-constrained, real-world devices.
[0003] In low-power, high-response applications such as edge computing nodes, mobile devices, or embedded platforms, continuously processing the entire video sequence at full frame rate often leads to processing bottlenecks. To alleviate this problem, some research has proposed strategies such as static frame rate downsampling, fixed-ratio frame skipping, and rule-based frame rate switching to reduce the video data processing burden.
[0004] However, traditional heuristic or static rule-based frame rate adjustment methods often lack the ability to perceive dynamic changes in the scene and are unable to perform real-time adaptive optimization based on factors such as the number of targets, motion characteristics, or scene complexity. In practical applications, when the number of targets increases suddenly, their motion states change dramatically, or the lighting environment suddenly changes, fixed frame rate solutions often fail to respond in a timely manner, easily leading to serious tracking performance degradation issues such as identity switching and trajectory loss, which in turn affects the stability and practicality of the system.
[0005] Therefore, there is an urgent need for a multi-target tracking method with scene perception capability and the ability to dynamically adjust the processing frame rate according to the current video status, so as to effectively reduce computing resource consumption while ensuring tracking accuracy, improve the system's real-time processing efficiency and cross-scene adaptability, and thus meet various practical needs such as intelligent monitoring, traffic analysis, and robot navigation. Summary of the Invention
[0006] The purpose of the present invention is to provide a multi-target tracking method with dynamic frame rate control based on reinforcement learning, so as to solve the problems of lack of adaptability, rigid frame rate strategy, and serious waste of resources in existing methods.
[0007] To achieve the above object, the present invention proposes the following technical solutions:
[0008] The method comprises the following steps:
[0009] S1: Initialize the video tracking system, set the original frame rate, frame rate candidate set and fixed time window parameters;
[0010] S2: Perform multi-target detection and tracking tasks in each time window, collect and extract video state features in the current window;
[0011] S3: At the end of the time window, the extracted state features are constructed into a state vector and input into the reinforcement learning policy model to predict the frame rate adjustment action for the next time window;
[0012] S4: Decide whether to adjust the frame rate based on the predicted action and its confidence, and update the frames that need to be sampled and processed in the next window accordingly;
[0013] S5: Evaluate the performance of the tracking results of the current window and calculate the reward value based on the comparison between the current window and the reference performance. This reward value is used to generate feedback indicators or log information in the inference phase and to update the reinforcement learning policy model in the training phase.
[0014] S6: Slide the time window backward and repeat steps S2 to S5 until all video sequences are processed.
[0015] Optionally, the state vector constructed in step S2 may include but is not limited to: target average speed, speed standard deviation, target number, ID switching times, current frame rate, system CPU load, memory usage and other multi-dimensional features.
[0016] In a preferred embodiment of the present invention, the frame rate adjustment action is an item in a discrete set, which is used to indicate that the frame rate is increased, decreased, or remains unchanged.
[0017] The reward function is not limited to a fixed formula. It can be dynamically compared with the reference score under the original frame rate processing based on weighted performance indicators such as MOTA (Multi-Object Tracking Accuracy) and IDF1 (Identity Preserving Precision) according to actual application requirements to build a frame rate-performance joint optimization strategy, guiding the policy model to converge towards a better efficiency-accuracy balance point.
[0018] In addition, the method can also integrate the following auxiliary control mechanisms to enhance the stability of the strategy and the practicality of the system:
[0019] Proportional frame rate control mechanism: Set the frame rate option to a set of proportional values of the original frame rate, such as [0.4, 0.6, 0.8, 1.0], to improve adaptability and versatility;
[0020] Frame rate smooth transition mechanism: During the frame rate adjustment process, a step-by-step increase or decrease method is adopted to prevent system instability caused by sudden changes;
[0021] Cooldown mechanism: Sets the minimum execution interval for frame rate adjustments to prevent excessive frame rate changes from interfering with system performance.
[0022] Therefore, the present invention adopts the above-mentioned dynamic frame rate control method based on reinforcement learning strategy, which has the following beneficial effects:
[0023] (1) This invention can perceive the motion characteristics and behavior of targets in video sequences in real time, including multi-dimensional dynamic information such as target speed, number, and trajectory change trends. It can also intelligently adjust the processing frame rate of subsequent windows based on the current system operating load and frame rate settings. This mechanism achieves a dynamic balance between tracking accuracy and computing resource consumption, and is particularly suitable for edge computing scenarios where resources are limited but high processing efficiency is required.
[0024] (2) This invention replaces the traditional static threshold rule with a reinforcement learning strategy network. Through a guided reward mechanism, the decision-making strategy is continuously and adaptively adjusted during training, thereby achieving automatic optimization of the frame rate control strategy. This approach not only has stronger generalization capabilities but also can adapt to dynamic changes in target density, motion intensity, and visual complexity in different scenarios, thereby improving the overall stability and robustness of the system.
[0025] (3) This invention uses a fixed-time window sliding processing mechanism, so that each decision is based on sufficient historical observation data for analysis and judgment, avoiding system oscillation problems caused by frequent frame rate switching. By combining window-level decision-making with cooling period control and transition mechanisms, the continuity and controllability of the frame rate adjustment process are effectively improved, ensuring the stable execution of the tracking task.
[0026] (4) The method of the present invention has a clear structure, reasonable module division, and good system compatibility and scalability. The frame rate control module can be independently integrated into the existing multi-target tracking framework without modifying the underlying tracking algorithm logic, and supports collaborative operation with mainstream detectors and trackers. This method is particularly suitable for application scenarios with high requirements for real-time performance and energy efficiency, such as intelligent monitoring, traffic video analysis, mobile terminal vision systems, etc., and has broad practical application prospects.
[0027] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 A flowchart of a dynamic frame rate control method based on reinforcement learning provided in an embodiment of the present invention;
[0029] Figure 2 This is a flowchart of the reward calculation and reinforcement learning strategy update module provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0030] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.
[0031] Unless otherwise defined, technical or scientific terms used in the present invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.
[0032] The words “include” or “comprising” and similar words used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms “inside”, “outside”, “upper”, “lower”, etc. is based on the orientation or position relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly stipulated and limited, the terms such as “attachment” should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral whole; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.
[0033] like Figure 1 、 Figure 2 As shown, an embodiment of the present invention discloses a multi-target tracking method with dynamic frame rate control based on reinforcement learning. The method is characterized in that the frame rate is dynamically determined by a reinforcement learning strategy to achieve an adaptive balance between tracking performance and computing resources. The method mainly includes the following steps:
[0034] S1: Initialize the video multi-target tracking system, set the original frame rate, frame rate candidate set and fixed time window parameters. The fixed time window is a time period W with a length of T (unit: seconds) k , the total number of frames in the window is based on the original frame rate F o Calculated as T×F o , the sampling frame set is recorded as:
[0035]
[0036] where n k The number of sampling frames corresponding to the current frame rate.
[0037] S2: In each fixed time window W k Within the window, the system performs multi-target detection and tracking tasks on the input video frame sequence and collects state features during the window.
[0038] Preferably, the tracking results based on the sampling frame extract statistical features and construct the state vector Its structure is as follows:
[0039] s k =[μ v ,σ v , N t ,FPS k ,ρ CPU ,ρ mem ] (2)
[0040] where μ v is the average speed of the target in the window, σ v is the standard deviation of the target speed in the window, N t is the number of detected targets, FPS k is the frame rate of the current window, ρ CPU ,ρ mem Indicates the usage of system resources CPU and memory.
[0041] Preferably, the number of sampling frames and the sampling index are determined according to the current frame rate, and the frames to be processed are selected from the complete window using a linear uniform sampling method to reduce redundant calculations.
[0042] S3: At the end of the time window, the constructed state vector is input into the reinforcement learning policy model to predict the frame rate adjustment strategy to be adopted in the next time window.
[0043] Preferably, the action space of the frame rate control strategy is a discrete set:
[0044]
[0045] Among them, -1 means reducing the frame rate by one level, 0 means maintaining the current frame rate, and +1 means increasing the frame rate by one level. The reinforcement learning policy network is based on the state vector s of the current window. k Output action probability distribution:
[0046] π θ (a k ∣s k ) (4)
[0047] Among them, π θ represents the probability distribution function after parameterization of the strategy model, a k Adjust the action for the candidate frame rate. The system determines whether to execute the action based on the maximum probability action and its confidence. If the following conditions are met:
[0048]
[0049] Then execute the action Otherwise, the frame rate of the previous window is maintained unchanged. is the action with the highest probability, and τ is the action confidence threshold. This mechanism introduces confidence control to effectively avoid highly uncertain decisions in the model output and enhance the stability of frame rate switching.
[0050] S4: Determine whether to adjust the current frame rate based on the predicted action and its confidence level.
[0051] Furthermore, the system performs frame rate adjustment only when the confidence level corresponding to the predicted action is higher than a preset threshold (eg, 0.7), otherwise it maintains the original frame rate to avoid system instability caused by frequent frame rate changes.
[0052] Assume the frame rate candidate set is:
[0053] FPS k ∈{r1,r2,…,r m},r i ∈(0,1] (6)
[0054] Action Corresponding to the frame rate index change, the update rules are:
[0055]
[0056] where i k Indicates the index position of the current frame rate. The currently used proportional frame rate value, such as [0.4, 0.6, 0.8, 1.0], is used to enhance the versatility and adaptability of the strategy. FPS k+1 The frame rate of the next window.
[0057] S5: At the end of each processing window, the tracking performance under the current window is evaluated.
[0058] Preferably, the tracking performance of the current time window is calculated by a weighted index:
[0059] S w =λ·MOTA k +(1-λ)·IDF1 k ,λ∈[0,1] (8)
[0060] Among them, MOTA k and IDF1 k are the multi-target tracking accuracy and identity preservation accuracy of the k-th window, respectively, and λ is the weight coefficient (for example, λ = 0.5 indicates that the two indicators are equally weighted).
[0061] Assume that the benchmark score of the corresponding window under the reference frame rate is S r , then the reward function R k is defined as:
[0062]
[0063] Among them, FPS k is the current frame rate, F o is the original (highest) frame rate, δ is the performance tolerance threshold (e.g., δ = 0.97), and β is a fixed penalty value (e.g., β = 0.5). If the weighted score at the current frame rate is greater than the tolerable threshold, such as 97% of the original (highest) frame rate, the reward value is Otherwise, a fixed penalty value β is imposed. This reward design takes into account both accuracy preservation and resource conservation, encouraging the model to use a lower frame rate within the accuracy range, thereby improving the overall efficiency of the system.
[0064] The reward function is used to update the reinforcement learning strategy model during the training phase. Figure 2 As shown in the figure, during the training phase, the system calculates the current tracking score based on the multi-target tracking results in the sampling window and compares it with the reference score at the original frame rate to generate a reward value. This reward serves as an important feedback signal for updating the reinforcement learning policy model, guiding policy optimization. The reward function is designed to consider the trade-off between maintaining accuracy and reducing frame rate. While ensuring that performance fluctuations do not exceed the tolerance threshold, it guides the policy to learn the optimal behavior at low frame rates. Ultimately, the policy model is iteratively updated through the reinforcement learning algorithm to form a stable frame rate control strategy.
[0065] S6: Slide the time window backward, update the window boundary, and repeat steps S2 to S5 until the entire video sequence is processed, thereby achieving dynamic frame rate control and continuous optimization tracking of the video data.
[0066] In a preferred embodiment of the present invention, the multi-target tracking module adopts the ByteTrack algorithm, which has the characteristics of high detection accuracy and stable ID, and is suitable for long sequence processing at low frame rates. The reinforcement learning strategy model is trained using the MaskablePPO algorithm. This algorithm introduces an action mask mechanism based on the traditional Proximal Policy Optimization (PPO) framework, which can effectively avoid learning redundant strategies in invalid action spaces, thereby improving training efficiency and strategy convergence.
[0067] During the training process, the system first uses the ByteTrack algorithm to process the complete video sequence at the original frame rate, and obtains the reference tracking score corresponding to each time window as the performance upper limit benchmark. Subsequently, the video frames in the sampling window are set at different frame rates, and the tracking module is run to calculate the weighted tracking indicators (such as the average of MOTA and IDF1), which are then compared with the reference scores to construct the reward function. The design of the reward function follows the following rules: If the weighted score at the current frame rate is greater than or equal to 97% of the reference score, the reward is set to Among them F t is the current frame rate, F o is the original frame rate; otherwise, a fixed penalty value (such as -0.5) is given to guide the strategy to avoid inefficient frame rate decisions.
[0068] The system's input state vector includes features such as target average speed, speed standard deviation, number of targets, current frame rate, number of ID switches, system load, and memory usage. The frame rate action space is a discrete set (e.g., gear changes {-1, 0, +1} on the relative frame rate scale [0.4, 0.6, 0.8, 1.0]). A masking mechanism is used to remove illegal or unexecutable actions, such as masking the "down" action when the frame rate is currently at its lowest, to ensure the legitimacy and stability of the policy output.
[0069] After training is complete, the reinforcement learning strategy model is used to make frame rate control decisions during the inference phase. This model does not need to be updated during actual deployment and only participates in the inference process as a decision-making module.
[0070] In practical applications, each of these modules can be deployed on edge computing devices, video analysis servers, or mobile terminal platforms, supporting flexible configuration of frame rate settings, window lengths, state feature dimensions, and model parameters. This method is adaptable to different types of video input sources and target behavior patterns, demonstrating excellent scalability and deployment versatility.
[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. A multi-target tracking method with dynamic frame rate control based on reinforcement learning, characterized in that: The method comprises the following steps: S1: Initialize the video tracking system, set the initial frame rate, frame rate candidate set and fixed time window parameters; S2: Perform multi-target detection and tracking in each fixed time window and collect video state features in the current window; S3: At the end of the window, a state vector is constructed based on the collected state features and input into the reinforcement learning strategy model to predict the frame rate adjustment action for the next window; S4: Determine whether to adjust the frame rate based on the action output by the model and its confidence, and control the frames processed in the next window accordingly; S5: Evaluate the performance of the tracking results of the current window and calculate the reward value based on the comparison between the current window and the reference performance. This reward value is used to generate feedback indicators or log information in the inference phase and to update the reinforcement learning policy model in the training phase. S6: Slide the time window range and repeat steps S2 to S5 until the entire video sequence is processed.
2. The method according to claim 1, characterized in that The step S2 specifically includes the following sub-steps: S2.1: Determine the number of frame samples and index position according to the current frame rate, and sample the video frames in the window; S2.2: Perform multi-target tracking on the sampling frame and obtain tracking results; S2.3: Extracting target state features based on the tracking results, including but not limited to average speed, speed fluctuation, number of targets, etc.; S2.4: Combine the current frame rate with the state feature to construct a state vector as the policy model input.
3. The method according to claim 2, characterized in that The step S2.3 specifically includes the following sub-steps: S2.3.1: Calculate the average speed and standard deviation of the speed of the target within the window; S2.3.2: Count the average number of targets detected within the window; S2.3.3: Record resource status parameters such as the current frame rate.
4. The method according to claim 1, wherein The frame rate adjustment action in step S3 is one of a finite discrete action set, used to indicate whether the frame rate is increased, decreased, or remains unchanged.
5. The method according to claim 1, wherein In step S4, the frame rate adjustment is performed only when the prediction confidence corresponding to the action is greater than a preset threshold, otherwise the current frame rate is maintained.
6. The method according to claim 1, characterized in that The reward value in step S5 is calculated based on the relative difference between the tracking performance in the current window and the preset reference performance to evaluate the pros and cons of the current frame rate strategy and is used to update the strategy of the reinforcement learning model.
7. The method according to claim 1, characterized in that The fixed time window is of fixed duration T, and the number of frames in the window is based on the original frame rate F o Calculated as T×F o .
8. The method according to claim 1, characterized in that Further including the following mechanisms: Proportional frame rate control mechanism: The frame rate option is the original frame rate F o Multiple ratio values are used to adapt to different video source frame rates; Smooth transition mechanism: Use a gradual increase or decrease method when performing frame rate adjustments to reduce system instability caused by sudden changes; Cooldown mechanism: Set the minimum interval period for frame rate adjustment to prevent excessively frequent changes.
Citation Information
Cited By
Low-power-consumption dynamic adjustment method and system based on multi-objective optimization decision
CN121541768A
