NVR and IPC management method for multi-area dynamic video stream switching and storage
By acquiring real-time video streams from multiple IPCs, performing frame extraction and preprocessing, identifying moving targets, calculating activity density, and dynamically adjusting bandwidth and storage strategies, the problem of resource waste in multi-area video surveillance systems is solved, achieving efficient bandwidth management and storage optimization.
Patent Information
- Application Number
- CN202411232594.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-09-04
AI Technical Summary
In multi-area video surveillance systems, traditional NVR and IPC management methods cannot effectively adjust the transmission bandwidth of video streams dynamically according to the activity density of the monitored area, resulting in wasted network resources or the inability to transmit high-definition images in high-activity areas in a timely manner.
By acquiring real-time video streams from multiple IPCs, performing frame extraction and preprocessing, identifying moving targets, calculating activity density, dynamically adjusting the transmission bandwidth of the IPC video streams and the NVR's storage strategy, and optimizing bandwidth management and storage strategy through a real-time feedback mechanism.
It achieves adaptive adjustment based on the activity density of the monitored area, rationally allocates storage resources, improves the operating efficiency and response speed of the monitoring system, and ensures the monitoring quality of high-activity areas.
Smart Images

Figure CN118741023B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of video network application technology, and more specifically, to an NVR and IPC management method for multi-region dynamic video stream switching and storage. Background Art
[0002] In modern video surveillance systems, with the increase of urbanization and security needs, multi-area monitoring has become an indispensable part. Especially in the fields of urban security, traffic monitoring, intelligent buildings, etc., a large number of video surveillance devices (IPC, network cameras) and network video recorders (NVR) are widely deployed. These devices are interconnected through the network to realize the real-time acquisition, storage and management of video data in multiple areas. However, with the increase in the number of monitoring areas and the complexity of monitoring needs, the traditional NVR and IPC management methods face some technical challenges; that is, in a multi-area video surveillance system, the activity density of different monitoring areas changes frequently, and some areas may have a high activity density during a specific period of time, while other areas are relatively quiet. If the video streams of each area are transmitted with the same bandwidth, this will lead to a waste of network resources, or high-definition images cannot be transmitted in time in areas with high activity density, affecting the monitoring effect. Therefore, how to dynamically adjust the transmission bandwidth of the IPC video stream according to the activity density of the monitoring area has become an important technical issue in the management of video surveillance systems. Summary of the invention
[0003] The present invention provides an NVR and IPC management method for multi-region dynamic video stream switching and storage, and intends to solve the problem of dynamically adjusting the transmission bandwidth of the IPC video stream based on the activity density of the monitored area.
[0004] The NVR and IPC management method for multi-area dynamic video stream switching and storage includes the following steps:
[0005] Step 1: Obtain real-time video streams from multiple IPCs to obtain video frame data, and perform frame extraction and preprocessing on the obtained video frame data to obtain a continuous image frame sequence;
[0006] Step 2: Perform activity detection based on a continuous image frame sequence to identify the moving target in the video and obtain relevant information about the moving target;
[0007] Step 3: Based on the relevant information of the moving target, the activity density of each monitoring area is calculated to obtain the activity density value of each monitoring area;
[0008] Step 4: Dynamically adjust the transmission bandwidth of the IPC video stream and the storage strategy of the NVR according to the changes in activity density;
[0009] Step 5: Provide real-time feedback on the effects of bandwidth and storage policy adjustments, analyze the gap between actual effects and expectations, obtain feedback data, and use optimization algorithms based on the feedback data to improve the accuracy of activity detection, bandwidth management, and storage policies.
[0010] The present invention obtains real-time video streams from multiple IPCs, and obtains a continuous image frame sequence through frame extraction and preprocessing, performs activity detection based on the continuous image frames, identifies and extracts the moving target information in the video, and calculates the activity density value of each monitoring area by analyzing the moving target information. After obtaining the activity density, the system dynamically adjusts the transmission bandwidth of the IPC video stream and the storage of the NVR according to the activity density of each area, ensuring that the storage resources in the high activity density area are reasonably allocated; finally, the system analyzes the actual effect of the bandwidth and storage strategy after adjustment through a real-time feedback mechanism, compares it with the expected effect, and uses an optimization algorithm based on the feedback data to continuously improve the accuracy of activity detection, bandwidth management and storage strategy, thereby realizing adaptive adjustment to dynamic changes.
[0011] Preferably, step 2 comprises the following steps:
[0012] Step 2.1: Calculate the difference between the current frame and the background model based on the background difference method to obtain a differential image;
[0013] Step 2.2: Adaptively binarize the difference image to convert pixels into binary format, distinguish between static and moving areas, and use an adaptive threshold method to ensure accurate identification of moving areas under different lighting conditions;
[0014] Step 2.3: In the binary image, use the Connected Components detection algorithm to identify independent moving target areas;
[0015] Step 2.4: For each identified moving target area, calculate the position of the centroid in the moving target area;
[0016] Calculate each moving target region R based on the pixel count of the moving target region class i area;
[0017] The speed of the target is calculated by combining Kalman filtering with the joint probability data association algorithm.
[0018] Preferably, the position of the centroid in the moving target area is calculated using a weighted method:
[0019]
[0020] Where: w(x,y) represents the weight of the pixel (x,y); (x i ,y i) represents the moving target area r i The centroid coordinates of the pixel point in the image; x and y represent the horizontal and vertical coordinates of the pixel point in the image respectively;
[0021] The weight of the pixel is calculated based on brightness and foreground probability:
[0022] w(x,y)=α·I t (x,y)+β·P f (x,y);
[0023] Where: I t (x, y) represents the brightness of the pixel (x, y) in the current frame t; P f (x, y) represents the probability that the pixel (x, y) belongs to the foreground; α represents the weight coefficient of the brightness term; β represents the weight coefficient of the foreground probability term.
[0024] Preferably, the steps of calculating the speed of the target are as follows:
[0025] Target position prediction and update: Use the Kalman filter to predict and update the position of each target i:
[0026] Target location prediction:
[0027]
[0028] Where: and are the predicted values of the centroid position of target i at time t; x i (t-1) and y i (t-1) is the actual center of mass position of target i at time t-1; v x,i (t-1) and v y,i (t-1) are the velocity components of target i at time t-1;
[0029] Location Updates:
[0030]
[0031] Where: and is the observed value of target i at time t; K x and K y is the Kalman gain;
[0032] Target velocity calculation: The velocity of each target is estimated based on the change in the center of mass position between consecutive frames:
[0033]
[0034] Where: v i(t) represents the speed of target i at time t; Δt is the frame time interval;
[0035] The joint probability data association algorithm calculates the joint probability between each observation and all observations to determine the best target-observation pair:
[0036]
[0037] Where: P ij represents the association probability between target i and observation j; S -1 represents the inverse matrix of the covariance matrix; n represents the total number of all targets; represents the exponential term of the Gaussian distribution function; Denotes the difference vector d ij The transpose of ij The representation is as follows:
[0038]
[0039] Where: The horizontal axis represents the observation value j at time t; The ordinate represents the observation value j at time t;
[0040] After target association, calculate the final velocity of each target:
[0041]
[0042] Where: v i (t)' represents the final velocity of target i; v ij (t) represents the velocity vector of target i and observation j; N represents the total number of all observations at time t.
[0043] Preferably, step 3 comprises the following steps:
[0044] Calculate the moving target's contribution to the activity density: map the moving target's position to the corresponding monitoring area, determine the monitoring area described by the moving target, traverse all the moving targets belonging to each monitoring area, and calculate the moving target's contribution to the activity density in the area:
[0045] D j =∑ i∈j (W1·A i +W2·v i ');
[0046] Where: D j represents the activity density of monitoring area j; A i represents the area of moving target i; v i' represents the speed of the moving target; W1 and W2 represent weight coefficients;
[0047] Normalize the activity density of each region by its maximum value:
[0048]
[0049] Where: D' j represents the normalized activity density value of the jth monitoring area; max(D j ) represents the maximum activity density value in all monitoring areas; ∈ represents a small value to prevent division by zero;
[0050] When the D' j Less than the preset minimum activity density threshold D threshold When , it is considered that there is no activity in the area and the density value is set to 0; finally, the activity density value D' of each monitoring area j is obtained j .
[0051] Preferably, step 4 comprises the following steps:
[0052] Calculate bandwidth and storage allocation for each monitoring area based on activity density values:
[0053]
[0054] Where: B min Indicates the minimum bandwidth of each monitoring area; D' j represents the normalized activity density value of the jth monitoring area; D' k represents the normalized activity density value of the kth monitoring area; B total Indicates the total bandwidth; S total Indicates the total storage space of the NVR; S min represents the minimum storage space of each monitoring area; λ1 and λ2 represent Lagrange multipliers; M represents the number of monitoring areas; B j represents the bandwidth allocation of monitoring area j; S j Indicates the storage allocation of the monitoring area;
[0055] Calculate the objective function based on the calculated bandwidth and storage allocation:
[0056]
[0057] Where: w1 and w2 represent weight coefficients;
[0058] When calculating the objective function, the following constraints need to be met:
[0059]
[0060] B j ≥B min ,S j ≥S min ;
[0061] Adjustment based on the objective function and constraints: If the objective function does not reach the desired optimal value or the constraints are not met, adjust the parameters λ1, λ2 or weight coefficients w1, w2, and recalculate the bandwidth and storage allocation until the objective function reaches the optimal value and all constraints are met.
[0062] Preferably, the specific steps of step 5 are as follows:
[0063] Initialization: Use the λ1, λ2, w1, and w2 calculated in the previous step as the initial values;
[0064] Minimize deviation: Fine-tune the parameters λ1, λ2, w1, and w2 by gradient descent. The goal is to minimize the objective function so that the deviation rate is close to 0. The objective function is:
[0065]
[0066] Where: Indicates the actual bandwidth usage and expected bandwidth of each area The relative deviation between Indicates the actual storage used and expected storage in each area The relative deviation between j Represents the storage redundancy coefficient; Minimze represents minimizing the value of the corresponding objective function;
[0067] In each round of iteration, the bandwidth and storage allocation values of each monitoring area are recalculated, and feedback data is collected and processed again until the objective function reaches the optimal value or meets the convergence conditions.
[0068] Preferably, in step 4, the bandwidth and storage allocation of the monitoring area are performed using the following objective function:
[0069]
[0070] Where: α and β represent adjustment parameters; Indicates the actual bandwidth usage and expected bandwidth of each area The relative deviation between Indicates the actual storage used and expected storage in each area The relative deviation between j represents the storage redundancy coefficient; B total Indicates the total bandwidth; S totalIndicates the total storage space of NVR; B j represents the bandwidth allocation of monitoring area j; S j represents the storage allocation of monitoring area j; w1 and w2 represent weight coefficients; D' j represents the normalized activity density value of the jth monitoring area; Minimize represents minimizing the value of the corresponding objective function; ΔB j Indicates the actual bandwidth usage and expected bandwidth of monitoring area j The difference between j Indicates the actual storage usage and expected storage capacity of monitoring area j The difference between .
[0071] The beneficial effects of the present invention include:
[0072] The present invention obtains real-time video streams from multiple IPCs, and obtains a continuous image frame sequence through frame extraction and preprocessing, performs activity detection based on the continuous image frames, identifies and extracts the moving target information in the video, and calculates the activity density value of each monitoring area by analyzing the moving target information. After obtaining the activity density, the system dynamically adjusts the transmission bandwidth of the IPC video stream and the storage of the NVR according to the activity density of each area, ensuring that the storage resources in the high activity density area are reasonably allocated; finally, the system analyzes the actual effect of the bandwidth and storage strategy after adjustment through a real-time feedback mechanism, compares it with the expected effect, and uses an optimization algorithm based on the feedback data to continuously improve the accuracy of activity detection, bandwidth management and storage strategy, thereby realizing adaptive adjustment to dynamic changes. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0074] Figure 1 An overall step block diagram provided for an embodiment of the present invention.
[0075] Figure 2 A specific step block diagram of step 2 provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0076] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0077] Example 1, see Figure 1 As shown, the NVR and IPC management method for multi-area dynamic video stream switching and storage includes the following steps:
[0078] Step 1: Obtain real-time video streams from multiple IPCs to obtain video frame data, and perform frame extraction and preprocessing on the obtained video frame data to obtain a continuous image frame sequence;
[0079] Exemplary technical solutions are as follows:
[0080] Get the video stream from IPC and extract a continuous sequence of image frames; use a Gaussian mixture model to distinguish the foreground and background and adapt to lighting changes and dynamic backgrounds; use adaptive histogram equalization and median filtering to enhance contrast and eliminate noise, respectively.
[0081] Step 2: Perform activity detection based on a continuous image frame sequence to identify the moving target in the video and obtain relevant information about the moving target;
[0082] See also Figure 2 As shown, step 2 includes the following steps:
[0083] Step 2.1: Calculate the current frame I based on the background difference method t The difference between (x, y) and the background model B(x, y) is used to obtain the difference image D t (x,y);
[0084] D t (x,y)=|I t (x,y(-B(x,y)|;
[0085] Where: D t (x, y) represents the difference between each pixel in the image and the background model, which is used to identify the moving area in the image.
[0086] Step 2.2: Difference image D t (x, y) is subjected to adaptive binarization processing to convert pixels into binary format, distinguishing between static and moving areas, and using adaptive thresholding to ensure accurate identification of moving areas under different lighting conditions;
[0087]
[0088] Where: T(x,y) is the threshold adaptively adjusted according to the local characteristics of the image; B t (x, y) represents the binary image matrix, which is used to indicate whether the pixel point (x, y) belongs to the motion area (value is 1) or the static area (value is 0);
[0089] The steps for determining T(x,y) are as follows:
[0090] T(x,y)=μ(x,y)+k·σ(x,y);
[0091]
[0092] Where: W(x,y) represents the number of pixels in the window; I t (u,v) represents the brightness value of the pixel (u,v) in the current frame; μ(x,y) represents the mean value of the pixels in the window; σ(x,y) represents the standard deviation of the pixels in the window; k represents the control factor, which is used to adjust the sensitivity of the threshold;
[0093] In this embodiment, by calculating the mean and standard deviation within the local window, the threshold is adaptively adjusted according to the local characteristics of the image, so that the threshold can dynamically adapt to different lighting conditions; ensuring that in scenes with uneven lighting or changing lighting conditions, the moving area can still be accurately identified, avoiding the problem of false detection or missed detection of traditional fixed threshold methods in such scenes.
[0094] Step 2.3: In the binary image, use the Connected Components detection algorithm to identify independent moving target areas;
[0095] The specific steps are as follows:
[0096] Mark initialization: Initialize a mark matrix L(x,y) with the binary image B t (x, y) have the same dimensions, and the initial value of all elements in the matrix is 0, indicating pixels that have not been marked;
[0097] Scan the binary image: Binarized image B t (x,y) scans pixel by pixel, and when pixel B is detected t When (x, y) = 1 and L(x, y) = 0, it means that the pixel belongs to a newly discovered moving target area and the area expansion begins;
[0098] Region expansion: Starting from the starting pixel (x, y), the adjacent pixels belonging to the same motion region are marked through four-connectivity or eight-connectivity, and expansion is achieved using recursion or stacks to mark all pixels in the current region and assign a unique region mark value L. i ;
[0099] Region marking is completed: After all motion regions are marked, each non-zero value in the marking matrix L(x, y) represents an independent motion target region, and the marking value L i represents the i-th region;
[0100] Generate moving target area: According to the label matrix L(x,y), extract all independent moving target areas R i , record the boundaries, area and other information of each area.
[0101] In this embodiment, the Connected Components detection algorithm can accurately separate different moving targets to avoid target overlap or confusion.
[0102] Step 2.4: For each identified moving target area, calculate the position of the centroid in the moving target area;
[0103] The position of the center of mass in the moving target area is calculated using a weighted method:
[0104]
[0105] Where: w(x,y) represents the weight of the pixel (x,y); (x i ,y i ) represents the moving target area R i The centroid coordinates of the pixel point in the image; x and y represent the horizontal and vertical coordinates of the pixel point in the image respectively;
[0106] The weight of the pixel is calculated based on brightness and foreground probability:
[0107] w(x,y)=α·I t (x,y)+β·P f (x,y);
[0108] Where: I t (x, y) represents the brightness of the pixel (x, y) in the current frame t; P f (x, y) represents the probability that the pixel (x, y) belongs to the foreground; α represents the weight coefficient of the brightness term; β represents the weight coefficient of the foreground probability term.
[0109] In this embodiment, the centroid is calculated for each moving target area, so that the position of the moving target can be accurately located. The pixel brightness and foreground probability are combined and a weighted average method is adopted to make the calculation result more accurate. The position of the centroid can not only reflect the central position of the moving target, but also provide information about the target shape and distribution, which is helpful for subsequent target tracking and behavior analysis. Since some areas of some moving targets may have a high similarity with the background, the accuracy of the target centroid calculation can be effectively improved by adjusting the weights of the brightness and foreground probability, thereby avoiding the influence of background interference on target positioning.
[0110] Calculate each moving target region R based on the pixel count of the moving target region class i Area A i =|Ri |;
[0111] The speed of the target is calculated by combining Kalman filtering with the joint probability data association algorithm.
[0112] The steps to calculate the target's velocity are as follows:
[0113] Target position prediction and update: Use the Kalman filter to predict and update the position of each target i:
[0114] Target location prediction:
[0115]
[0116] Where: and are the predicted values of the centroid position of target i at time t; x i (t-1) and y i (t-1) is the actual center of mass position of target i at time t-1; v x,i (t-1) and v y,i (t-1) are the velocity components of target i at time t-1;
[0117] Location Updates:
[0118]
[0119] Where: and is the observed value of target i at time t; K x and K y is the Kalman gain;
[0120] Target velocity calculation: The velocity of each target is estimated based on the change in the center of mass position between consecutive frames:
[0121]
[0122] Where: v i (t) represents the speed of target i at time t; Δt is the frame time interval;
[0123] The joint probability data association algorithm calculates the joint probability between each observation and all observations to determine the best target-observation pair:
[0124]
[0125] Where: P ij represents the association probability between target i and observation j; S -1 represents the inverse matrix of the covariance matrix; n represents the total number of all targets; represents the exponential term of the Gaussian distribution function; Denotes the difference vector d ij The transpose of ij The representation is as follows:
[0126]
[0127] Where: The horizontal axis represents the observation value j at time t; The ordinate represents the observation value j at time t;
[0128] After target association, calculate the final velocity of each target:
[0129]
[0130] Where: v i (t)' represents the final velocity of target i; v ij (t) represents the velocity vector of target i and observation j; N represents the total number of all observations at time t.
[0131] In order to handle the interaction between multiple targets and avoid mismatching, the JPDA algorithm is used for target association, which can accurately estimate the speed of each target in multi-target application scenarios and ensure robustness and accuracy in complex scenarios.
[0132] This embodiment combines Kalman filtering with the joint probability data association algorithm to form an efficient and robust multi-target detection and tracking system. The system can not only accurately detect moving targets in complex environments, but also provide accurate target position and speed information, ensuring reliable operation under complex conditions such as noise, occlusion, and dense targets.
[0133] Step 3: Based on the relevant information of the moving target, the activity density of each monitoring area is calculated to obtain the activity density value of each monitoring area;
[0134] The step 3 comprises the following steps:
[0135] Calculate the moving target's contribution to the activity density: map the moving target's position to the corresponding monitoring area, determine the monitoring area described by the moving target, traverse all the moving targets belonging to each monitoring area, and calculate the moving target's contribution to the activity density in the area:
[0136] D j =∑ i∈j (W1·A i +W2·v i ');
[0137] Where: D jrepresents the activity density of monitoring area j; A i represents the area of moving target i; v i ' represents the speed of the moving target; W1 and W2 represent weight coefficients;
[0138] Normalize the activity density of each region by its maximum value:
[0139]
[0140] Where: D' j represents the normalized activity density value of the jth monitoring area; max(D i ) represents the maximum activity density value in all monitoring areas; ∈ represents a small value to prevent division by zero;
[0141] When the D' j Less than the preset minimum activity density threshold D threshold When , it is considered that there is no activity in the area and the density value is set to 0; finally, the activity density value D' of each monitoring area j is obtained j .
[0142] In this embodiment, by taking the area and speed of the moving target as the key factors for calculating the activity density and weighting and summing their contributions according to the weight coefficients W1 and W2, the method can effectively capture the actual movement in the monitoring area. The activity density is normalized with the maximum value, and the effective monitoring area is screened out according to the preset threshold. This process ensures the accuracy and comparability of the activity density, especially in the scene with multiple targets or complex motion trajectories.
[0143] Step 4: Dynamically adjust the transmission bandwidth of the IPC video stream and the storage strategy of the NVR according to the changes in activity density;
[0144] The step 4 comprises the following steps:
[0145] Calculate bandwidth and storage allocation for each monitoring area based on activity density values:
[0146]
[0147] Where: B min Indicates the minimum bandwidth of each monitoring area; D' j represents the normalized activity density value of the jth monitoring area; D' k represents the normalized activity density value of the kth monitoring area; B total Indicates the total bandwidth; S total Indicates the total storage space of the NVR; S minrepresents the minimum storage space of each monitoring area; λ1 and λ2 represent Lagrange multipliers; M represents the number of monitoring areas; B j represents the bandwidth allocation of monitoring area j; S j Indicates the storage allocation of the monitoring area;
[0148] Calculate the objective function based on the calculated bandwidth and storage allocation:
[0149]
[0150] Where: w1 and w2 represent weight coefficients;
[0151] When calculating the objective function, the following constraints need to be met:
[0152]
[0153] B j ≥B min ,S j ≥S min ;
[0154] Adjustment based on the objective function and constraints: If the objective function does not reach the desired optimal value or the constraints are not met, adjust the parameters λ1, λ2 or weight coefficients w1, w2, and recalculate the bandwidth and storage allocation until the objective function reaches the optimal value and all constraints are met.
[0155] In this embodiment, based on the calculated activity density, the IPC video stream transmission bandwidth and NVR storage strategy of the monitoring area are dynamically adjusted to make resource allocation more flexible and efficient. This method considers the weight of activity density, so that bandwidth and storage resources can be intelligently adjusted according to real-time needs to avoid resource waste or overload, thereby improving the operating efficiency and response speed of the entire monitoring system. For example, in an intelligent traffic monitoring system, the traffic flow of different roads and sections varies greatly. This solution can dynamically adjust the video stream bandwidth and storage strategy according to the actual traffic density, ensure the monitoring quality of high-flow sections, and reduce resource consumption in low-flow areas.
[0156] In this embodiment, by calculating the objective function and combining the adjustment of the Lagrange multipliers λ1 and λ2 and the weight coefficients w1 and w2, it is possible to make optimal allocation under the total resource limit, ensure that the resource allocation of each monitoring area meets the overall optimization goal of the system, and avoid system bottleneck problems caused by unreasonable resource allocation. At the same time, through the optimization adjustment of the objective function and the constraint conditions, the scheme ensures the robustness and stability of the system.
[0157] Step 5: Provide real-time feedback on the effects of bandwidth and storage policy adjustments, analyze the gap between actual effects and expectations, obtain feedback data, and use optimization algorithms based on the feedback data to improve the accuracy of activity detection, bandwidth management, and storage policies.
[0158] The specific steps of step 5 are as follows:
[0159] Initialization: Use the λ1, λ2, w1, and w2 calculated in the previous step as the initial values;
[0160] Minimize deviation: Fine-tune the parameters λ1, λ2, w1, and w2 by gradient descent. The goal is to minimize the objective function so that the deviation rate is close to 0. The objective function is:
[0161]
[0162] Where: Indicates the actual bandwidth usage and expected bandwidth of each area The relative deviation between Indicates the actual storage used and expected storage in each area The relative deviation between j Represents the storage redundancy coefficient; Minimize represents minimizing the value of the corresponding objective function;
[0163] In each round of iteration, the bandwidth and storage allocation values of each monitoring area are recalculated, and feedback data is collected and processed again until the objective function reaches the optimal value or meets the convergence conditions.
[0164] The present invention obtains real-time video streams from multiple IPCs, and obtains a continuous image frame sequence through frame extraction and preprocessing, performs activity detection based on the continuous image frames, identifies and extracts the moving target information in the video, and calculates the activity density value of each monitoring area by analyzing the moving target information. After obtaining the activity density, the system dynamically adjusts the transmission bandwidth of the IPC video stream and the storage of the NVR according to the activity density of each area, ensuring that the storage resources in the high activity density area are reasonably allocated; finally, the system analyzes the actual effect of the bandwidth and storage strategy after adjustment through a real-time feedback mechanism, compares it with the expected effect, and uses an optimization algorithm based on the feedback data to continuously improve the accuracy of activity detection, bandwidth management and storage strategy, thereby realizing adaptive adjustment to dynamic changes.
[0165] Embodiment 2: The difference between Embodiment 2 and Embodiment 1 is that the objective function used in step 4 is different. The objective function of Embodiment 2 is as follows: The following objective function is used in step 4 to perform bandwidth and storage allocation in the monitoring area:
[0166]
[0167] Where: α and β represent adjustment parameters; Indicates the actual bandwidth usage and expected bandwidth of each area The relative deviation between Indicates the actual storage used and expected storage in each area The relative deviation between j represents the storage redundancy coefficient; B total Indicates the total bandwidth; S total Indicates the total storage space of NVR; B j represents the bandwidth allocation of monitoring area j; S j represents the storage allocation of monitoring area j; w1 and w2 represent weight coefficients; D' j Represents the normalized activity density value of the j-th monitoring area; Minimize represents minimizing the value of the corresponding objective function.
[0168] In this embodiment, by simultaneously considering the objective functions of activity density and deviation minimization, we have considered the problem of deviation minimization in the initial stage. Therefore, when the optimization effect is good, we do not need to adopt the objective function of deviation minimization in step 5; if fine-grained adjustment is required, the deviation minimization function in step 5 can be used for further optimization to ensure that on the basis of the fusion objective function (the objective function considering activity density and deviation minimization), the system can still further reduce the deviation; if the preliminary optimization stage based on the original fusion objective function has shown good convergence and sufficient accuracy, then the deviation minimization objective function in step 5 is not needed; therefore, in this embodiment, through the design of the fusion objective function, the optimization needs can be taken into account in the initial stage of resources, thereby reducing the magnitude of subsequent adjustments and improving the overall stability and efficiency of the system.
[0169] However, in Example 1 of the present application, two objective functions are adopted, namely, the initial allocation objective function in step 4 is intended to reasonably allocate bandwidth and storage resources according to the activity density of each monitoring area; the optimization objective function is the objective function of minimizing the deviation; therefore, the allocation scheme given by the initial allocation objective function deviates greatly from the demand in actual operation, which may lead to cumbersome adjustments and parameter changes in the optimization process, affecting the stability of the system; however, this technical problem also has a technical solution, namely, in the initial stage, the initial values of λ1, λ2, w1, and w2 are set as accurately as possible to ensure the rationality and stability of the initial allocation; then we can set the initial values of the above four parameters by testing; through the test method selection, the problem of large deviation, cumbersome adjustments and parameter changes in the optimization process, and affecting the stability of the system is avoided.
[0170] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. NVR and IPC management method for multi-region dynamic video stream switching and storage, characterized in that: The following steps are involved: Step 1: Obtain real-time video streams from multiple IPCs to obtain video frame data, and perform frame extraction and preprocessing on the obtained video frame data to obtain a continuous image frame sequence; Step 2: Perform activity detection based on a continuous image frame sequence to identify the moving target in the video and obtain relevant information about the moving target; Step 3: Based on the relevant information of the moving target, the activity density of each monitoring area is calculated to obtain the activity density value of each monitoring area; The step 3 comprises the following steps: Calculate the moving target's contribution to the activity density: map the moving target's position to the corresponding monitoring area, determine the monitoring area described by the moving target, traverse all the moving targets belonging to each monitoring area, and calculate the moving target's contribution to the activity density in the area: D j =∑ i∈j (W1·A i +W2·v i '); Where: D j represents the activity density of monitoring area j; A i represents the area of moving target i; v i ' represents the speed of the moving target; W1 and W2 represent weight coefficients; Normalize the activity density of each region by its maximum value: Where: D' j represents the normalized activity density value of the jth monitoring area; max(D j ) represents the maximum activity density value in all monitoring areas; ∈ represents a small value to prevent division by zero; When the D' j Less than the preset minimum activity density threshold D threshold When , it is considered that there is no activity in the area and the density value is set to 0; finally, the activity density value D' of each monitoring area j is obtained j ; Step 4: Dynamically adjust the transmission bandwidth of the IPC video stream and the storage strategy of the NVR according to the changes in activity density; Step 5: Provide real-time feedback on the effects of bandwidth and storage policy adjustments, analyze the gap between actual effects and expectations, obtain feedback data, and use optimization algorithms based on the feedback data to improve the accuracy of activity detection, bandwidth management, and storage policies.
2. The NVR and IPC management method for multi-region dynamic video stream switching and storage according to claim 1 is characterized in that: The step 2 comprises the following steps: Step 2.1: Calculate the difference between the current frame and the background model based on the background difference method to obtain a differential image; Step 2.2: Adaptively binarize the difference image to convert pixels into binary format, distinguish between static and moving areas, and use an adaptive threshold method to ensure accurate identification of moving areas under different lighting conditions; Step 2.3: In the binary image, use the Connected Components detection algorithm to identify independent moving target areas; Step 2.4: For each identified moving target area, calculate the position of the centroid in the moving target area; Calculate each moving target region R based on the pixel count of the moving target region class i area; The speed of the target is calculated by combining Kalman filtering with the joint probability data association algorithm.
3. The NVR and IPC management method for multi-region dynamic video stream switching and storage according to claim 2 is characterized in that: The position of the center of mass in the moving target area is calculated using a weighted method: Where: w(x,y) represents the weight of the pixel (x,y); (x i ,y i ) represents the moving target area R i The centroid coordinates of the pixel point in the image; x and y represent the horizontal and vertical coordinates of the pixel point in the image respectively; The weight of the pixel is calculated based on brightness and foreground probability: w(x,y)=α·I t (x,y)+β·P f (x,y); Where: I t (x, y) represents the brightness of the pixel (x, y) in the current frame t; P f (x, y) represents the probability that the pixel (x, y) belongs to the foreground; α represents the weight coefficient of the brightness term; β represents the weight coefficient of the foreground probability term.
4. The NVR and IPC management method for multi-region dynamic video stream switching and storage according to claim 2 is characterized in that: The steps to calculate the target's velocity are as follows: Target position prediction and update: Use the Kalman filter to predict and update the position of each target i: Target location prediction: Where: and are the predicted values of the centroid position of target i at time t; x i (t-1) and y i (t-1) is the actual center of mass position of target i at time t-1; v x,i (t-1) and v y,i (t-1) are the velocity components of target i at time t-1; Location Updates: Where: and is the observed value of target i at time t; K x and K y is the Kalman gain; Target velocity calculation: The velocity of each target is estimated based on the change in the center of mass position between consecutive frames: Where: v i (t) represents the speed of target i at time t; Δt is the frame time interval; The joint probability data association algorithm calculates the joint probability between each observation and all observations to determine the best target-observation pair: Where: P ij represents the association probability between target i and observation j; S -1 represents the inverse matrix of the covariance matrix; n represents the total number of all targets; represents the exponential term of the Gaussian distribution function; Denotes the difference vector d ij The transpose of ij The representation is as follows: Where: The horizontal axis represents the observation value j at time t; The ordinate represents the observation value j at time t; After target association, calculate the final velocity of each target: Where: v i (t)' represents the final velocity of target i; v ij (t) represents the velocity vector of target i and observation j; N represents the total number of all observations at time t.
5. The NVR and IPC management method for multi-region dynamic video stream switching and storage according to claim 1 is characterized in that: The step 4 comprises the following steps: Calculate bandwidth and storage allocation for each monitoring area based on activity density values: Where: B min Indicates the minimum bandwidth of each monitoring area; D' j represents the normalized activity density value of the jth monitoring area; D' k represents the normalized activity density value of the kth monitoring area; B total Indicates the total bandwidth; S total Indicates the total storage space of the NVR; S min represents the minimum storage space of each monitoring area; λ1 and λ2 represent Lagrange multipliers; M represents the number of monitoring areas; B j represents the bandwidth allocation of monitoring area j; S j represents the storage allocation of monitoring area j; Calculate the objective function based on the calculated bandwidth and storage allocation: Where: w1 and w2 represent weight coefficients; When calculating the objective function, the following constraints need to be met: B j ≥B min ,S j ≥S min ; Adjustment based on the objective function and constraints: If the objective function does not reach the desired optimal value or the constraints are not met, adjust the parameters λ1, λ2 or weight coefficients w1, w2, and recalculate the bandwidth and storage allocation until the objective function reaches the optimal value and all constraints are met.
6. The NVR and IPC management method for multi-region dynamic video stream switching and storage according to claim 5, characterized in that: The specific steps of step 5 are as follows: Initialization: Use the λ1, λ2, w1, and w2 calculated in the previous step as the initial values; Minimize deviation: Fine-tune the parameters λ1, λ2, w1, and w2 by gradient descent, with the goal of minimizing the objective function, where the objective function is: Where: Indicates the actual bandwidth usage and expected bandwidth of each area The relative deviation between Indicates the actual storage used and expected storage in each area The relative deviation between j Represents the storage redundancy coefficient; Minimize represents minimizing the value of the corresponding objective function; In each round of iteration, the bandwidth and storage allocation values of each monitoring area are recalculated, and feedback data is collected and processed again until the objective function reaches the optimal value or meets the convergence conditions.
7. The NVR and IPC management method for multi-region dynamic video stream switching and storage according to claim 1, characterized in that: In step 4, the following objective function is used to perform bandwidth and storage allocation in the monitoring area: Where: α and β represent adjustment parameters; Indicates the actual bandwidth usage and expected bandwidth of each area The relative deviation between Indicates the actual storage used and expected storage in each area The relative deviation between j Represents the storage redundancy coefficient; B total Indicates the total bandwidth; S total Indicates the total storage space of NVR; B j represents the bandwidth allocation of monitoring area j; S j represents the storage allocation of monitoring area j; w1 and w2 represent weight coefficients; D' j represents the normalized activity density value of the jth monitoring area; Minimize represents minimizing the value of the corresponding objective function; ΔB j Indicates the actual bandwidth usage and expected bandwidth of monitoring area j The difference between j Indicates the actual storage usage and expected storage capacity of monitoring area j The difference between .
