Multi-camera dynamic bandwidth allocation method and system

Abstract

The invention relates to the technical field of video monitoring and network transmission, and provides a multi-camera dynamic bandwidth allocation method and system in order to realize optimal allocation of bandwidth resources. A bandwidth weight calculation formula is determined based on the importance score, the object movement speed and the picture complexity, and the total bandwidth is allocated according to the bandwidth weight calculation formula, so that the bandwidth utilization rate is improved; coding parameters are dynamically adjusted based on the object movement speed and the regional importance, and the resolution of the key target is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video monitoring and network transmission, and particularly to a multi-camera dynamic bandwidth allocation method and system. Background Art

[0002] Existing camera bandwidth allocation schemes adopt a fixed allocation mode, that is, the same bandwidth (such as 4 Mbps) is fixedly allocated to each camera. This method does not consider the dynamic changes of the scene, resulting in wasted bandwidth for static pictures and degraded quality of burst activity pictures. Actual measurements show that 30%-60% of the bandwidth is consumed by redundant transmission under the fixed allocation mode. Summary of the Invention

[0003] In order to achieve the optimal allocation of bandwidth resources, the present application provides a multi-camera dynamic bandwidth allocation method and system.

[0004] The technical solution adopted by the present invention to solve the above problems is as follows:

[0005] The multi-camera dynamic bandwidth allocation method includes:

[0006] Step 1: Divide the importance levels of different monitoring areas according to the content of the monitoring picture and set the scores corresponding to the area importance levels. The more important the area, the higher the score;

[0007] Step 2: Establish a bandwidth weight calculation formula: w i =x(v i / v max )+yL i +zC i , where w i is the bandwidth weight corresponding to the i-th camera, v i is the maximum speed of the moving object in the monitoring picture of the i-th camera, v max is the preset maximum speed, L i is the maximum value of the area importance level score in the monitoring picture of the i-th camera, C i is the normalized picture complexity of the i-th camera, and x, y, and z are the weights corresponding to the object motion speed, area importance level score, and picture complexity respectively, and x + y + z = 1;

[0008] Step 3: Analyze the object motion speed, area importance level, and picture complexity in the pictures of each camera in real time, determine the maximum value of the area importance level score in the monitoring picture according to the area importance level, and determine the bandwidth weights of each camera according to the bandwidth weight calculation formula;

[0009] Step 4: Allocate the total bandwidth based on the bandwidth weights of each camera.

[0010] Further, the importance levels include the first level, the second level, and the third level; in the monitoring screen, the area containing identity recognition is the first level, and the importance level score is 0; the area where the moving object is located is the second level, and the importance level score is p; the static background area is the third level, and the importance level score is q, where 0 > p > q.

[0011] Further, the bandwidth weight calculation formula is: w i = 0.5(v i / v max ) + 0.3L i + 0.2C i , where v max = 10 m / s.

[0012] Further, it also includes step 5: determining the bit rate, QP, and GOP interval according to the regional importance level.

[0013] Further, if the regional importance level is the first level or the second level, the regional QP value is reduced by 6 - 10, the GOP length is shortened to within 30 frames, and the bit rate ratio corresponding to the maximum importance level score is increased to more than 70%.

[0014] Further, when the network is congested, the transmission of the first-level and second-level areas is preferentially guaranteed, and frame skipping transmission is enabled for the third-level areas.

[0015] Further, when the network is congested, it also includes: dynamically switching the TCP protocol to the UDP protocol and adding a forward error correction code.

[0016] The multi-camera dynamic bandwidth allocation system includes:

[0017] Regional importance level division unit: dividing the importance levels of different monitoring areas according to the monitoring screen content and setting the scores corresponding to the regional importance levels, with the more important the area, the higher the score;

[0018] Establishing the bandwidth weight calculation formula: w i = x(v i / v max ) + yL i + zC i , where w i is the bandwidth weight corresponding to the i-th camera, v i is the maximum speed of the moving object in the monitoring screen of the i-th camera, v max is the preset maximum speed, L i is the maximum value of the regional importance level score in the monitoring screen of the i-th camera, C i is the normalized picture complexity of the i-th camera, and x, y, and z are the weights corresponding to the object movement speed, regional importance level score, and picture complexity respectively, and x + y + z = 1;

[0019] Information acquisition unit: Analyze the object motion speed, area importance level, and picture complexity in the pictures of each camera in real time, and determine the maximum value of the area importance level score in the monitoring picture according to the area importance level.

[0020] Bandwidth weight calculation unit: Determine the bandwidth weights of each camera based on the information obtained by the information acquisition unit and the bandwidth weight calculation formula.

[0021] Bandwidth allocation unit: Allocate the total bandwidth based on the bandwidth weights of each camera.

[0022] Furthermore, it also includes an encoding control unit: Determine the bit rate, QP, and GOP interval according to the area importance level.

[0023] The beneficial effects of the present invention compared with the prior art are: Classify the importance of the monitoring area according to the monitoring content and determine the importance score, determine the bandwidth weight calculation formula based on the importance score, object motion speed, and picture complexity, and allocate the total bandwidth according to the bandwidth weight calculation formula, thereby improving the bandwidth utilization rate; Dynamically adjust the encoding parameters based on the object motion speed and area importance, and improve the resolution of key targets. Brief Description of the Drawings

[0024] Figure 1 It is a flow chart of a multi-camera dynamic bandwidth allocation method. Detailed Embodiment

[0025] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the following further details the present invention with reference to the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0026] As Figure 1 shown, the multi-camera dynamic bandwidth allocation method includes:

[0027] Step 1: Classify the importance levels of different monitoring areas according to the monitoring picture content and set the scores corresponding to the area importance levels. The more important the area, the higher the score.

[0028] The importance of the monitored area can be classified according to whether the monitored object (face / license plate / hazardous material, etc.) is included. The more important the monitored area is, the more bandwidth it will be allocated in the subsequent bandwidth allocation to meet the data transmission requirements. In this embodiment, the importance level is divided into three levels. Among them, the area containing identity recognition is the first level, the area where the moving object is located is the second level, and the static background area is the third level. The corresponding scores are recorded as o, p, q, and o>p>q. For the convenience of calculation, in this embodiment, o, p, q are taken as 3, 2, 1 respectively; the first level has the highest importance, so the score is the largest, and the third level has the lowest importance, so the score is the lowest. The regional importance classification can be automatically divided by training the DeepLabv3+ model, and the specific implementation method will not be elaborated here.

[0029] Step 2: Establish a bandwidth weight calculation formula.

[0030] In addition to the regional importance classification, the moving speed of the object and the complexity of the monitored video are also important parameters affecting the bandwidth allocation. Based on this, the bandwidth weight calculation formula designed in the present invention is: w i =x(v i / v max )+yL i +zC i , where w i is the bandwidth weight corresponding to the i-th camera, v i is the maximum speed of the moving object in the monitored video of the i-th camera, v max is the preset maximum speed, which can be set according to the monitored scenario. For example, in low-speed areas such as people moving or cars moving in the community, the value range of v max can be 3 - 5m / s, and in high-speed areas such as lanes, the value range of v max can be 20 - 30m / s, L i is the maximum value of the regional importance level score in the monitored video of the i-th camera, C i is the normalized video complexity, which takes values in (0 - 1). x, y, z are the weights corresponding to the moving speed of the object, the regional importance level score, and the video complexity respectively, and x + y + z = 1. x, y, z can be taken according to the actual situation. The moving speed of the object can be calculated by the optical flow method, and the accuracy can reach ±0.2m / s. The video complexity can be determined by the entropy method. In this embodiment, x takes 0.5, y takes 0.3, z takes 0.2, and v max takes 10m / s, that is, w i =0.5(v i / v max )+0.3L i +0.2C i .

[0031] Step 3: Analyze the object movement speed, area importance level, and picture complexity in the real-time images of each camera. Determine the maximum score of the area importance level in the surveillance images according to the area importance level, and determine the bandwidth weight of each camera according to the bandwidth weight calculation formula.

[0032] Step 4: Allocate the total bandwidth based on the bandwidth weights of each camera: R base is the total available bandwidth, N is the total number of cameras, and R i is the bandwidth allocated to the i-th camera.

[0033] Furthermore, in order to obtain better image quality, enhanced coding can be implemented for high-importance areas, reducing the quantization parameter (QP) and shortening the GOP interval. For example, for surveillance areas with an area importance level score ≥ 2, the QP value is reduced by 6 - 10; the GOP length is shortened to less than 30 frames; the bitrate ratio corresponding to the maximum area importance level score is increased to more than 70%. When network congestion occurs, the transmission of areas with an area importance level score ≥ 2 is preferentially guaranteed; frame skipping transmission (such as transmitting 1 frame every 3 frames) is enabled for areas with an area importance level score = 1; the TCP protocol can also be dynamically switched to the UDP protocol and forward error correction codes can be added to significantly improve real-time performance and transmission efficiency while ensuring the reliability of data transmission.

[0034] By adopting the dynamic bandwidth allocation method of the present invention, the bandwidth utilization rate is improved. Under the same picture quality, the bandwidth can be saved by 35% - 60%; the recognition accuracy of key targets is improved, and the recognition accuracy of key targets is increased from 78% to 95%; the real-time response ability is enhanced, and the bitrate adjustment delay < 150ms (the traditional scheme > 500ms); the network adaptability is stronger, and the end-to-end delay during peak hours is reduced to less than 120ms.

[0035] Correspondingly, the present invention also provides a multi-camera dynamic bandwidth allocation system, including:

[0036] Area importance level division unit: Divide the importance levels of different surveillance areas according to the surveillance picture content and set the scores corresponding to the area importance levels. The more important the area, the higher the score;

[0037] Bandwidth weight calculation formula determination unit: Used to establish the bandwidth weight calculation formula, w i = x(v i / v max ) + yL i + zC i , where w i is the bandwidth weight corresponding to the i-th camera, v i is the maximum speed of the moving object in the surveillance picture of the i-th camera, v max is the preset maximum speed, Li is the maximum value of the regional importance level score in the monitoring screen of the i-th camera, C i is the normalized picture complexity of the i-th camera, and x, y, and z are the weights corresponding to the object movement speed, regional importance level score, and picture complexity respectively, and x + y + z = 1;

[0038] Information acquisition unit: Analyze the object movement speed, regional importance level, and picture complexity in the pictures of each camera in real time, and determine the maximum value of the regional importance level score in the monitoring screen according to the regional importance level;

[0039] Bandwidth weight calculation unit: Determine the bandwidth weights of each camera based on the information obtained by the information acquisition unit and the bandwidth weight calculation formula;

[0040] Bandwidth allocation unit: Allocate the total bandwidth based on the bandwidth weights of each camera.

[0041] Furthermore, it further includes an encoding control unit: Determine the bit rate, QP, and GOP interval according to the regional importance level.

[0042] Embodiment 1

[0043] Taking the security monitoring scenario as an example, in this scenario, there are camera 1 and camera 2. Camera 1 detects 2 people walking, and camera 2 is a static corridor picture without moving targets; the total bandwidth is 20 Mbps.

[0044] Since camera 1 detects a face area and camera 2 has no moving targets, therefore, L corresponding to camera 1 is taken as 3, and L corresponding to camera 2 is taken as 1; after calculation, the maximum speed of the people in camera 1 is 1.2 m / s, then the weight of camera 1, W1 = 0.5*(1.2 / 10) + 0.3*3 + 0.2*0.8 = 1.24; the weight of camera 2, W2 = 0.5*0 + 0.3*1 + 0.2*0.3 = 0.36; bandwidth allocation result: R1 = 20*(1.24 / 1.6) = 15.5 Mbps, R2 = 4.5 Mbps. Encoding effect: QP of the face area = 22 (original 30), bit rate occupancy ratio 70%, QP of the background area = 35, bit rate occupancy ratio 30%.

[0045] Through dynamic bandwidth allocation and hierarchical encoding strategy (camera 1 occupies 77.5% of the bandwidth, QP = 22), it preferentially guarantees the high-definition transmission of key areas such as faces (recognition rate 95%+), and at the same time compresses static pictures (camera 2 only occupies 22.5%, QP = 35), achieving picture quality optimization and bandwidth utilization improvement (saving about 55% redundant traffic) under the total bandwidth of 20 Mbps.

[0046] Embodiment 2

[0047] Taking the traffic monitoring scenario as an example, camera 3 detects that a vehicle is speeding (v = 25 m / s, and the area importance level score is 3), and network congestion causes the available bandwidth to drop to 10 Mbps.

[0048] Dynamic response: Reduce the bitrate of non-critical cameras: The still intersection image is reduced from 4 Mbps to 1 Mbps; for the area of the speeding vehicle, QP = 20 and GOP = 15; Start the UDP fast transmission protocol (FEC redundancy 20%).

[0049] Performance data: The peak signal-to-noise ratio PSNR in the license plate area is increased from 32 dB to 38 dB; the bandwidth occupancy is reduced by 40%.

[0050] Example 3

[0051] Taking the multi-priority hybrid scenario as an example, scenario configuration: Camera 4 monitors the warehouse entrance, detects a face (area importance level score is 3) and a moving package (area importance level score is 2), the object speed v = 1.5 m / s, and the picture complexity C = 0.7; Camera 5 monitors the static shelf (L = 1, v = 0 m / s, C = 0.3), and the total bandwidth is 8 Mbps. According to the bandwidth weight formula calculation: The weight of camera 4, W4 = 0.5×(1.5 / 10)+0.3×3+0.2×0.7 = 1.115, and the weight of camera 5, W5 = 0.3×1+0.2×0.3 = 0.36. The total weight sum is 1.475, and the bandwidth allocation result is approximately 6.07 Mbps for camera 4 and approximately 1.93 Mbps for camera 5.

[0052] Encoding control: Camera 4 allocates 70% of the bandwidth (4.25 Mbps) to the area with an area importance level score of 3 (face), QP = 22, GOP = 30 frames; allocates 25% (1.52 Mbps) to the area with an area importance level score of 2 (package), QP = 28. Camera 5 enables frame skipping transmission (transmit 1 frame every 3 frames), QP = 35.

[0053] Performance data: The PSNR in the face area is increased to 38 dB, and the total bandwidth utilization rate is increased by 35%.

Claims

1. Method for dynamically allocating bandwidth of multiple cameras, characterized in that, Including: Step 1: Divide the importance levels of different monitoring areas according to the content of the monitoring screen and set the scores corresponding to the area importance levels. The more important the area, the higher the score. Step 2: Establish the bandwidth weight calculation formula: w i = x(v i / v max ) + yL i + zC i , where w i is the bandwidth weight corresponding to the i-th camera, v i is the maximum speed of the moving object in the monitoring screen of the i-th camera, v max is the preset maximum speed, L i is the maximum value of the regional importance level score in the monitoring screen of the i-th camera, C i is the standardized picture complexity of the i-th camera, and x, y, and z are the weights corresponding to the object movement speed, regional importance level score, and picture complexity respectively, and x + y + z = 1; Step 3: Analyze the object movement speed, area importance level, and screen complexity in the images of each camera in real time. Determine the maximum value of the area importance level score in the monitoring screen according to the area importance level, and determine the bandwidth weight of each camera according to the bandwidth weight calculation formula. Step 4: Allocate the total bandwidth based on the bandwidth weights of each camera.

2. The multi-camera dynamic bandwidth allocation method according to claim 1, wherein The importance levels include the first level, the second level, and the third level. In the monitoring screen, the area containing identity recognition is the first level, and the importance level score is taken as o. The area where the moving object is located is the second level, and the importance level score is taken as p. The static background area is the third level, and the importance level score is taken as q, where o > p > q.

3. The multi-camera dynamic bandwidth allocation method according to claim 2, wherein The bandwidth weight calculation formula is: w i = 0.5(v i / v max ) + 0.3L i + 0.2C i , v max = 10 m / s.

4. The multi-camera dynamic bandwidth allocation method according to claim 3, wherein It also includes Step 5: Determine the bit rate, QP, and GOP interval according to the area importance level.

5. The multi-camera dynamic bandwidth allocation method according to claim 4, wherein If the area importance level is the first level or the second level, then the area QP value is reduced by 6 - 10, the GOP length is shortened to less than 30 frames, and the bit rate ratio corresponding to the maximum value of the importance level score is increased to more than 70%.

6. The multi-camera dynamic bandwidth allocation method according to claim 4, wherein When the network is congested, give priority to ensuring the transmission of the first-level and second-level areas, and enable frame skipping transmission for the third-level areas.

7. The multi-camera dynamic bandwidth allocation method according to claim 6, characterized in that, When the network is congested, it also includes: Dynamically switch the TCP protocol to the UDP protocol and add a forward error correction code.

8. Multi-camera dynamic bandwidth allocation system, characterized in that, Including: Area importance level division unit: Divide the importance levels of different monitoring areas according to the content of the monitoring screen and set the scores corresponding to the area importance levels. The more important the area, the higher the score. Establish the bandwidth weight calculation formula: w i = x(v i / v max ) + yL i + ZC i , where w i is the bandwidth weight corresponding to the i-th camera, v i is the maximum speed of the moving object in the monitoring screen of the i-th camera, v max is the preset maximum speed, L i is the maximum value of the regional importance level score in the monitoring screen of the i-th camera, C i is the normalized picture complexity of the i-th camera, and x, y, and z are the weights corresponding to the object movement speed, the regional importance level score, and the picture complexity respectively, and x + y + z = 1; Information acquisition unit: Analyze the object movement speed, area importance level, and screen complexity in the images of each camera in real time, and determine the maximum value of the area importance level score in the monitoring screen according to the area importance level. Bandwidth weight calculation unit: Determine the bandwidth weights of each camera based on the information obtained by the information acquisition unit and the bandwidth weight calculation formula. Bandwidth allocation unit: Allocate the total bandwidth based on the bandwidth weights of each camera.

9. The multi-camera dynamic bandwidth allocation system according to claim 8, wherein, It also includes an encoding control unit: Determine the bit rate, QP, and GOP interval according to the area importance level.

Images