Multi-view camera and its method for dynamically configuring bit rate

Abstract

The present application provides a multi-camera and its dynamic bitrate configuration method, which relates to the technical field of monitoring devices. The multi-camera includes a first lens assembly and a second lens assembly; a first bitrate control module for storing a first byte amount, which includes the sum of the first byte differences between the first real-time bitrate and the first preset bitrate of the first lens assembly within a preset period, and the sum of the products of the second byte differences between the second real-time bitrate and the second preset bitrate of the second lens assembly and the first weight within a preset period; a bitrate allocation management module for, when a moving target is detected by any of the above lens assemblies, dynamically configuring the real-time bitrate of the lens assembly to be greater than the associated preset bitrate until the lens assembly no longer detects a moving target or the first byte amount stored in the first bitrate control module is lower than the first threshold. The present application can effectively improve the overall quality of the monitoring images of the multi-camera.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of monitoring devices, and in particular, to a multi-camera and a method for dynamically configuring its bit rate. Background Art

[0002] A multi-camera usually includes at least two cameras. One camera focuses on panoramic monitoring to ensure a wide monitoring coverage; and another camera is specifically used for target tracking or fixed-point cruising tasks, so as to provide a highly flexible and targeted monitoring perspective.

[0003] Existing multi-cameras usually need to configure independent rated bit rates for the monitoring channels of each camera. Through advanced linkage technology, the multi-camera can synchronously control multiple cameras and output monitoring images of different fields of view in real time.

[0004] However, in practical applications, when a certain camera moves omnidirectionally, there may be a problem of insufficient bit rate allocation. In this case, there will be motion mosaics, ghosting, and blurring during zooming in the monitoring image, seriously affecting the overall quality of the monitoring image. Summary of the Invention

[0005] Embodiments of the present application provide a multi-camera and a method for dynamically configuring its bit rate, which can effectively improve the overall quality of the monitoring image of the multi-camera.

[0006] In a first aspect, the present application provides a multi-camera, which includes:

[0007] A first lens assembly and a second lens assembly, the viewing angle of the second lens assembly is adjustable; the first lens assembly is configured to be associated with a first preset bit rate, and the second lens assembly is configured to be associated with a second preset bit rate; the real-time encoding methods of the first lens assembly and the second lens assembly are configured to be variable bit rate;

[0008] A first bit rate control module, the first bit rate control module is configured to store a first byte amount, and the first byte amount includes: the sum of the first byte differences between the first real-time bit rate and the first preset bit rate of the first lens assembly within a preset period, and the sum of the products of the second byte differences between the second real-time bit rate and the second preset bit rate of the second lens assembly and a first weight within a preset period;

[0009] A bit rate allocation management module, the bit rate allocation management module is configured to: when a moving target is detected in any one of the first lens assembly and the second lens assembly, dynamically configure the real-time bit rate of the lens assembly to be greater than the preset bit rate associated with it until the moving target is no longer detected in the lens assembly, or the first byte amount stored in the first bit rate control module is lower than a first threshold.

[0010] In a possible implementation, the above multi-camera further includes a second bitrate control module, which is configured to store a second byte amount, and the second byte amount includes: the product of the above second byte difference and the second weight within a preset period;

[0011] The bitrate allocation management module is configured to: when the second lens assembly is instructed to adjust the viewing angle, dynamically configure the real-time bitrate of the second lens assembly to be greater than the second preset bitrate until the second byte amount is lower than the second threshold or the viewing angle of the second lens assembly stops adjusting.

[0012] In a possible implementation, the above preset period is the first period, the calculation periods of the above first byte difference and the second byte difference are the second period, the first period is greater than the second period, the first bitrate control module obtains the first byte difference and the second byte difference with the second period, and the first bitrate control module updates the stored first byte amount with the first period; the second bitrate control module updates the stored second byte amount with the first period.

[0013] In a possible implementation, the second lens assembly is used to perform a preset operation, the sum of the first weight and the second weight is 1, and the above preset operation includes any one of the following: timed cruise, preset point cruise, intelligent target tracking, or gun-ball linkage.

[0014] In a possible implementation, when the second lens assembly is configured to perform a preset operation, the second weight is not 0.

[0015] In a possible implementation, when the number of moving pixel points detected in the monitoring screen of the first lens assembly or the second lens assembly exceeds a preset number, it is determined that the first lens assembly or the second lens assembly has detected a moving target.

[0016] In a possible implementation, the bitrate allocation management module is configured to:

[0017] When the second lens assembly detects a moving target, dynamically configure the real-time bitrate of the second lens assembly to an integer multiple of the second preset bitrate until the second lens assembly no longer detects a moving target or the sum of the first byte amount stored in the first bitrate control module and the second byte amount stored in the second bitrate control module is lower than the first threshold.

[0018] In a possible implementation, the bitrate allocation management module is configured to:

[0019] When the second lens assembly performs a preset operation and there is an area of interest in the monitoring screen, dynamically configure the real-time bitrate of the image content other than the area of interest in the above monitoring screen to an integer multiple of the second preset bitrate, and the real-time bitrate of the image content in the area of interest is the specified bitrate.

[0020] In a possible implementation, the first lens assembly is a gun-shaped lens assembly, and the second lens assembly is a spherical lens assembly.

[0021] Second, the present application provides a method for dynamically configuring the bit rate of a multi-camera. The multi-camera includes a first lens assembly and a second lens assembly, and the viewing angle of the second lens assembly is adjustable. The method includes:

[0022] Configure a first preset bit rate associated with the first lens assembly;

[0023] Configure a second preset bit rate associated with the second lens assembly;

[0024] Configure the real-time encoding mode of the first lens assembly and the second lens assembly as variable bit rate;

[0025] Store a first byte amount in a pre-constructed first bit rate control module. The first byte amount includes: the sum of the first byte differences between the first real-time bit rate and the first preset bit rate of the first lens assembly within a preset period, and the sum of the products of the second byte differences between the second real-time bit rate and the second preset bit rate of the second lens assembly and the first weight within a preset period;

[0026] When a moving target is detected in any one of the first lens assembly and the second lens assembly, dynamically configure the real-time bit rate of the lens assembly to be greater than the preset bit rate associated with it until the lens assembly no longer detects a moving target, or the first byte amount stored in the first bit rate control module is lower than the first threshold.

[0027] Third, the present application provides a computer-readable storage medium. Computer-executable instructions are stored in the computer-readable storage medium. When the computer-executable instructions are executed by a computer, they are used to implement the method provided in the second aspect.

[0028] Fourth, the present application provides a computer program product, including a computer program. When the computer program is executed by a computer, it implements the method provided in the second aspect.

[0029] The multi-camera and its method for dynamically configuring the bit rate provided by the embodiments of the present application can use the byte difference, that is, the surplus bit rate, between the actual bit rate used and the allocated bit rate of each camera during monitoring within a preset period to perform bit rate compensation on the camera that detects a moving target. It can not only control the size of video storage, but also improve the encoding effect of the camera that detects a moving target and improve the quality of the monitoring screen. Description of the Drawings

[0030] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0031] Figure 1 It is a schematic structural diagram of a multi-camera provided in the embodiment of the present application;

[0032] Figure 2 It is a schematic diagram of the bitrate dynamic configuration architecture of a multi-camera provided in the embodiment of the present application;

[0033] Figure 3 It is a schematic diagram of the bitrate dynamic configuration process of a multi-camera provided in the embodiment of the present application;

[0034] Figure 4 It is a schematic diagram of the bitrate allocation control strategy for each encoding channel in the embodiment of the present application;

[0035] Figure 5 It is a schematic diagram of the process of a bitrate dynamic configuration method for a multi-camera provided in the embodiment of the present application.

[0036] Through the above-mentioned drawings, the clear embodiments of the present application have been shown, and there will be more detailed descriptions in the following text. These drawings and text descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0038] It should be noted that although the terms "first", "second", etc. are used in the embodiments of the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. Optionally, without departing from the scope of the present application, the first information can also be called the second information, and similarly, the second information can also be called the first information.

[0039] It should be understood that the terms "comprising" and "including" indicate the existence of the previously mentioned features, steps, and operations, but do not exclude the existence, occurrence, or addition of one or at least one other feature, step, or operation. The term "and / or" used in this application can be interpreted inclusively, or means any one or any combination. Optionally, "A and / or B" means "any one of the following: A; B; A and B". Additionally, the character " / " in this text generally indicates an "or" relationship between the associated objects before and after.

[0040] The following explains some terms involved in the embodiments of this application:

[0041] 1. PTZ (Pan / Tilt / Zoom)

[0042] PTZ represents the omnidirectional (left / right / up / down) movement of the pan-tilt head and the control of lens zooming and variable focal length. These three functions are usually comprehensively applied to pan-tilt cameras, enabling the cameras to be remotely controlled to achieve a wider monitoring coverage and clearer detail capture.

[0043] Pan (horizontal rotation of the pan-tilt head): Allows the camera to rotate left and right in the horizontal direction to expand the monitoring field of view.

[0044] Tilt (vertical rotation of the pan-tilt head): Allows the camera to rotate up and down in the vertical direction to monitor targets at different heights.

[0045] Zoom (lens zooming): By adjusting the focal length of the lens, the field of view of the camera can be changed, thereby enabling detailed observation of distant targets or magnified display of nearby targets.

[0046] PTZ cameras can be widely applied to various scenarios that require large-scale monitoring and detail capture, such as urban monitoring, traffic management, large venues, banks, airports, schools, etc. Through PTZ control, monitoring personnel can flexibly adjust the position and focal length of the camera to meet various complex and dynamic monitoring requirements.

[0047] In addition, with the development of technology, PTZ cameras also have many other functions, such as automatic cruise, preset positions, intelligent tracking, etc., further improving the efficiency and accuracy of monitoring.

[0048] 2. Constant Bit Rate (CBR)

[0049] CBR means that the bit rate of the file per second is fixed. After encoding is completed, the bit rate will remain near the preset constant value. CBR encoding uses a feedback method. After encoding is completed, the current bit rate is calculated to see if it is consistent with the target bit rate. If the current bit rate is greater than the target bit rate, the generated bit rate is reduced by increasing the quantization coefficient of the next frame or dropping frames. If the current bit rate is less than the target bit rate, the generated bit rate is increased by reducing the quantization coefficient of the next frame. In this way, the encoder will try to make the output bit rate close to and remain near the target bit rate.

[0050] 3. Variable Bit Rate (VBR)

[0051] VBR means that the bit rate of the file per second can vary. The encoder will dynamically adjust the bit rate according to the complexity of the image content. During VBR encoding, the encoder analyzes the complexity and texture structure of the current encoded frame image to obtain the Mean Absolute Deviation (MAD) value. Then, according to the size of the MAD value, the quantization parameter (Qp) is dynamically adjusted, thereby dynamically adjusting the bit rate. For complex image frames, a higher bit rate is allocated to ensure image quality. For simple image frames, a lower bit rate is allocated to save space.

[0052] 4. Region of Interest (ROI)

[0053] In encoding, ROI refers to the key target areas in an image or video, and these areas need to be specially protected or optimized during the encoding process.

[0054] The encoder will identify and mark the ROI areas during the encoding process and then perform special processing on these areas. For example, more encoding resources (such as a higher bit rate, a lower quantization parameter) can be allocated to the ROI areas to ensure that the image quality of these areas is guaranteed first.

[0055] 5. Bits Per Second (BPS)

[0056] BPS is a unit for measuring data transmission rate or bit rate, indicating the number of bits transmitted per second.

[0057] 6. Dome Camera and Bullet Camera

[0058] A dome camera, namely an intelligent spherical camera, is a high-performance monitoring device integrating a camera system, a zoom lens, and an electronic pan-tilt. A dome camera can achieve 360-degree rotation without dead angles, covering a wider monitoring range. Through the zoom lens, a dome camera can capture detailed images at a long distance.

[0059] A bullet camera, also known as a gun-style camera, is a fixed-position surveillance camera. Bullet cameras are usually equipped with wide-angle lenses and can capture a large range of surveillance footage.

[0060] 7. Dome Camera Channels and Bullet Camera Channels

[0061] In a surveillance system, dome camera channels are the video channels responsible for detailed surveillance and are usually composed of dome cameras. Dome camera channels have PTZ functions and can implement surveillance functions such as target tracking and fixed-point trajectory cruising.

[0062] In a surveillance system, bullet camera channels are the video channels responsible for panoramic surveillance and are usually composed of bullet cameras. Bullet camera channels mainly provide surveillance functions with a wide field of view and a large surveillance range and are suitable for capturing panoramic images.

[0063] 8. Group of Pictures (GOP)

[0064] GOP refers to a set composed of a series of consecutive pictures (frames) in a video sequence. These pictures (frames) include I-frames (intra-coded frames), P-frames (forward-predicted frames), and B-frames (bi-directional interpolated frames).

[0065] A multi-camera can be a multi-channel and multi-field-of-view surveillance device that combines panoramic surveillance and detailed surveillance. For example, a multi-camera can include at least one dome camera and one bullet camera. The dome camera is responsible for detailed surveillance and target tracking, while the bullet camera is responsible for panoramic surveillance.

[0066] In existing multi-cameras, the video encoding of the dome camera surveillance channel and the bullet camera surveillance channel is independently controlled. Through advanced linkage technology, the multi-camera can output surveillance footage of different fields of view in real time.

[0067] However, in practical applications, since the dome camera needs to frequently adjust the surveillance angle and focal length to track the target, there is often not enough bitrate to encode the details of the moving target at the rated configured channel bitrate, resulting in problems such as moving mosaics, smear, and zoom blurring in the dome camera surveillance footage, seriously affecting the overall quality of the surveillance footage.

[0068] To address the above technical problems, in the embodiments of the present application, a multi-camera is provided that can use the byte difference between the actual bitrate used and the allocated bitrate by each camera during surveillance within a preset period, that is, the surplus bitrate, to perform bitrate compensation on the camera that detects a moving target, which can not only control the size of video storage but also improve the encoding effect of the camera that detects a moving target and enhance the quality of the surveillance footage.

[0069] The technical solutions provided by this application will be described in detail through specific embodiments below. It should be noted that the following several embodiments can exist independently or be combined with each other. For the same or similar content, it will not be repeated in different embodiments.

[0070] Referring to Figure 1 , Figure 1 FIG. 1 is a schematic structural diagram of a multi-camera provided in an embodiment of the present application. In some embodiments, the multi-camera 100 includes a first lens assembly 101, a second lens assembly 102, a first bitrate control module 103, and a bitrate allocation and management module 104.

[0071] Optionally, the first lens assembly 101 and the second lens assembly 102 are electrically connected.

[0072] In some embodiments, the first lens assembly 101 can provide a monitoring function with a wide field of view and a large monitoring range.

[0073] In some embodiments, the viewing angle of the second lens assembly 102 is adjustable, that is, the second lens assembly 102 supports the PTZ function and can implement monitoring functions such as target tracking and fixed-point trajectory cruising.

[0074] In some embodiments, the first lens assembly 101 can be a bullet lens assembly, and the second lens assembly 102 can be a dome lens assembly.

[0075] Alternatively, the first lens assembly 101 can be a component in a bullet camera, and the second lens assembly 102 can be a component in a dome camera.

[0076] In some embodiments, the first lens assembly 101 is configured to be associated with a first preset bitrate, and the second lens assembly 102 is configured to be associated with a second preset bitrate; the real-time encoding mode of the first lens assembly 101 and the second lens assembly 102 is configured to be variable bitrate.

[0077] In real-time encoding, variable bitrate allows the encoder to dynamically adjust the bitrate according to the complexity of the image content. For example, when the image content in the monitoring scene changes greatly (such as a suddenly appearing moving object or a light change), variable bitrate can ensure that the image quality will not be reduced due to bitrate limitations.

[0078] In some embodiments, the first bitrate control module 103 is configured to store a first byte amount, which includes: the sum of the first byte differences between the first real-time bitrate and the first preset bitrate of the first lens assembly within a preset period, and the sum of the products of the second byte differences between the second real-time bitrate and the second preset bitrate of the second lens assembly and a first weight within a preset period.

[0079] In some embodiments, the preset period may be a first period, the calculation period of the first byte difference and the second byte difference is a second period, and the first bitrate control module 103 may obtain the first byte difference and the second byte difference according to the second period, and the first bitrate control module 103 updates the stored first byte amount in the first period.

[0080] In each second period, the first bitrate control module 103 may calculate the difference between the first real-time bitrate of the first lens assembly 101 and the first preset bitrate, and this difference reflects the surplus between the actual bitrate and the expected bitrate of the first lens assembly 101.

[0081] At the same time, the first bitrate control module 103 may also calculate the difference between the second real-time bitrate of the second lens assembly 102 and the second preset bitrate, and multiply this difference by the first weight. This weight is used to adjust the influence degree of the second lens assembly on the first bitrate control module 103.

[0082] In some embodiments, the first period is greater than the second period.

[0083] Optionally, the time unit of the first period may be days (d), and the time unit of the second period may be hours (h).

[0084] Exemplarily, assuming that the first period is 7 days and the second period is 1 hour, the first bitrate control module 103 may calculate the first byte difference and the second byte difference every 1h, and save the calculated first byte difference and the second byte difference for 7 days. When it exceeds 7 days, the first byte difference and the second byte difference can be deleted.

[0085] Similarly, the first bitrate control module 103 may calculate the product of the second byte difference and the first weight every 1h, and save the calculated product of the second byte difference and the first weight for 7 days. When it exceeds 7 days, the product of the second byte difference and the first weight can be deleted.

[0086] In some embodiments, within the first period, the first bitrate control module 103 may accumulate all the calculated first byte differences to obtain a sum, and this sum can reflect the overall surplus situation of the bitrate usage of the first lens assembly 101.

[0087] Similarly, within the first period, the first bitrate control module 103 may accumulate all the calculated products of the second byte difference and the first weight to obtain a sum, and this sum can reflect the overall surplus situation of the bitrate usage of the second lens assembly 102.

[0088] In some embodiments, the bitrate allocation management module 104 may be configured to: when any one of the first lens assembly 101 and the second lens assembly 102 detects a moving target, the real-time bitrate of this lens assembly is dynamically configured to be greater than the preset bitrate associated therewith until this lens assembly no longer detects a moving target, or the first byte amount stored in the first bitrate control module 103 is lower than the first threshold.

[0089] In some embodiments, when any one of the first lens assembly 101 and the second lens assembly 102 detects a moving target, the real-time bitrate associated with this lens assembly is dynamically configured to be greater than its preset bitrate. This adjustment can ensure that during the movement of the target, the multi-camera 100 can provide higher image quality and more detailed information, thereby helping the user to more accurately identify and analyze the target.

[0090] In some embodiments, the adjustment of the real-time bitrate continues until one of the following two conditions is met:

[0091] a. This lens assembly no longer detects a moving target, indicating that the target may have left the monitoring area or stopped moving.

[0092] b. The first byte amount stored in the first bitrate control module 103 is lower than the first threshold, indicating that the current bitrate usage has approached the remaining threshold of the surplus bitrate, that is, the surplus bitrate of the multi-camera 100 is almost used up.

[0093] The multi-camera provided by the embodiments of the present application can use the byte difference, that is, the surplus bitrate, between the actual bitrate used and the allocated bitrate by each camera during monitoring within a preset period to perform bitrate compensation on the camera that detects a moving target, which can not only control the video storage size, but also improve the encoding effect of the camera that detects a moving target and improve the quality of the monitoring screen.

[0094] Based on the content described in the above embodiments, in some embodiments, the above multi-camera further includes a second bitrate control module, and the second bitrate control module is configured to store a second byte amount, and the second byte amount includes: the product of the above second byte difference and the second weight within the above preset period.

[0095] In some embodiments, the above second lens assembly 102 may be used to perform a preset operation. Optionally, the preset operation includes any one of the following:

[0096] Timed cruise: allowing the second lens assembly 102 to automatically switch between multiple preset positions according to a preset schedule. By setting the cruise path and time, the second lens assembly 102 can automatically monitor different key areas at different time periods.

[0097] Preset Point Cruise: A function where the second lens assembly 102 automatically switches between multiple preset positions. The user can pre-set multiple preset points in the camera control software, and each preset point represents a specific orientation and focal length of the camera. In the cruise mode, the second lens assembly 102 will switch between these preset points according to the preset order and time interval.

[0098] Intelligent Target Tracking: Utilize advanced image processing and artificial intelligence technologies to identify and track moving targets in the monitoring scene. Once the second lens assembly 102 detects a moving target, it will automatically adjust the focal length and orientation to ensure that the target always remains in the field of view.

[0099] Gun-Ball Linkage: When the gun-type camera detects a moving target, it will send the position information of the target to the spherical camera. The spherical camera will then automatically adjust the focal length and orientation according to this information to track and monitor the target in more detail. In the embodiments of this application, when the first lens assembly 101 detects a moving target, it will send the position information of the target to the second lens assembly 102. The second lens assembly 102 will then automatically adjust the focal length and orientation according to this information to track and monitor the target in more detail.

[0100] In some embodiments, the sum of the above first weight and the second weight is 1. For example, assuming the above second weight is K%, then the above first weight is 1 - K%.

[0101] In some embodiments, when the second lens assembly is configured to perform the above preset operations, the above second weight is not 0.

[0102] In some embodiments, when the number of moving pixel points detected in the monitoring screen of the first lens assembly 101 exceeds a preset number, it can be determined that the first lens assembly 101 has detected a moving target. Similarly, when the number of moving pixel points detected in the monitoring screen of the second lens assembly 102 exceeds a preset number, it can be determined that the second lens assembly 102 has detected a moving target.

[0103] In some embodiments, the multi-camera can continuously capture images of the monitoring area to form a series of video frames. By comparing the differences between adjacent frames, it is possible to identify which pixel points have changed significantly, and these changes are usually caused by moving targets. When the number of detected moving pixel points exceeds a certain preset threshold, it can be determined that a moving target has been detected.

[0104] In some embodiments, the above bitrate allocation management module 104 is configured to:

[0105] When the second lens assembly 102 detects a moving target, the real-time bitrate of the second lens assembly 102 is dynamically configured to an integer multiple of the second preset bitrate until the second lens assembly 102 no longer detects a moving target, or the sum of the first byte amount stored in the first bitrate control module 103 and the second byte amount stored in the second bitrate control module is lower than the first threshold.

[0106] When the second lens assembly 102 detects a moving target, the bitrate allocation management module 104 can ensure that when an important event (such as a moving target) occurs in the monitoring scenario, the second lens assembly 102 can allocate more bitrates to capture and transmit high-quality video data by dynamically configuring the real-time bitrate of the second lens assembly 102 to an integer multiple (such as 2 times) of the second preset bitrate. Once the moving target is no longer detected, or when the sum of the first byte amount stored in the first bitrate control module 103 and the second byte amount stored in the second bitrate control module is lower than a preset first threshold, the bitrate allocation management module 104 will adjust or restore the bitrate of the second lens assembly 102 accordingly.

[0107] In some embodiments, the above bitrate allocation management module 104 is configured as follows:

[0108] When the second lens assembly 102 performs the above preset operation and there is an area of interest in the monitoring screen, the real-time bitrate of the image content in the monitoring screen except for the area of interest is dynamically configured to an integer multiple of the second preset bitrate, and the real-time bitrate of the image content in the area of interest is the specified bitrate.

[0109] For the image content in the monitoring screen except for the area of interest, the bitrate allocation management module 104 dynamically configures its real-time bitrate to an integer multiple (such as 3 times) of the second preset bitrate, while ensuring the overall video smoothness, allocating a larger bitrate to the area of interest to capture and transmit higher-quality video data.

[0110] Based on the content described in the above embodiments, refer to Figure 2 , Figure 2 which is a schematic diagram of the bitrate dynamic configuration architecture of a multi-camera provided by an embodiment of the present application. In some embodiments, the above multi-camera includes a first lens assembly, a second lens assembly, a first bitrate control module, a second bitrate control module, a bitrate balance management module, and a bitrate allocation management module.

[0111] In some embodiments, each encoding channel (such as the first lens assembly encoding channel, the second lens assembly encoding channel) can provide the preset bitrate associated with it to the bitrate balance management module. For example, the first lens assembly provides the first preset bitrate L1 associated with it to the bitrate balance management module, and the second lens assembly provides the second preset bitrate L2 associated with it to the bitrate balance management module.

[0112] In some embodiments, the total video storage period T (which can be configured by the user) can be preset in advance, that is, the above-mentioned first period. For example, if 7 days is set as a storage period, the total video storage size of all encoding channels within any consecutive 7 days does not exceed the sum of the actual storage sizes of each encoding channel.

[0113] In some embodiments, the bitrate balance management module can be configured to: count the encoding bitrate balance of each encoding channel and record the data in the balance bitrate statistics pool.

[0114] Optionally, the above-mentioned balance bitrate statistics pool can be a memory data management unit implemented by software, and the data unit is Byte.

[0115] In some embodiments, the first bitrate control module can establish a common balance bitrate statistics pool. The attribute of this common balance bitrate statistics pool is a storage pool shared by each encoding channel. When a certain encoding channel needs to increase the bitrate to improve the encoding effect in a motion scene, it can obtain a compensation bitrate from the common balance bitrate statistics pool to achieve a better encoding effect.

[0116] In some embodiments, the first bitrate control module can also establish a first balance bitrate statistics pool, which is a private balance bitrate statistics pool for the first lens assembly.

[0117] Optionally, the storage period of the first balance bitrate statistics pool can be the above-mentioned second period, such as 1h. When the actual first real-time bitrate R1 of the first lens assembly is less than the expected allocated first preset bitrate S1, the storage amount for 1h is: (S1 - R1) × 1 × 60 × 60. Every 1h, the stored value of the first balance bitrate statistics pool needs to be transferred to the above-mentioned common balance bitrate statistics pool.

[0118] In some embodiments, the first bitrate control module can also establish a second balance bitrate statistics pool, which is a private balance bitrate statistics pool for the second lens assembly.

[0119] In some embodiments, when the second lens assembly is configured with PTZ intelligent target tracking or timed fixed-point cruise, a second weight can be assigned to the second lens assembly and fed back to the bitrate balance management module.

[0120] In addition, the second bitrate control module can create a PTZ balance bitrate statistics pool. When the encoded second real-time bitrate R2 of the second lens assembly is less than the expected allocated second preset bitrate S2, the stored value of the second balance bitrate statistics pool is (S2 - R2) × (1 - K) × 60 × 60, and the stored value of the PTZ balance bitrate statistics pool is (S2 - R2) × K × 60 × 60.

[0121] In some embodiments, every 1 h, it is necessary to convert the stored value of the second balance bitrate statistics pool to the common balance bitrate statistics pool, and the stored value of the PTZ balance bitrate pool can always be accumulated without conversion.

[0122] As described above, the stored value of the common balance bitrate statistics pool is periodically converted from the balance bitrate statistics pools of each encoding channel.

[0123] In some embodiments, the above bitrate allocation management module can be used to manage and allocate the real-time bitrates of the first lens assembly and the second lens assembly; when the scene is stationary, the real-time bitrate is the initialized preset bitrate, and when there is a moving scene or the second lens assembly is in the PTZ motion tracking state, the real-time bitrate will be adjusted to a larger bitrate to improve the encoding effect. Among them, the additional bitrate source is the above common balance bitrate statistics pool.

[0124] In addition, when the second lens assembly is in the PTZ state, a compensation bitrate can be additionally obtained from the PTZ balance bitrate statistics pool. Thus, even if there is no balance bitrate in the common balance bitrate statistics pool, the encoding effect when the second lens assembly is in the PTZ motion state can be improved.

[0125] In some embodiments, it is assumed that the user pre-sets the first period T (such as 7 days), and then configures the first preset bitrate of the first lens assembly as L1 and the first preset bitrate of the second lens assembly as L2, with the unit of bps.

[0126] In some embodiments, each encoding channel performs variable bitrate intelligent encoding according to the above L1 or L2; the rate control strategy of this intelligent encoding can adjust the internal encoding strategy according to motion and stillness detection.

[0127] For example, in a stationary scene, low bitrate encoding can be achieved by reducing the frame rate, etc., and the saved bitrate can be stored in their respective private balance bitrate statistics pools, with the storage upper limit being the second period (such as 1 hour).

[0128] Exemplarily, the sum of the first byte differences between the first real-time bitrate and the first preset bitrate of the first lens assembly within the second period The calculation formula is:

[0129] ;

[0130] Among them, can represent the first preset bitrate associated with the first lens assembly, can represent the first real-time bitrate of the first lens assembly.

[0131] In some embodiments, for the second lens assembly, if the user configures PTZ cruise or ROI tracking of a moving target, a second weight K (between 0 and 1) needs to be configured. The configuration source can be operations that require controlling the PTZ movement of the camera, such as timed cruise, fixed-point cruise, intelligent target detection and tracking, and linkage tracking between a gun camera and a dome camera. Different sources can be assigned different coefficients.

[0132] In some embodiments, for the surplus bitrate BpsSave generated by the second lens assembly in a static scene, the byte amount of K×BpsSave can be stored in the PTZ surplus bitrate statistics pool, and the byte amount of (1 - K)×BpsSave can be stored in the second surplus bitrate statistics pool. The storage upper limit of the PTZ surplus bitrate statistics pool is the first period T. Every time one hour is exceeded, the surplus record of the earliest hour needs to be deleted.

[0133] In some embodiments, every second period, the surplus bitrates stored in the private surplus bitrate statistics pools of each encoding channel need to be converted to the public surplus bitrate statistics pool, and the attribute of the bitrate surplus is changed from private to public. The storage upper limit of the public surplus bitrate statistics pool is the first period T.

[0134] Exemplarily, the sum of the first byte differences between the first real-time bitrate of the first lens assembly and the first preset bitrate stored in the public surplus bitrate statistics pool within the first period can be expressed as:

[0135] ;

[0136] wherein, can represent the sum of the first byte differences between the first real-time bitrate of the first lens assembly and the first preset bitrate in the i-th second period, and n is the number of second periods within the first period.

[0137] Exemplarily, referring to Figure 3 , Figure 3 is a schematic diagram of the bitrate dynamic configuration process of a multi-camera provided in an embodiment of the present application.

[0138] In some embodiments, every time the first lens assembly encodes a group of GOPs, the first byte difference between the first real-time bitrate of the first lens assembly and the first preset bitrate, that is, the surplus bitrate, can be calculated and stored in the first surplus bitrate statistics pool, and the storage ratio is 100%.

[0139] Furthermore, every second period, the stored value in the first surplus bitrate statistics pool can be converted to the public surplus bitrate statistics pool.

[0140] When the first lens component detects a moving target, obtain a compensation bit rate from the common balance bit rate statistical pool, and adjust the real-time bit rate of the first lens component based on the compensation bit rate.

[0141] In some embodiments, for each GOP encoded by the second body lens component, the second real-time bit rate of the second lens component and the second byte difference between the second preset bit rate, that is, the balance bit rate, can also be calculated. The product of the second byte difference and the first weight (1 - K%) is stored in the second balance bit rate statistical pool, and the product of the second byte difference and the second weight (K%) is stored in the PTZ balance bit rate statistical pool.

[0142] Furthermore, every second period, the stored value in the second balance bit rate statistical pool can be transferred to the common balance bit rate statistical pool.

[0143] When the second lens component detects a moving target, obtain a compensation bit rate from the common balance bit rate statistical pool, and adjust the real-time bit rate of the second lens component based on the compensation bit rate.

[0144] When there is PTZ movement in the second lens component (such as being instructed to adjust the viewing angle), obtain a compensation bit rate from the PTZ balance bit rate statistical pool, and adjust the real-time bit rate of the second lens component based on the compensation bit rate.

[0145] In some embodiments, when any one of the first lens component and the second lens component detects a moving target, a compensation bit rate can be obtained from the common balance bit rate statistical pool, so that the target bit rate of the current lens component channel is adjusted to an integer multiple (such as twice) of the preset bit rate. This state will continue until the movement state ends, or there is no balance bit rate in the common balance bit rate pool that can be allocated to this channel (such as the first byte amount stored in the common balance bit rate pool is lower than the first threshold).

[0146] In some embodiments, when there is no balance bit rate in the common balance bit rate statistical pool, the lens component can directly use the balance bit rate in its private balance bit rate statistical pool.

[0147] In some embodiments, if the second lens component needs PTZ tracking or fixed-point timing cruising, the PTZ balance bit rate statistical pool can be additionally used to further increase the real-time bit rate of the second lens component. For example, the real-time bit rate of the second lens component can reach an integer multiple (such as three times) of the second preset bit rate, so as to achieve a better encoding effect.

[0148] In some embodiments, if the remaining bitrate in the PTZ remaining bitrate statistical pool is insufficient, it can degrade to twice the preset bitrate; or, when the remaining bitrate in the common remaining bitrate statistical pool is insufficient, it can also degrade to twice the target bitrate; if there is no remaining bitrate in both the PTZ remaining bitrate statistical pool and the common remaining bitrate statistical pool, it can be encoded at the preset bitrate, that is, there is no additional bitrate compensation.

[0149] In some embodiments, during the PTZ tracking or cruising phase, the second lens assembly can perform ROI tracking protection and quality protection with a downward adjustment of MaxQP (the maximum quantization parameter, used to limit the worst picture quality) using the coordinates detected by AI target detection to ensure better encoded picture quality during the PTZ phase.

[0150] In some embodiments, the target bitrate after compensation in the above motion scenario does not represent the actual bitrate consumed by the channel. If too much bitrate is allocated and it is clamped by the minQP (the minimum quantization parameter, used to limit the best picture quality) quality and the actual bitrate does not reach the target bitrate, the remaining bitrate can still be stored according to the remaining bitrate.

[0151] In some embodiments, the remaining bitrates stored in the common remaining bitrate statistical pool and the PTZ remaining bitrate statistical pool use the first period T as the unit window and the second period (such as 1 hour) as the stepping unit. That is, when the cumulative bitrate control time exceeds T, every hour, the statistical value of the remaining bitrate for the oldest hour is deleted to ensure that the actual remaining bitrates stored in the common remaining bitrate statistical pool and the PTZ remaining bitrate statistical pool are always controlled according to the period of T.

[0152] In some embodiments, if the monitoring scenario of the first lens assembly is always a simple static scenario, the average bitrate of this channel within T time will be lower than its associated first preset bitrate. If the bitrate balance in the common remaining bitrate statistical pool is always sufficient, the average bitrate of the second lens assembly with more motion scenarios within T time will be higher than its associated second preset bitrate. However, the sum of the storage sizes of all channels within T time will not exceed the total expected storage size.

[0153] Exemplarily, referring to Figure 4 , Figure 4 is a schematic diagram of the bitrate allocation control strategy for each encoding channel in the embodiments of the present application.

[0154] In some embodiments, for static scenarios, low-bitrate encoding can be selected; for motion scenarios, if there is no PTZ motion, twice the preset bitrate can be selected; if there is PTZ motion, three times the preset bitrate can be selected when there is no ROI target tracking, and a specified bitrate can be selected when there is ROI target tracking.

[0155] The multi-camera provided by the embodiment of the present application can significantly improve the problem of insufficient bit rate in the PTZ (detail path) channel during target tracking and fixed-point cruise.

[0156] Based on the content described in the embodiment, in some embodiments of the present application, a method for dynamically configuring the bit rate of a multi-camera is further provided. Refer to Figure 5 , Figure 5 It is a schematic flowchart of a method for dynamically configuring the bit rate of a multi-camera provided in the embodiment of the present application. In some embodiments, the method for dynamically configuring the bit rate of the above multi-camera includes:

[0157] S501. Store a first byte amount in a pre-constructed first bit rate control module, where the first byte amount includes: the sum of the first byte differences between the first real-time bit rate and the first preset bit rate of the first lens assembly within a preset period, and the sum of the products of the second byte differences between the second real-time bit rate and the second preset bit rate of the second lens assembly and the first weight within a preset period;

[0158] S502. When a moving target is detected by any one of the first lens assembly and the second lens assembly, dynamically configure the real-time bit rate of this lens assembly to be greater than the associated preset bit rate until this lens assembly no longer detects a moving target, or the first byte amount stored in the first bit rate control module is lower than a first threshold.

[0159] In some embodiments, the method for dynamically configuring the bit rate of the above multi-camera further includes:

[0160] Configure the first preset bit rate associated with the first lens assembly, configure the second preset bit rate associated with the second lens assembly, and configure the real-time encoding modes of the first lens assembly and the second lens assembly as variable bit rate.

[0161] It can be understood that the implementation content of each step of the method for dynamically configuring the bit rate of the multi-camera provided in the embodiment of the present application can refer to the description in the corresponding embodiment of the above multi-camera, and will not be elaborated here.

[0162] The present application further provides a computer-readable storage medium, which stores a computer program (which can also be called code, or instruction). When this computer program is run, the method for dynamically configuring the bit rate of the multi-camera provided in the above-mentioned embodiment can be implemented.

[0163] The present application further provides a computer program product, which includes: a computer program (which can also be called code, or instruction), and when this computer program is run, the method for dynamically configuring the bit rate of the multi-camera provided in the above-mentioned embodiment can be implemented.

[0164] It can be understood that the division of each module in the above multi-view camera is only a division of logical functions. Each function can correspond to a functional module, or two or more functions can be integrated into one functional module. In actual implementation, all or part of the modules can be integrated into one physical entity, or distributed among different physical entities. In addition, according to the actual situation, the above functional modules may be implemented in the form of hardware, may also be implemented in the form of software, or may be implemented in a combination of hardware and software. Whether a certain function is executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described function for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0165] In the above embodiments, the processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method provided in combination with this application can be directly implemented by a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0166] In each embodiment of this application, the various functional modules can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one unit.

[0167] If the functional module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0168] Finally, it should be noted that this application is intended to cover any variations, uses or adaptations of this application, which follow the general principles of this application and include the common general knowledge or conventional technical means in the technical field not claimed in this application. It is not limited to the exact structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is only limited by the appended claims.

Claims

1. A multi-camera, characterized in that: The multi-camera comprises: A first lens assembly and a second lens assembly, wherein the viewing angle of the second lens assembly is adjustable; the first lens assembly is configured to be associated with a first preset bit rate, and the second lens assembly is configured to be associated with a second preset bit rate; and the real-time encoding mode of the first lens assembly and the second lens assembly is configured to be a variable bit rate; a first bit rate control module, the first bit rate control module being configured to store a first byte amount, the first byte amount comprising: a sum of a first byte difference between a first real-time bit rate of the first lens assembly and the first preset bit rate within a preset period, and a sum of a product of a second byte difference between a second real-time bit rate of the second lens assembly and the second preset bit rate and a first weight within the preset period; A bit rate allocation management module, wherein the bit rate allocation management module is configured to: when any lens assembly of the first lens assembly and the second lens assembly detects a moving target, dynamically configure the real-time bit rate of the lens assembly to be greater than a preset bit rate associated therewith, until the lens assembly no longer detects the moving target, or the first byte amount stored by the first bit rate control module is lower than a first threshold.

2. The multi-eye camera according to claim 1, characterized in that: The multi-eye camera further includes a second bit rate control module, the second bit rate control module is configured to store a second byte amount, the second byte amount includes: within the preset period, the product of the second byte difference and the second weight; The bit rate allocation management module is configured to: when the second lens assembly is instructed to adjust the viewing angle, dynamically configure the real-time bit rate of the second lens assembly to be greater than the second preset bit rate until the second byte amount is lower than a second threshold, or the viewing angle of the second lens assembly stops adjusting.

3. The multi-camera according to claim 2, characterized in that: The preset period is a first period, the calculation period of the first byte difference and the second byte difference is a second period, the first period is greater than the second period, the first bit rate control module obtains the first byte difference and the second byte difference in the second period, and the first bit rate control module updates the stored first byte amount in the first period; The second bit rate control module updates the stored second byte amount at the first period.

4. The multi-camera according to claim 2, characterized in that: The second lens assembly is used to perform a preset operation, the sum of the first weight and the second weight is 1, and the preset operation includes any one of the following: timed cruise, preset point cruise, intelligent target tracking or gun-ball linkage.

5. The multi-eye camera according to claim 4, characterized in that: When the second lens assembly is configured to perform the preset operation, the second weight is not 0.

6. The multi-eye camera according to claim 5, characterized in that: When the number of moving pixels detected in the monitoring picture of the first lens assembly or the second lens assembly exceeds a preset number, it is determined that the first lens assembly or the second lens assembly has detected the moving target.

7. The multi-eye camera according to claim 6, characterized in that: The code rate allocation management module is configured as follows: When the second lens assembly detects the moving target, the real-time bit rate of the second lens assembly is dynamically configured to be an integer multiple of the second preset bit rate until the second lens assembly no longer detects the moving target, or the sum of the first byte amount stored in the first bit rate control module and the second byte amount stored in the second bit rate control module is lower than the first threshold.

8. The multi-eye camera according to claim 6, characterized in that: The code rate allocation management module is configured as follows: When the second lens assembly performs the preset operation and there is an area of ​​interest in the monitoring screen, the real-time bit rate of the image content in the monitoring screen excluding the area of ​​interest is dynamically configured to be an integer multiple of the second preset bit rate, and the real-time bit rate of the image content in the area of ​​interest is the specified bit rate.

9. The multi-eye camera according to any one of claims 1 to 8, characterized in that: The first lens assembly is a gun-type lens assembly, and the second lens assembly is a spherical lens assembly.

10. A method for dynamically configuring bit rates of multi-cameras, characterized in that: The multi-view camera comprises a first lens assembly and a second lens assembly, wherein the viewing angle of the second lens assembly is adjustable; and the method comprises: configuring a first preset bit rate associated with the first lens assembly; configuring a second preset bit rate associated with the second lens assembly; configuring the real-time encoding mode of the first lens assembly and the second lens assembly to be a variable bit rate; A first byte quantity is stored in a pre-built first bit rate control module, wherein the first byte quantity includes: a sum of a first byte difference between a first real-time bit rate of the first lens assembly and the first preset bit rate within a preset period, and a sum of a product of a second byte difference between a second real-time bit rate of the second lens assembly and the second preset bit rate and a first weight within the preset period; When any lens assembly of the first lens assembly and the second lens assembly detects a moving target, the real-time bit rate of the lens assembly is dynamically configured to be greater than a preset bit rate associated with it, until the lens assembly no longer detects the moving target, or the first byte amount stored by the first bit rate control module is lower than a first threshold.

Images