Region of interest based rate control method, apparatus, device and medium

Abstract

Embodiments of the present disclosure provide a region of interest based rate control method and device, electronic equipment, storage medium and computer program product, relating to the technical field of image processing. The method determines first and second rate control reference values for a region of interest and a non-region of interest, respectively, based on rate estimation of a video frame to be encoded; determines a quantization parameter adjustment value for each coding block based on the first and second rate control reference values, and controls the rate of each coding block based on the quantization parameter adjustment value. The method differentiates the rate control of the region of interest and the non-region of interest, so that the subjective perception of the video picture is better. The first and second rate control reference values are determined based on the rate estimation of the video frame to be encoded, which ensures that the overall rate fluctuation of the video remains stable. The overall picture quality is improved through block-level quantization parameter adjustment.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to a bitrate control method, apparatus, electronic device, storage medium, and computer program product based on region of interest. Background Technology

[0002] In real-time transmission scenarios, viewers focus more on the facial areas in the frame and pay relatively less attention to the background. However, traditional encoding technologies often fail to fully utilize information such as face detection to optimize the encoding quality of critical areas, resulting in unclear images, loss of facial details, and negatively impacting the user's subjective experience. To address this issue, face detection technology can be integrated into the video encoding process. After the client identifies the region of interest, this information is provided to the encoder, allowing it to prioritize the protection and optimization of data for important parts such as faces. This targeted strategy not only significantly improves the quality of facial areas in the video but also enhances the overall viewing experience, providing users with a clearer and more natural visual experience.

[0003] Bitrate control aims to optimize bandwidth usage, ensuring video clarity while reducing bandwidth consumption. Its core lies in adjusting the quantization parameter (QP) to balance image quality and bitrate: a smaller QP value results in higher quality but a larger bitrate, and vice versa. Combined with region-of-interest (ROI) recognition technology, it's possible to prioritize improving the quality of key regions such as faces while maintaining overall encoding efficiency, achieving a better visual experience and higher bandwidth utilization. Summary of the Invention

[0004] This disclosure provides a bitrate control method, apparatus, electronic device, storage medium, and computer program product based on region of interest.

[0005] According to a first aspect of the present disclosure, a bitrate control method based on a region of interest (ROI) is provided, comprising: determining an interest target in a video frame to be encoded; determining at least one ROI and non-ROI in the video frame to be encoded based on the interest target; performing bitrate estimation on the video frame to be encoded to obtain a first estimated number of bits; determining a first bitrate control reference value and a second bitrate control reference value corresponding to the ROI and non-ROI, respectively, based on the first estimated number of bits; determining a quantization parameter adjustment value corresponding to each coding block in the video frame to be encoded based on the first bitrate control reference value and the second bitrate control reference value; and performing bitrate control on each coding block based on the quantization parameter adjustment value.

[0006] In some exemplary embodiments of this disclosure, determining at least one region of interest and non-region of interest in the video frame to be encoded based on the interest target includes: generating a binarized image based on the interest target; the binarized image is used to label the first region of interest and the first non-region of interest; performing erosion processing on the first region of interest to obtain an eroded first region of interest; and performing dilation processing on the eroded first region of interest to determine the at least one region of interest and non-region of interest.

[0007] In some exemplary embodiments of this disclosure, the extent of the expansion treatment is not less than the extent of the corrosion treatment.

[0008] In some exemplary embodiments of this disclosure, the method further includes: calculating the area of ​​the at least one region of interest; calculating the area percentage of the at least one region of interest in the video frame to be encoded; and performing the bitrate control on the video frame to be encoded in response to the area percentage being within a preset area percentage range.

[0009] In some exemplary embodiments of this disclosure, determining a first code control reference value and a second code control reference value corresponding to the region of interest and the region of non-interest based on the first estimated number of bits includes: setting a first code control setting value corresponding to the region of interest; performing code rate estimation on the at least one region of interest based on the first code control setting value to obtain a second estimated number of bits corresponding to the region of interest; calculating a third estimated number of bits corresponding to the region of non-interest based on the first estimated number of bits and the second estimated number of bits; calculating a second code control setting value corresponding to the region of non-interest based on the third estimated number of bits; and determining the first code control setting value as the first code control reference value and the second code control setting value as the second code control reference value.

[0010] In some exemplary embodiments of this disclosure, determining that the first code control setting value is the first code control reference value and the second code control setting value is the second code control reference value includes: determining whether the second code control setting value is greater than a non-interest region adjustment threshold; adjusting the first code control setting value and recalculating the second code control setting value in response to the second code control setting value being greater than the non-interest region adjustment threshold; and determining that the first code control setting value is the first code control reference value and the second code control setting value is the second code control reference value in response to the second code control setting value not being greater than the non-interest region adjustment threshold.

[0011] In some exemplary embodiments of this disclosure, the method further includes: obtaining a first complexity and a second complexity corresponding to the at least one region of interest and a region of non-interest, respectively; calculating a second estimated number of bits based on the first complexity and a first code control setting value; and calculating a second code control setting value based on the second complexity and a third estimated number of bits.

[0012] In some exemplary embodiments of this disclosure, determining the quantization parameter adjustment value corresponding to each coding block in the video frame to be encoded based on the first code control reference value and the second code control reference value includes: determining a first coding block in the video frame to be encoded; the first coding block belongs to each coding block; determining a first quantization parameter adjustment value of the first coding block relative to each region of interest based on the first code control reference value and the second code control reference value; and determining that the minimum value among the first quantization parameter adjustment values ​​is the quantization parameter adjustment value corresponding to the first coding block.

[0013] In some exemplary embodiments of this disclosure, determining the first quantization parameter adjustment value of the first coding block relative to each of the regions of interest based on the first code control reference value and the second code control reference value includes: in response to the first coding block being located in a target region of interest, determining a rectangular region corresponding to the target region of interest; determining the center point of the rectangular region and a first distance from the vertex to the center point; determining a second distance from the first coding block to the center point; and determining the first quantization parameter adjustment value of the first coding block relative to the target region of interest based on the first distance, the second distance, and the first code control reference value.

[0014] In some exemplary embodiments of this disclosure, determining the first quantization parameter adjustment value of the first coding block relative to each of the regions of interest based on the first code control reference value and the second code control reference value includes: in response to the first coding block not being located in the target region of interest, determining a rectangular region corresponding to the target region of interest; determining the center point of the rectangular region and a first distance from the vertex to the center point; determining a second distance from the first coding block to the center point; determining that the first coding block is located in a quadrant with the center point as a reference; determining a third distance from the vertex of the video frame to be encoded in the quadrant to the center point; and determining the first quantization parameter adjustment value of the first coding block relative to the target region of interest based on the first distance, the second distance, the third distance, and the second code control reference value.

[0015] In some exemplary embodiments of this disclosure, the step of performing rate control on each coding block based on the quantization parameter adjustment value includes: obtaining a second quantization parameter adjustment value corresponding to a second coding block based on a block-level code control algorithm; the second coding block belongs to each coding block; determining whether the rate control direction of the second quantization parameter adjustment value is consistent with that of the quantization parameter adjustment value of the second coding block; in response to the consistent rate control direction, superimposing the second quantization parameter adjustment value with the quantization parameter adjustment value of the second coding block to perform rate control on the second coding block; in response to the inconsistent rate control direction, adjusting the second quantization parameter adjustment value based on an adjustment coefficient; and superimposing the adjusted second quantization parameter adjustment value with the quantization parameter adjustment value of the second coding block to perform rate control on the second coding block.

[0016] In some exemplary embodiments of this disclosure, determining the target of interest in a video frame to be encoded includes: obtaining region of interest (ROI) identifier information corresponding to the video frame to be encoded; the ROI identifier information is used to identify the at least one ROI; and determining the target of interest based on the ROI identifier information.

[0017] According to a second aspect of the present disclosure, a bitrate control apparatus based on a region of interest (ROI) is provided, comprising: a target determination module configured to determine a target of interest in a video frame to be encoded; a region determination module configured to determine at least one ROI and a non-ROI in the video frame to be encoded based on the target of interest; a bitrate estimation module configured to perform bitrate estimation on the video frame to be encoded to obtain a first estimated number of bits; a bitrate control reference value determination module configured to determine a first bitrate control reference value and a second bitrate control reference value corresponding to the ROI and the non-ROI, respectively, based on the first estimated number of bits; a quantization parameter adjustment value determination module configured to determine a quantization parameter adjustment value corresponding to each coding block in the video frame to be encoded based on the first bitrate control reference value and the second bitrate control reference value; and a bitrate control module configured to perform bitrate control on each coding block based on the quantization parameter adjustment value.

[0018] According to a third aspect of the present disclosure, an electronic device is provided, characterized in that it includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the bitrate control method based on region of interest as described in any one of the present disclosures.

[0019] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform any of the region-of-interest-based bitrate control methods described in the present disclosure.

[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program / instructions, characterized in that, when the computer program / instructions are executed by a processor, they implement the bitrate control method based on the region of interest as described in any one of the present disclosures.

[0021] The bitrate control method based on regions of interest (ROIs) provided in this disclosure identifies ROIs and non-ROIs in the video frame to be encoded. Based on the bitrate estimation of the video frame to be encoded, a first bitrate control reference value and a second bitrate control reference value are determined for each ROI and non-ROI, respectively. Based on these first and second bitrate control reference values, quantization parameter adjustment values ​​for each coding block are determined, and bitrate control is performed on each coding block based on these quantization parameter adjustment values. This method improves the subjective viewing experience of the video image by applying differentiated bitrate control to ROIs and non-ROIs. Determining the first and second bitrate control reference values ​​separately based on the bitrate estimation of the video frame ensures stable overall bitrate fluctuations in the video. Block-level quantization parameter adjustment improves overall image quality.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0024] Figure 1 A schematic diagram of an exemplary system architecture to which the methods of embodiments of this disclosure can be applied is shown.

[0025] Figure 2 This is a flowchart illustrating a bitrate control method based on a region of interest, according to an exemplary embodiment.

[0026] Figure 3 This is a flowchart illustrating a method for determining regions of interest and non-regions of interest according to an exemplary embodiment.

[0027] Figure 4A This is a flowchart illustrating a code control reference value determination method according to an exemplary embodiment. Figure 1 .

[0028] Figure 4B This is a flowchart illustrating a code control reference value determination method according to an exemplary embodiment. Figure 2 .

[0029] Figure 5A This is a flowchart illustrating a method for determining quantization parameter adjustment values ​​according to an exemplary embodiment.

[0030] Figure 5B This is a flowchart illustrating a method for determining the adjustment value of a quantization parameter within a region of interest, according to an exemplary embodiment.

[0031] Figure 5C This is a schematic diagram illustrating a method for determining the adjustment value of a quantization parameter within a region of interest, according to an exemplary embodiment.

[0032] Figure 5D This is a flowchart illustrating a method for determining quantization parameter adjustment values ​​outside the region of interest according to an exemplary embodiment.

[0033] Figure 5E This is a schematic diagram illustrating a method for determining the adjustment value of quantization parameters outside the region of interest according to an exemplary embodiment.

[0034] Figure 6 This is a flowchart illustrating a rate control method according to an exemplary embodiment.

[0035] Figure 7 This is a block diagram illustrating a bitrate control device based on a region of interest, according to an exemplary embodiment.

[0036] Figure 8 This is a schematic diagram illustrating the structure of an electronic device suitable for implementing exemplary embodiments of the present disclosure, according to an exemplary embodiment. Detailed Implementation

[0037] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0038] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0039] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in at least one hardware module or integrated circuit, or in different network and / or processor devices and / or microcontroller devices.

[0040] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0041] In this specification, the terms “a,” “an,” “the,” “the,” and “at least one” are used to indicate the presence of at least one element / component / etc.; the terms “comprising,” “including,” and “having” are used to indicate an open-ended inclusion and to mean that there may be other elements / components / etc. in addition to the listed elements / components / etc.; the terms “first,” “second,” and “third,” etc., are used only as markings and are not a limitation on the number of objects.

[0042] Figure 1 A schematic diagram of an exemplary system architecture to which the methods of embodiments of this disclosure can be applied is shown.

[0043] like Figure 1 As shown, the system architecture may include server 101, network 102, terminal device 103, terminal device 104, and terminal device 105. Network 102 serves as the medium for providing a communication link between terminal device 103, terminal device 104, or terminal device 105 and server 101. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0044] Server 101 can be a server that provides various services, such as a back-end management server that supports the devices operated by users using terminal devices 103, 104, or 105. The back-end management server can analyze and process received requests and other data, and feed back the processing results to terminal devices 103, 104, or 105.

[0045] Terminal devices 103, 104, and 105 can be smartphones, tablets, laptops, desktop computers, smart speakers, wearable smart devices, virtual reality devices, augmented reality devices, etc., but are not limited to these.

[0046] It should be understood that Figure 1 The number of terminal devices 103, 104, 105, network 102, and server 101 in the diagram is merely illustrative. Server 101 can be a single physical server, a server cluster consisting of multiple servers, or a cloud server. Depending on actual needs, it can have any number of terminal devices, networks, and servers.

[0047] The steps of the method in the exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings and examples.

[0048] Figure 2 This is a flowchart illustrating a bitrate control method based on a region of interest, according to an exemplary embodiment. Figure 2 The method provided in the embodiments can be executed by any electronic device, such as the one described above. Figure 1 Terminal devices in or Figure 1 The server in, or Figure 1 The terminal devices and servers in the process are executed together, but this disclosure does not limit this.

[0049] In step S210, the target of interest in the video frame to be encoded is determined.

[0050] In this embodiment of the disclosure, in a real-time transmission scenario, the information of regions of interest / non-interest is combined with the encoding to guide the bitrate allocation during the encoding process. Increasing the bitrate allocation to regions of interest reduces the relevant coding QP; decreasing the bitrate allocation to non-interest regions increases the relevant coding QP.

[0051] In this embodiment of the disclosure, a target of interest is determined in a video frame to be encoded. The video frame to be encoded is any video frame during video transmission. The target of interest is a target of interest within that video frame, such as a face region. It should be noted that there are various ways to determine the target of interest in the video frame to be encoded. This disclosure does not limit the specific method used.

[0052] In an exemplary embodiment, a facial recognition algorithm can be used to identify facial objects in the video frame to be encoded, and these facial objects can be used as the target of interest. Of course, other recognition algorithms can also be used to identify targets of interest in other scenarios. For example, in e-commerce scenarios, people pay close attention to products, and the identified products can be used as the target of interest.

[0053] In an exemplary embodiment, the target of interest can also be identified through other pre-processing steps; here, the target of interest is determined directly based on existing identification information. For example, the region of interest (ROI) identification information corresponding to the video frame to be encoded is obtained; the ROI identification information is used to identify the at least one ROI; and the target of interest is determined based on the ROI identification information.

[0054] This approach can be applied to the video transcoding side. On the transcoding side, the encoded video sent by the transmitter is received, enhanced, and transcoded before being sent back to the receiver. Therefore, the transmitting end may have already identified the target of interest (ROI) in the video frame to be encoded and written it into the Region of Interest (SEI) using region-of-interest (ROI) identifiers. Thus, the transcoding side does not need to re-identify the ROI and can directly reuse the RIO identifier information.

[0055] In step S220, based on the interest target, at least one region of interest and non-region of interest in the video frame to be encoded are determined.

[0056] In this embodiment of the disclosure, based on the determined interest target, regions of interest (ROIs) and regions of non-interest (ROIs) are marked in the video frame to be encoded. There can be one or more ROIs. Furthermore, these multiple ROIs can be independent of each other or overlap. Regions in the video frame to be encoded other than the ROIs are considered ROIs. By determining the ROIs and ROIs in the video frame to be encoded, differentiated bitrate control can be performed.

[0057] In step S230, the bitrate of the video frame to be encoded is estimated to obtain a first estimated number of bits.

[0058] In this embodiment of the disclosure, bitrate estimation is performed on a video frame to be encoded that has not undergone bitrate control to obtain a first estimated number of bits. This first estimated number of bits is an estimate of the number of bits consumed after encoding the video frame to be encoded. It should be noted that various bitrate estimation algorithms can be used to estimate the number of bits consumed by the video frame to be encoded. This disclosure does not limit the specific bitrate estimation algorithm used.

[0059] In an exemplary embodiment, the encoder can estimate the bitrate of the video frame to be encoded using the following formula.

[0060] I=(coeff*satd+offset) / (qscale*count)

[0061] Here, `coeff` is an adjustment factor used to adjust the impact of the SATD value on the final estimated bit count based on the characteristics of the video content. `Satd` (Sum of Absolute Transformed Differences) is used in video coding to evaluate the distortion of the residual block after motion compensation. `Offset` is the offset used to correct errors introduced by model simplification. `Qscale` (Quantization Scale) is a quantization parameter in video coding that determines the degree of data compression. A smaller `Qscale` value results in finer quantization, while a larger `Qscale` value results in coarser quantization. This `Qscale` is related to the quantization parameter `QP`. `Count` is the number of macroblocks or pixels. `I` is the first estimated bit count, representing the estimated number of bits consumed by the video frame to be encoded.

[0062] In step S240, based on the first estimated number of bits, a first code control reference value and a second code control reference value corresponding to the region of interest and the region of non-interest are determined respectively.

[0063] In this embodiment of the disclosure, to ensure that the overall bitrate fluctuation of the video remains as stable as possible after bitrate control based on the region of interest (ROI), this disclosure needs to balance the bitrate increase for the ROI and the bitrate decrease for the non-ROI, so that the number of bits consumed by the video frame to be encoded after bitrate control is close to the first estimated number of bits. Based on this, bitrate control reference values ​​corresponding to the ROI and non-ROI are determined according to the first estimated number of bits. The ROI corresponds to the first bitrate control reference value, and the non-ROI corresponds to the second bitrate control reference value. By adjusting and balancing the first and second bitrate control reference values, the number of bits consumed by the video frame to be encoded is close to the first estimated number of bits after bitrate control is performed on the ROI and non-ROI based on the relevant bitrate control reference values.

[0064] It should be noted that there are many ways to determine the first and second bitrate control reference values. As long as the bitrate fluctuation of the video remains stable after bitrate control is performed based on the relevant bitrate control reference values, they should all be considered to be within the scope of protection of this disclosure.

[0065] In step S250, the quantization parameter adjustment values ​​corresponding to each coding block in the video frame to be encoded are determined based on the first code control reference value and the second code control reference value.

[0066] In this embodiment, the first and second bitrate control reference values ​​are merely reference values ​​for bitrate control, used to ensure that the overall bitrate fluctuation of the video remains stable after bitrate control. However, directly controlling the bitrate of the entire region of interest or non-region of interest based on these reference values ​​would result in a rather abrupt transition in bitrate control and would not achieve a good video enhancement effect.

[0067] In this embodiment, based on the relevant bitrate control benchmark value, the quantization parameter adjustment value corresponding to each coding block in the video frame to be encoded is determined separately. By performing differentiated bitrate control on each coding block, the bitrate control optimization results are more accurate and efficient, thus improving the picture quality.

[0068] In step S260, the bitrate of each coding block is controlled based on the quantization parameter adjustment value.

[0069] The bitrate control method based on regions of interest (ROIs) provided in this disclosure identifies ROIs and non-ROIs in the video frame to be encoded. Based on the bitrate estimation of the video frame to be encoded, a first bitrate control reference value and a second bitrate control reference value are determined for each ROI and non-ROI, respectively. Based on these first and second bitrate control reference values, quantization parameter adjustment values ​​for each coding block are determined, and bitrate control is performed on each coding block based on these quantization parameter adjustment values. This method improves the subjective viewing experience of the video image by applying differentiated bitrate control to ROIs and non-ROIs. Determining the first and second bitrate control reference values ​​separately based on the bitrate estimation of the video frame ensures stable overall bitrate fluctuations in the video. Block-level quantization parameter adjustment improves overall image quality.

[0070] Figure 3 This is a flowchart illustrating a method for determining regions of interest and regions of non-interest according to an exemplary embodiment. In this disclosure embodiment, in... Figure 2 Based on the bitrate control method based on region of interest shown, the aforementioned step S220 may include the following steps.

[0071] In step S310, a binarized image is generated based on the target of interest; the binarized image is used to label the first region of interest and the first region of non-interest.

[0072] In this embodiment of the disclosure, a binarized image corresponding to the video frame to be encoded is generated based on the target of interest. This binarized image can be a mask image marked with two numerical values, 0 and 1. In this binarized image, pixels belonging to the region of interest are marked as 1 (or white), while pixels in the non-region of interest are marked as 0 (or black). Through this marking, the corresponding first region of interest and first non-region of interest can be determined.

[0073] In step S320, the first region of interest is subjected to etching to obtain the etched first region of interest.

[0074] In this embodiment, region of interest (ROI) detection sometimes misses even very small pedestrian faces, which are not actually regions of interest in the video. To remove these smaller, disconnected ROIs, the aforementioned first ROI is eroded to obtain an eroded first ROI. The erosion process involves shrinking the target region inward by a preset number of pixels along its edge, forming a smaller corresponding region. By eroding the first ROI, smaller, disconnected ROIs can be removed, while larger ROIs are retained.

[0075] In step S330, the first region of interest after corrosion is subjected to dilation processing to determine the at least one region of interest and non-region of interest.

[0076] In this embodiment, the first region of interest (ROI) after the aforementioned erosion process is further dilated. This dilation process is the reverse of the erosion process, expanding outwards along the edge of the target region by a predetermined number of pixels to form an enlarged corresponding region. It should be noted that if the ROI is small and has been removed by the aforementioned erosion process, it will not be expanded and restored during this dilation process; only the ROI retained after the erosion process will be dilated. Through erosion and dilation, smaller ROIs can be removed, and the original ROI can be restored. Based on the determined at least one ROI, other regions can be determined as non-ROIs.

[0077] In an exemplary embodiment, the magnitude of the dilation process is not less than the magnitude of the etching process. When the magnitude of the dilation process is equal to the magnitude of the etching process, the remaining regions of interest are restored after removing the smaller region of interest. When the magnitude of the dilation process is greater than the magnitude of the etching process, the remaining regions of interest are appropriately enlarged after removing the smaller region of interest.

[0078] The bitrate control method based on region of interest provided in this disclosure can remove smaller and disconnected regions of interest by performing erosion and then dilation on the region of interest, while retaining larger regions of interest, thereby optimizing the bitrate of the more important regions of interest and improving image quality.

[0079] In some embodiments of this disclosure, the determination of whether to perform bitrate control on the video frame to be encoded can also be based on the proportion of the area occupied by the region of interest in the entire video frame to be encoded. This may include the following steps.

[0080] Calculate the area of ​​the at least one region of interest.

[0081] Calculate the area percentage of the at least one region of interest in the video frame to be encoded.

[0082] In response to the area ratio being within a preset area ratio range, the bitrate control is performed on the video frame to be encoded.

[0083] In this embodiment, the area of ​​each region of interest (ROI) is first calculated. This area can be calculated using the number of pixels within the ROI. Based on the ROI's area, the percentage of that area in the video frame to be encoded is calculated. This percentage can be calculated using the area of ​​non-ROI regions or the total number of pixels in the entire video frame. When this percentage is within a preset range, bitrate control continues for the video frame. When the percentage is outside the preset range, the bitrate control process is skipped, and bitrate control is not applied to the video frame. This is because if the ROI's proportion in the video frame is too large or too small, the visual benefit of bitrate control is minimal; skipping the bitrate control process saves computational overhead.

[0084] In an exemplary embodiment, the area ratio range is set to A ∈ [0.05B, 2B]. Where A is the area of ​​the region of interest, and B is the area of ​​the region of non-interest.

[0085] The bitrate control method based on the region of interest (ROI) provided in this disclosure determines whether to control the bitrate of a video frame based on the area ratio of the ROI within that frame. This ensures that bitrate control is applied to video frames that show significant image quality improvement, while skipping video frames that do not, thus saving computational overhead.

[0086] Figure 4A This is a flowchart illustrating a code control reference value determination method according to an exemplary embodiment. Figure 1 In this embodiment of the disclosure, in Figure 2 Based on the bitrate control method based on region of interest shown, the aforementioned step S240 may include the following steps.

[0087] In step S410, a first code control setting value corresponding to the region of interest is set.

[0088] In this embodiment of the disclosure, a first code control setting value is first set for the region of interest. This first code control setting value can be a value set manually, a value calculated by a relevant algorithm, or a value obtained by gradually adjusting an initial value.

[0089] In an exemplary embodiment, in order to ensure that the overall bitrate of the video frame to be encoded remains stable, a limit value G can be preset for the first bitrate control setting value, where G is a negative number (e.g., -6). This ensures that the first bitrate control setting value is within a certain range.

[0090] In step S420, based on the first code control setting value, the code rate is estimated for the at least one region of interest to obtain a second estimated number of bits corresponding to the region of interest.

[0091] In this embodiment of the disclosure, based on the first code control setting value, a bitrate estimation is performed on the at least one region of interest to obtain a second estimated number of bits corresponding to the region of interest. This second estimated number of bits is the estimated number of bits required by the region of interest after bitrate control is performed on the region of interest based on the first code control setting value. The bitrate estimation is similar to the bitrate estimation algorithm in step S230 described above, and will not be repeated here.

[0092] In an exemplary embodiment, the encoder can estimate the bitrate for at least one region of interest using the following formula.

[0093] K=(coeff*E+offset) / ((qscale+J)*count)

[0094] Where J is the first code control setting value, E is the first complexity corresponding to the region of interest, and K is the second estimated number of bits.

[0095] In an exemplary embodiment, a first complexity E corresponding to at least one region of interest can be obtained by performing complexity statistics on at least one region of interest. This first complexity E can be the average of the complexities of each region of interest, or it can be the sum of the complexities of each region of interest. It should be noted that there are many algorithms for statistically calculating image complexity, and this disclosure does not limit the specific algorithm used.

[0096] Based on the first complexity E obtained from the statistics and the first code control setting value J set above, the second estimated number of bits K can be calculated based on the above formula.

[0097] In step S430, a third estimated number of bits corresponding to the non-interest region is calculated based on the first estimated number of bits and the second estimated number of bits.

[0098] As mentioned earlier, we aim to ensure that the overall bitrate fluctuation of the video remains stable after bitrate control. Therefore, we hope that the sum of the number of bits in the region of interest (ROI) and the number of bits in the non-ROI region in the video frame to be encoded after bitrate control is close to the first estimated number of bits before bitrate control. Therefore, in this embodiment, based on the first estimated number of bits I and the second estimated number of bits K, a third estimated number of bits IK corresponding to the non-ROI region is calculated. That is, after ensuring the number of bits in the ROI region after bitrate enhancement, there are still bits available for the non-ROI region.

[0099] In step S440, a second code control setting value corresponding to the non-interest region is calculated based on the third estimated number of bits.

[0100] In this embodiment of the disclosure, a second code control setting value corresponding to the non-interest region is calculated based on the third estimated number of bits. This calculation algorithm is the inverse operation of the aforementioned code rate estimation, and therefore the code rate estimation algorithm in step S230 can be referred to.

[0101] In an exemplary embodiment, the encoder can calculate the second code control setting value using the following formula.

[0102] L=((coeff*F+offset) / ((IK)*count))-qscale

[0103] Where IK is the third estimated number of bits, F is the second complexity corresponding to the non-interest region, and L is the second code control setting value.

[0104] In an exemplary embodiment, a second complexity F corresponding to the non-interested region can be obtained by performing complexity statistics on the non-interested region. This second complexity F can be the average of the complexities of the non-interested regions or the cumulative value of the complexities of the non-interested regions.

[0105] Based on the second complexity F obtained from the statistics and the aforementioned third estimated number of bits IK, the second code control setting value L can be calculated based on the above formula.

[0106] In this embodiment, a first code control setting value corresponding to the region of interest (ROI) is first determined, and then a second estimated number of bits for the ROI is calculated. To ensure that the sum of the number of bits in the ROI and the number of bits in the non-ROI after code rate control is close to the first estimated number of bits, a third estimated number of bits corresponding to the non-ROI is calculated based on the first and second estimated number of bits. Then, based on this third estimated number of bits and the inverse operation of code rate estimation, the second code control setting value corresponding to the non-ROI is calculated.

[0107] In step S450, the first code control setting value is determined to be the first code control reference value, and the second code control setting value is determined to be the second code control reference value.

[0108] In this embodiment of the disclosure, the first code control setting value is determined as a first code control reference value and the second code control setting value is determined as a second code control reference value through the above steps. In this way, bitrate control of the video frame to be encoded based on the first code control reference value and the second code control reference value can ensure that the overall bitrate fluctuation of the video remains stable.

[0109] Figure 4B This is a flowchart illustrating a code control reference value determination method according to an exemplary embodiment. Figure 2 .

[0110] In this embodiment of the disclosure, Figure 4B In the code control reference value determination method shown, steps S410, S420, S430, and S440 are respectively related to... Figure 4A Steps S410, S420, S430, and S440 in the code control reference value determination method shown correspond to each other and will not be repeated here.

[0111] In this embodiment of the disclosure, Figure 4A Based on the code control reference value determination method shown, the aforementioned step S450 may include the following steps.

[0112] In step S451, it is determined whether the second code control setting value is greater than the non-interest area adjustment threshold.

[0113] Figure 4A While the illustrated embodiment can ensure that the sum of the number of bits in the region of interest (ROI) and the number of bits in the non-ROI after bitrate control is close to the first estimated number of bits, thus ensuring stable overall video bitrate fluctuations, an unreasonable first bitrate control setting can lead to an excessive increase in the bitrate of the ROI, which cannot be compensated for by reducing the bitrate of the non-ROI. This would result in excessive overall bitrate fluctuations, thus requiring a reduction in the bitrate allocation for the ROI.

[0114] In this embodiment of the disclosure, in order to ensure that the bitrate reduction in non-interested regions is not too large, a non-interested region adjustment threshold H is set for the non-interested regions. That is, the bitrate adjustment for non-interested regions must not exceed this threshold H.

[0115] Based on this, the second code control setting value calculated in the aforementioned step S440 is compared with the non-interest region adjustment threshold H to determine whether the second code control setting value is greater than the non-interest region adjustment threshold H.

[0116] In step S452, in response to the second code control setting value being greater than the non-interest region adjustment threshold, the first code control setting value is adjusted, and the second code control setting value is recalculated.

[0117] In this embodiment of the disclosure, when the second code control setting value L is greater than the non-interest region adjustment threshold H, it indicates that the bitrate of the non-interest region has decreased too much, and the first code control setting value J set in the aforementioned step S410 needs to be adjusted. Based on the adjusted first code control setting value J, and based on the aforementioned... Figure 4A The calculation process shown recalculates the second code control setting value L so that the second code control setting value L is not greater than the non-interest region adjustment threshold H.

[0118] In an exemplary embodiment, the first code control setting value J can be adjusted in a step-by-step manner. That is, the first code control setting value J is first set to an initial value. This initial value can be the aforementioned limit value G. When the second code control setting value L calculated based on the initial value is greater than the non-interest region adjustment threshold H, the first code control setting value J is adjusted in a step-by-step manner. The first code control setting value J is adjusted in this step-by-step manner in a cyclical manner until the second code control setting value L is no greater than the non-interest region adjustment threshold H.

[0119] In step S453, in response to the second code control setting value not being greater than the non-interest region adjustment threshold, the first code control setting value is determined to be the first code control reference value, and the second code control setting value is determined to be the second code control reference value.

[0120] In this embodiment of the disclosure, when the second code control setting value L is not greater than the adjustment threshold H of the non-interest region, it indicates that the reduction in bit rate of the non-interest region is reasonable, and it can be determined that the current first code control setting value is the first code control reference value and the current second code control setting value is the second code control reference value.

[0121] The bitrate control method based on region of interest provided in this embodiment obtains a first estimated number of bits by estimating the bitrate of the video frame to be encoded, and determines a first bitrate control reference value and a second bitrate control reference value based on the first estimated number of bits, so that the number of bits consumed by the video frame to be encoded after bitrate control is close to the first estimated number of bits, thereby ensuring that the overall bitrate fluctuation of the video remains stable.

[0122] Figure 5A This is a flowchart illustrating a method for determining quantization parameter adjustment values ​​according to an exemplary embodiment. In this disclosure, in... Figure 2 Based on the bitrate control method based on region of interest shown, the aforementioned step S250 may include the following steps.

[0123] In step S510, a first coding block in the video frame to be encoded is determined; the first coding block belongs to each of the coding blocks.

[0124] As previously stated, this disclosure determines the quantization parameter adjustment value for each coding block in the video frame to be encoded, based on the relevant bitrate control benchmark value. By applying differentiated bitrate control to each coding block, the bitrate control optimization results are more accurate and efficient, thus improving image quality.

[0125] In this embodiment of the disclosure, the quantization parameter adjustment value corresponding to each coding block is determined one by one. First, the first coding block in the video frame to be encoded is determined. The first coding block can be any coding block in the video frame to be encoded.

[0126] In step S520, a first quantization parameter adjustment value of the first coding block relative to each of the regions of interest is determined based on the first code control reference value and the second code control reference value.

[0127] As mentioned earlier, there may be multiple regions of interest (ROIs) in the video frame to be encoded. Based on the relative positional relationship between the first coding block and each ROI, a first quantization parameter adjustment value is determined relative to each ROI. This first quantization parameter adjustment value is a rate control adjustment value determined based on the relative positional relationship between the first coding block and the target ROI.

[0128] As mentioned earlier, in order to ensure that the overall video bitrate fluctuation remains stable, the first bitrate control reference value and the second bitrate control reference value determined above need to be referenced when determining the adjustment value of the first quantization parameter, so that the determined adjustment value of the first quantization parameter is within a reasonable range and will not cause excessive fluctuation in the overall bitrate.

[0129] In step S530, the minimum value among the various first quantization parameter adjustment values ​​is determined to be the quantization parameter adjustment value corresponding to the first coding block.

[0130] As mentioned earlier, multiple regions of interest (ROIs) may exist in the video frame to be encoded. Therefore, according to step S520, several first quantization parameter adjustment values ​​for the first coding block relative to each ROI can be obtained. Since a negative quantization parameter adjustment value means increasing the bitrate of the coding block, and a positive value means decreasing the bitrate of the coding block, that is, the smaller the quantization parameter adjustment value, the higher the bitrate of the coding block. Based on this, the minimum value among the several first quantization parameter adjustment values ​​is selected as the quantization parameter adjustment value corresponding to the first coding block, so as to control the bitrate of the first coding block based on the quantization parameter adjustment value.

[0131] Figure 5BThis is a flowchart illustrating a method for determining the adjustment value of a quantization parameter within a region of interest, according to an exemplary embodiment. Figure 5C This is a schematic diagram illustrating a method for determining quantization parameter adjustment values ​​within a region of interest according to an exemplary embodiment. In this embodiment, in Figure 5A Based on the method for determining the quantization parameter adjustment value shown, the aforementioned step S520 may include the following steps.

[0132] In step S521, in response to the first coding block being located in the target region of interest, a rectangular region corresponding to the target region of interest is determined.

[0133] As mentioned earlier, the first quantization parameter adjustment value of the first coded block relative to each region of interest is determined based on the relative positional relationship between the first coded block and each region of interest. Since the first coded block may be located within or outside the region of interest, differentiated processing is required depending on whether the first coded block is located within or outside the region of interest.

[0134] In this embodiment of the disclosure, when the first coding block is located within a target region of interest (ROI), a rectangular region corresponding to the target ROI is determined. The target ROI can be any one of multiple ROIs in the video frame to be encoded. Since the ROI is typically an irregular shape, a rectangular region corresponding to the target ROI is first determined for ease of calculation.

[0135] In an exemplary embodiment, the rectangular region is an external rectangular region based on the target region of interest.

[0136] In step S522, the center point of the rectangular region and the first distance from the vertex to the center point are determined.

[0137] In this embodiment of the disclosure, based on the rectangular region, the center point N of the rectangular region and the first distance R0 between the vertex of the rectangular region and the center point N are determined.

[0138] In step S523, a second distance from the first coded block to the center point is determined.

[0139] In this embodiment of the disclosure, a second distance R1 is determined from the first coding block to the center point N of the rectangular region based on the rectangular region. Specifically, the second distance R1 can be the distance between the center point of the first coding block and the center point N of the rectangular region.

[0140] In step S524, the first quantization parameter adjustment value of the first coding block relative to the target region of interest is determined based on the first distance, the second distance, and the first code control reference value.

[0141] In this embodiment, the first quantization parameter adjustment value of the first coding block relative to the target region of interest is determined based on the first distance R0, the second distance R1, and the aforementioned determined first code control reference value J. This disclosure only indicates that the first quantization parameter adjustment value can be determined by referring to the aforementioned first distance R0, second distance R1, and first code control reference value J. The specific calculation algorithm can be adjusted according to actual control needs, and this disclosure does not limit the specific adjustment algorithm.

[0142] In an exemplary embodiment, the adjustment value of the first quantization parameter can be calculated using the following formula.

[0143] roi_qpoffset=(R0-R1) / R1*J

[0144] Wherein, roi_qpoffset is the first quantization parameter adjustment value within the region of interest, R0 is the first distance, R1 is the second distance, and J is the first code control reference value determined in the aforementioned step S240.

[0145] Figure 5D This is a flowchart illustrating a method for determining quantization parameter adjustment values ​​outside the region of interest according to an exemplary embodiment. Figure 5E This is a schematic diagram illustrating a method for determining quantization parameter adjustment values ​​outside the region of interest according to an exemplary embodiment. In this embodiment of the disclosure, in Figure 5A Based on the method for determining the quantization parameter adjustment value shown, the aforementioned step S520 may include the following steps.

[0146] In step S525, in response to the first coding block not being located in the target region of interest, a rectangular region corresponding to the target region of interest is determined.

[0147] In this embodiment of the disclosure, when the first coding block is located outside the target region of interest, a rectangular region corresponding to the target region of interest is determined. The target region of interest is any one of the multiple regions of interest in the video frame to be encoded. Since the region of interest is usually an irregular shape, a rectangular region corresponding to the target region of interest is first determined for ease of calculation.

[0148] In an exemplary embodiment, the rectangular region is an external rectangular region based on the target region of interest.

[0149] In step S526, the center point of the rectangular region and the first distance from the vertex to the center point are determined.

[0150] In this embodiment of the disclosure, based on the rectangular region, the center point N of the rectangular region and the first distance R0 between the vertex of the rectangular region and the center point N are determined.

[0151] In step S527, a second distance from the first coded block to the center point is determined.

[0152] In this embodiment of the disclosure, a second distance R1 is determined from the first coding block to the center point N of the rectangular region based on the rectangular region. Specifically, the second distance R1 can be the distance between the center point of the first coding block and the center point N of the rectangular region.

[0153] In step S528, it is determined that the first coded block is located in a quadrant with the center point as the reference.

[0154] Since the non-interested region directly uses the aforementioned Figure 5B The algorithm for the region of interest shown may result in some areas not being covered, or the area being too large, exceeding the entire image area, leading to inaccurate estimations. Therefore, a four-quadrant method is used to adjust non-interest regions. That is, the adjustment value of the first quantization parameter is determined based on the quadrant direction of the first coding block within the target region of interest.

[0155] In this embodiment of the disclosure, the video frame to be encoded is divided into four quadrants, namely the upper right quadrant, the upper left quadrant, the lower left quadrant, and the lower right quadrant, based on the center point N of the target region of interest. The corresponding quadrant in which the first coding block is located is determined according to the four quadrants.

[0156] In step S529, the third distance from the vertex of the video frame to be encoded in the quadrant to the center point is determined.

[0157] In this embodiment of the disclosure, based on the aforementioned four quadrants, it can be known that each quadrant corresponds to a vertex of the video frame to be encoded. According to the quadrant in which the first coding block is located, the distance between the vertex of the video frame to be encoded in that quadrant and the center point N is determined as the third distance R2.

[0158] In step S5210, the first quantization parameter adjustment value of the first coding block relative to the target region of interest is determined based on the first distance, the second distance, the third distance, and the second code control reference value.

[0159] In this embodiment, the first quantization parameter adjustment value of the first coding block relative to the target region of interest is determined based on the first distance R0, the second distance R1, the third distance R2, and the aforementioned determined second code control reference value L. This disclosure only indicates that the first quantization parameter adjustment value can be determined by referring to the aforementioned first distance R0, second distance R1, third distance R2, and second code control reference value L. The specific calculation algorithm can be adjusted according to actual control needs, and this disclosure does not limit the specific adjustment algorithm.

[0160] In an exemplary embodiment, the adjustment value of the first quantization parameter can be calculated using the following formula.

[0161] noroi_qpoffset=MAX(0,R1-R0) / (R2-R0)*L

[0162] Wherein, noroi_qpoffset is the first quantization parameter adjustment value outside the region of interest, R0 is the first distance, R1 is the second distance, and J is the first code control reference value determined in the aforementioned step S240.

[0163] The bitrate control method based on regions of interest (ROIs) provided in this disclosure calculates the bitrate for each coded block in the video frame to be encoded relative to each ROI. Based on the relative positional relationship between the first coded block and each ROI, a first quantization parameter adjustment value is determined for the first coded block relative to each ROI, thereby determining the quantization parameter adjustment value for the first coded block. By adjusting the quantization parameters at the block level, differentiated bitrate control can be applied to different coded blocks, thereby improving the overall image quality.

[0164] Figure 6 This is a flowchart illustrating a rate control method according to an exemplary embodiment. In this disclosure embodiment, in Figure 2 Based on the bitrate control method based on region of interest shown, the aforementioned step S260 may include the following steps.

[0165] In step S610, based on the block-level code control algorithm, the second quantization parameter adjustment value corresponding to the second coding block is obtained; the second coding block belongs to each of the coding blocks.

[0166] Since other bitrate control algorithms can be used to control the bitrate of the video frames to be encoded, in addition to the bitrate control methods described above, it is necessary to fuse the quantization parameter adjustment values ​​calculated in step S250 with the quantization parameter adjustment values ​​obtained from other bitrate control algorithms in order to ensure compatibility with different bitrate control algorithms.

[0167] In this embodiment of the disclosure, a second quantization parameter adjustment value corresponding to a second coding block is obtained based on a block-level code control algorithm. The second coding block can be any coding block in the video frame to be encoded. A block-level code control algorithm refers to a technique used in video encoding to perform differentiated processing on different regions of an image (usually macroblocks or smaller blocks) to optimize overall coding efficiency and visual quality. These algorithms achieve differentiated bitrate control for different regions by adjusting the quantization parameters (QP) of different blocks. Examples include AQ (Adaptive Quantization) and CUTREE (Coding Unit Tree Rate Estimation and Encoding) algorithms. The second quantization parameter adjustment value is the quantization parameter adjustment value calculated based on this block-level code control algorithm.

[0168] In step S620, it is determined whether the rate control direction of the second quantization parameter adjustment value is consistent with that of the second coding block quantization parameter adjustment value.

[0169] In this embodiment of the disclosure, it is determined whether the rate control direction of the second quantization parameter adjustment value corresponding to the same coding block (such as the second coding block) is consistent with that of the quantization parameter adjustment value calculated in the aforementioned step S250. The rate control direction refers to either increasing or decreasing the code rate of the coding block. This is reflected in the quantization parameter adjustment value, i.e., both quantization parameter adjustment values ​​are either positive or both are negative.

[0170] In step S630, in response to the consistent bit rate control direction, the second quantization parameter adjustment value is superimposed with the quantization parameter adjustment value of the second coding block to perform bit rate control on the second coding block.

[0171] In this embodiment of the disclosure, in response to the consistent direction of the rate control, the two quantization parameter adjustment values ​​are superimposed, and the rate control of the second coding block is performed based on the superimposed quantization parameter adjustment value.

[0172] In step S640, in response to the inconsistency in the bit rate control direction, the adjustment value of the second quantization parameter is adjusted based on the adjustment coefficient.

[0173] In this embodiment of the disclosure, in response to the inconsistent direction of the bit rate control, that is, one quantization parameter adjustment value is positive and the other quantization parameter adjustment value is negative, the second quantization parameter adjustment value is adjusted based on a preset adjustment coefficient.

[0174] In an exemplary embodiment, the adjusted value of the second quantization parameter is multiplied by the adjustment coefficient to obtain the adjusted value of the second quantization parameter. For example, the adjustment coefficient is 0.5.

[0175] In step S650, the adjusted second quantization parameter adjustment value is superimposed with the quantization parameter adjustment value of the second coding block to perform bitrate control on the second coding block.

[0176] In this embodiment of the disclosure, the adjusted second quantization parameter adjustment value is superimposed with the corresponding quantization parameter adjustment value, and the bitrate is controlled based on the superimposed quantization parameter adjustment value for the second coding block.

[0177] The bitrate control method based on region of interest provided in this disclosure considers that, in addition to the bitrate control method described above, other bitrate control algorithms can also be used to control the bitrate of the video frames to be encoded during the bitrate control process. To ensure compatibility with different bitrate control algorithms, a fusion scheme for obtaining quantization parameter adjustment values ​​from other bitrate control algorithms is designed to achieve compatibility with other bitrate control algorithms.

[0178] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0179] Figure 7 This is a block diagram illustrating a region-of-interest (ROI) based bitrate control device according to an exemplary embodiment. (Refer to...) Figure 7 The bitrate control device 700 based on region of interest may include: a target determination module 710, a region determination module 720, a bitrate estimation module 730, a bitrate control reference value determination module 740, a quantization parameter adjustment value determination module 750, and a bitrate control module 760.

[0180] The target determination module 710 is configured to determine the target of interest in the video frame to be encoded.

[0181] The region determination module 720 is configured to determine at least one region of interest and non-region of interest in the video frame to be encoded based on the interest target.

[0182] The bitrate estimation module 730 is configured to perform bitrate estimation on the video frame to be encoded to obtain a first estimated number of bits.

[0183] The code control reference value determination module 740 is configured to determine a first code control reference value and a second code control reference value corresponding to the region of interest and the region of non-interest, respectively, based on the first estimated number of bits.

[0184] The quantization parameter adjustment value determination module 750 is configured to determine the quantization parameter adjustment value corresponding to each coding block in the video frame to be encoded based on the first code control reference value and the second code control reference value.

[0185] The rate control module 760 is configured to perform rate control on each coding block based on the quantization parameter adjustment value.

[0186] In some exemplary embodiments of this disclosure, the region determination module 720 is further configured to generate a binarized image based on the target of interest; the binarized image is used to label the first region of interest and a first region of non-interest; the first region of interest is subjected to erosion processing to obtain an eroded first region of interest; the eroded first region of interest is subjected to dilation processing to determine the at least one region of interest and a region of non-interest.

[0187] In some exemplary embodiments of this disclosure, the region determination module 720 is further configured such that the magnitude of the expansion treatment is not less than the magnitude of the corrosion treatment.

[0188] In some exemplary embodiments of this disclosure, the region determination module 720 is further configured to: count the area of ​​the at least one region of interest; calculate the area percentage of the at least one region of interest in the video frame to be encoded; and, in response to the area percentage being within a preset area percentage range, perform the bitrate control on the video frame to be encoded.

[0189] In some exemplary embodiments of this disclosure, the code control reference value determination module 740 is further configured to: set a first code control setting value corresponding to the region of interest; perform code rate estimation on the at least one region of interest based on the first code control setting value to obtain a second estimated number of bits corresponding to the region of interest; calculate a third estimated number of bits corresponding to the region of non-interest based on the first estimated number of bits and the second estimated number of bits; calculate a second code control setting value corresponding to the region of non-interest based on the third estimated number of bits; and determine the first code control setting value as the first code control reference value and the second code control setting value as the second code control reference value.

[0190] In some exemplary embodiments of this disclosure, the code control reference value determination module 740 is further configured to determine whether the second code control setting value is greater than the non-interest region adjustment threshold; in response to the second code control setting value being greater than the non-interest region adjustment threshold, adjust the first code control setting value and recalculate the second code control setting value; in response to the second code control setting value not being greater than the non-interest region adjustment threshold, determine that the first code control setting value is the first code control reference value and the second code control setting value is the second code control reference value.

[0191] In some exemplary embodiments of this disclosure, the code control reference value determination module 740 is further configured to obtain a first complexity and a second complexity corresponding to the at least one region of interest and a region of non-interest, respectively; calculate the second estimated number of bits based on the first complexity and the first code control setting value; and calculate the second code control setting value based on the second complexity and the third estimated number of bits.

[0192] In some exemplary embodiments of this disclosure, the quantization parameter adjustment value determination module 750 is further configured to determine a first coding block in the video frame to be encoded; the first coding block belongs to each of the coding blocks; determine a first quantization parameter adjustment value of the first coding block relative to each of the regions of interest based on the first code control reference value and the second code control reference value; and determine that the minimum value among the first quantization parameter adjustment values ​​is the quantization parameter adjustment value corresponding to the first coding block.

[0193] In some exemplary embodiments of this disclosure, the quantization parameter adjustment value determination module 750 is further configured to, in response to the first coding block being located in a target region of interest, determine a rectangular region corresponding to the target region of interest; determine the center point of the rectangular region and a first distance from the vertex to the center point; determine a second distance from the first coding block to the center point; and determine the first quantization parameter adjustment value of the first coding block relative to the target region of interest based on the first distance, the second distance, and a first code control reference value.

[0194] In some exemplary embodiments of this disclosure, the quantization parameter adjustment value determination module 750 is further configured to, in response to the first coding block not being located in the target region of interest, determine a rectangular region corresponding to the target region of interest; determine the center point of the rectangular region and a first distance from the vertex to the center point; determine a second distance from the first coding block to the center point; determine that the first coding block is located in a quadrant with the center point as a reference; determine a third distance from the vertex of the video frame to be encoded in the quadrant to the center point; and determine the first quantization parameter adjustment value of the first coding block relative to the target region of interest based on the first distance, the second distance, the third distance, and a second code control reference value.

[0195] In some exemplary embodiments of this disclosure, the rate control module 750 is further configured to: obtain a second quantization parameter adjustment value corresponding to a second coding block based on a block-level code control algorithm; the second coding block belongs to the respective coding blocks; determine whether the rate control direction of the second quantization parameter adjustment value is consistent with the rate control direction of the quantization parameter adjustment value of the second coding block; in response to the consistent rate control direction, superimpose the second quantization parameter adjustment value with the quantization parameter adjustment value of the second coding block to perform rate control on the second coding block; in response to the inconsistent rate control direction, adjust the second quantization parameter adjustment value based on an adjustment coefficient; and superimpose the adjusted second quantization parameter adjustment value with the quantization parameter adjustment value of the second coding block to perform rate control on the second coding block.

[0196] In some exemplary embodiments of this disclosure, the target determination module 710 is further configured to acquire region of interest (ROI) identification information corresponding to the video frame to be encoded; the ROI identification information is used to identify the at least one ROI; and the ROI identification information is used to determine the interest target.

[0197] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0198] The following reference Figure 8 To describe an electronic device 800 according to such an embodiment of the present disclosure. Figure 8 The electronic device 800 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0199] like Figure 8 As shown, the electronic device 800 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, a bus 830 connecting different system components (including storage unit 820 and processing unit 810), and a display unit 840.

[0200] The storage unit stores program code, which can be executed by the processing unit 810 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 810 can perform actions such as... Figure 2 The steps shown.

[0201] For example, electronic devices can achieve such Figure 2 The steps shown.

[0202] Storage unit 820 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 821 and / or cache memory 822, and may further include a read-only memory (ROM) 823.

[0203] The storage unit 820 may also include a program / utility 824 having a set (at least one) of program modules 825, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0204] Bus 830 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0205] Electronic device 800 can also communicate with one or more external devices 870 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 800, and / or with any device that enables electronic device 800 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 850. Furthermore, electronic device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 860. As shown, network adapter 860 communicates with other modules of electronic device 800 via bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0206] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0207] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions that can be executed by a processor of the device to perform the described method. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0208] In an exemplary embodiment, a computer program product is also provided, including a computer program / instructions that, when executed by a processor, implement the methods described above.

[0209] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0210] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A bitrate control method based on region of interest, characterized in that, include: Identify the target of interest in the video frame to be encoded; Based on the stated interest target, at least one region of interest and non-region of interest are determined in the video frame to be encoded; The bitrate of the video frame to be encoded is estimated to obtain the first estimated number of bits; Based on the first estimated number of bits, a first code control reference value and a second code control reference value corresponding to the region of interest and the region of non-interest are determined respectively; Based on the first code control reference value and the second code control reference value, determine the quantization parameter adjustment value corresponding to each coding block in the video frame to be encoded; The bitrate of each coding block is controlled based on the quantization parameter adjustment value. The step of determining the quantization parameter adjustment values ​​corresponding to each coding block in the video frame to be encoded based on the first code control reference value and the second code control reference value includes: Determine the first coded block in the video frame to be encoded; the first coded block belongs to each of the coded blocks. Based on the first code control reference value and the second code control reference value, determine the first quantization parameter adjustment value of the first coding block relative to each of the regions of interest; The minimum value among the various first quantization parameter adjustment values ​​is determined to be the quantization parameter adjustment value corresponding to the first coding block.

2. The method according to claim 1, characterized in that, The step of determining at least one region of interest and non-region of interest in the video frame to be encoded based on the interest target includes: Based on the stated interest target, a binarized image is generated; the binarized image is used to identify the first region of interest and the first region of non-interest. The first region of interest is subjected to etchation to obtain the etched first region of interest. The first region of interest after corrosion is expanded to determine the at least one region of interest and a region of non-interest. The extent of the expansion treatment is not less than the extent of the corrosion treatment.

3. The method according to claim 1, characterized in that, The method further includes: Calculate the area of ​​the at least one region of interest; Calculate the area percentage of the at least one region of interest in the video frame to be encoded; In response to the area ratio being within a preset area ratio range, the bitrate control is performed on the video frame to be encoded.

4. The method according to claim 1, characterized in that, The step of determining the first code control reference value and the second code control reference value corresponding to the region of interest and the non-region of interest, respectively, based on the first estimated number of bits, includes: Set a first code control setting value corresponding to the region of interest; Based on the first code control setting value, the code rate is estimated for the at least one region of interest to obtain a second estimated number of bits corresponding to the region of interest; Based on the first and second estimated bit counts, calculate the third estimated bit count corresponding to the non-interest region; Based on the third estimated number of bits, calculate the second code control setting value corresponding to the non-interest region; The first code control setting value is determined to be the first code control reference value, and the second code control setting value is determined to be the second code control reference value.

5. The method according to claim 4, characterized in that, Determining the first code control setting value as the first code control reference value and the second code control setting value as the second code control reference value includes: Determine whether the second code control setting value is greater than the non-interest area adjustment threshold; In response to the second code control setting value being greater than the non-interest region adjustment threshold, the first code control setting value is adjusted, and the second code control setting value is recalculated; In response to the second code control setting value not being greater than the non-interest region adjustment threshold, the first code control setting value is determined to be the first code control reference value, and the second code control setting value is determined to be the second code control reference value.

6. The method according to claim 4, characterized in that, The method further includes: Obtain the first complexity and the second complexity corresponding to the at least one region of interest and the region of non-interest, respectively; Based on the first complexity and the first code control setting value, calculate the second estimated number of bits; The second code control setting value is calculated based on the second complexity and the third estimated number of bits.

7. The method according to claim 1, characterized in that, The step of determining the first quantization parameter adjustment value of the first coding block relative to each of the regions of interest based on the first code control reference value and the second code control reference value includes: In response to the first coding block being located in the target region of interest, a rectangular region corresponding to the target region of interest is determined; Determine the center point of the rectangular region and the first distance from the vertex to the center point; Determine the second distance from the first coded block to the center point; Based on the first distance, the second distance, and the first code control reference value, the first quantization parameter adjustment value of the first coding block relative to the target region of interest is determined.

8. The method according to claim 1, characterized in that, The step of determining the first quantization parameter adjustment value of the first coding block relative to each of the regions of interest based on the first code control reference value and the second code control reference value includes: In response to the first coded block not being located in the target region of interest, a rectangular region corresponding to the target region of interest is determined; Determine the center point of the rectangular region and the first distance from the vertex to the center point; Determine the second distance from the first coded block to the center point; The first coded block is determined to be located in a quadrant with the center point as the reference. Determine the third distance from the vertex of the video frame to be encoded in the quadrant to the center point; Based on the first distance, the second distance, the third distance, and the second code control reference value, the first quantization parameter adjustment value of the first coding block relative to the target region of interest is determined.

9. The method according to claim 1, characterized in that, The step of controlling the bitrate of each coding block based on the quantization parameter adjustment value includes: Based on a block-level code control algorithm, a second quantization parameter adjustment value corresponding to the second coding block is obtained; the second coding block belongs to each of the coding blocks. Determine whether the rate control direction of the second quantization parameter adjustment value is consistent with that of the second coding block quantization parameter adjustment value; In response to the consistent bitrate control direction, the second quantization parameter adjustment value is superimposed with the quantization parameter adjustment value of the second coding block to perform bitrate control on the second coding block; In response to the inconsistency in the bit rate control direction, the adjustment value of the second quantization parameter is adjusted based on the adjustment coefficient; The adjusted second quantization parameter value is superimposed on the quantization parameter adjustment value of the second coding block to perform bitrate control on the second coding block.

10. The method according to claim 1, characterized in that, The process of determining the target of interest in the video frame to be encoded includes: Obtain the region of interest (ROI) identifier information corresponding to the video frame to be encoded; the ROI identifier information is used to identify the at least one ROI. The target of interest is determined based on the region of interest identification information.

11. A bitrate control device based on a region of interest, characterized in that, include: The target determination module is configured to determine the target of interest in the video frame to be encoded. The region determination module is configured to determine at least one region of interest and non-region of interest in the video frame to be encoded based on the interest target; The bitrate estimation module is configured to perform bitrate estimation on the video frame to be encoded to obtain a first estimated number of bits. The code control reference value determination module is configured to determine a first code control reference value and a second code control reference value corresponding to the region of interest and the region of non-interest, respectively, based on the first estimated number of bits. The quantization parameter adjustment value determination module is configured to determine the quantization parameter adjustment value corresponding to each coding block in the video frame to be encoded based on the first code control reference value and the second code control reference value. The bitrate control module is configured to perform bitrate control on each coding block based on the quantization parameter adjustment value; The quantization parameter adjustment value determination module is further configured to determine the first coding block in the video frame to be encoded; The first coded block belongs to each of the coded blocks; Based on the first code control reference value and the second code control reference value, a first quantization parameter adjustment value for the first coding block relative to each of the regions of interest is determined; the minimum value among the first quantization parameter adjustment values ​​is determined to be the quantization parameter adjustment value corresponding to the first coding block.

12. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the executable instructions to implement the bitrate control method based on the region of interest as described in any one of claims 1 to 10.

13. A computer-readable storage medium, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the bitrate control method based on a region of interest as described in any one of claims 1 to 10.

14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the bitrate control method based on the region of interest as described in any one of claims 1 to 10.

Images