A ROI-based encoding method

Abstract

The present invention relates to the technical field of digital video coding and discloses a coding method based on ROI. The method comprises the following steps: S1: obtaining a ROI area ratio of a video to be processed, and pre-training a quality parameter curve of the video to be processed according to a relationship between a quality adjustment parameter of the video to be processed and a bit rate change; S2: sending a current frame of the video to be processed to a visual saliency model for detection to obtain ROI region coordinates and ROI region area ratio; S3: obtaining ROI quality adjustment parameters of the current frame according to a preset target bit rate change rate and a quality parameter curve suitable for the current frame; S4: obtaining a non-ROI region quality adjustment parameter by calculating a distance between a non-ROI region and a center coordinate of the ROI region; and S5: setting the ROI region coordinates and a non-ROI region, as well as the ROI region and non-ROI region quality adjustment parameters of the current frame, and sending the ROI region and non-ROI region quality adjustment parameters to an encoder for coding, thereby solving the technical problem of blocking effect and disharmony caused by a large difference in coding parameters between the ROI region and surrounding non-ROI regions.

Description

Technical Field

[0001] The present invention relates to the technical field of digital video encoding, and more particularly, to a ROI-based encoding method. Background Art

[0002] In recent years, with the development of video technology and media communication technology, users have put forward higher requirements for video quality. In order to provide a better video playback experience, the video bit rate of Internet video platforms has shown an exponential growth trend.

[0003] Among the existing ROI (Region of Interest) rate control methods, one is to adjust the ROI QP through multiple encoding and decoding with multiple quality parameters. Another method is to set a QP Offset table for the detected ROI and adjust the QP according to the QP Offset table during encoding. Most methods use hierarchical QP Offset settings for encoding.

[0004] Multiple encoding and decoding with multiple quality parameters increases the encoding time. In addition, the hierarchical QP offset table is set according to empirical values during the encoding process and does not change during the encoding process. Summary of the Invention

[0005] The present invention aims to provide a ROI-based coding method. The present invention proposes determining the QP offset of the ROI region based on the area ratio of the ROI region and the relationship between the ROI region QP offset and the bit rate change. A transition smoothing region is set in the non-ROI region based on the center distance between the ROI region and the non-ROI region to resolve the blocking effect and disharmony caused by the large difference in coding parameters between the ROI region and the surrounding non-ROI regions.

[0006] To achieve the above object, the present invention provides a ROI-based encoding method, comprising the following steps:

[0007] S1: Obtain the ROI area ratio of the video to be processed, and pre-train a set of curves based on the relationship between the quality adjustment parameters and the bit rate change of the video to be processed. The curves are curves of the quality parameters of the video to be processed, reflecting the area ratios of different visual regions of interest (ROIs);

[0008] S2: Send the current frame of the video to be processed to the visual saliency model for detection to obtain the ROI region coordinates and ROI region area ratio;

[0009] S3: Obtaining the ROI quality adjustment parameter of the current frame according to a preset target bit rate change rate and the quality parameter curve graph applicable to the current frame;

[0010] S4: obtaining a quality adjustment parameter of the non-ROI area according to the distance between the non-ROI area and the center coordinate of the ROI area, so as to prevent a significant blocking effect caused by a large difference in encoding parameters between the ROI area and the non-ROI area;

[0011] S5: Setting the coordinates of the ROI region and the ROI region quality adjustment parameters of the current frame, as well as the coordinates of the non-ROI region and the non-ROI region quality adjustment parameters of the current frame, and sending them to an encoder for encoding.

[0012] Preferably, in step S1, the curve diagram of the quality adjustment parameters of the video to be processed further includes:

[0013] Preset the quality adjustment parameters of the video to be processed, calculate the adjusted bit rate of the video to be processed under different quality adjustment parameters of the video to be processed, and the normal bit rate of the video to be processed without the quality adjustment parameters of the video to be processed, so as to determine the bit rate change rate of each video to be processed.

[0014] Preferably, in step S1, the curve diagram of the quality adjustment parameters of the video to be processed further includes:

[0015] According to the adjusted bit rate and the normal bit rate, all the bit rate change rates of the video to be processed are sorted out, the ROI area ratio, the bit rate change rate and the quality adjustment parameter of the video to be processed are matched, and the quality parameter curve graph corresponding to the ROI area ratio of the video to be processed is constructed according to the matching result.

[0016] Preferably, a first quantization offset Qa for the current frame is obtained according to the current frame quality adjustment parameter, thereby obtaining a second quantization offset Qb for the non-ROI area in the current frame.

[0017] Preferably, the post-bit rate of the video to be processed under different quality adjustment parameters further includes:

[0018] According to the preset quality adjustment parameter of the video to be processed, the first quality adjustment parameter is the current quantization offset, the first quantization offset of the video to be processed is determined according to the current quantization offset, and the ROI of the video to be processed is encoded according to the first quantization offset of the video to be processed.

[0019] Preferably, the post-bit rate of the video to be processed under different quality adjustment parameters further includes:

[0020] The adjusted bit rate is obtained according to the encoding result of the video to be processed. When the first quantization offset of the video to be processed is not the last one, the next quality adjustment parameter of the video to be processed is selected as the current quantization offset, and the current quantization offset is re-performed to determine the first quantization offset.

[0021] Preferably, the second quantization offset Qb includes:

[0022] When the coordinates of the center point of the ROI area of the current frame are (Xa, Ya), the coordinates of the center point of the non-ROI area are (Xb, Yb), and the distance between the center point of the ROI area and the center point of the non-ROI area is D, the second quantization offset Qb of the non-ROI area of the current frame is calculated according to the following formula, wherein the calculation formula of the second quantization offset Qb is as follows:

[0023]

[0024] .

[0025] Preferably, during the encoding process, the first quantization offset Qa and the second quantization offset Qb are controlled by the quality adjustment parameter, thereby achieving the goal of controlling the quality of the video to be processed.

[0026] Preferably, controlling the quality of the video to be processed further includes:

[0027] The smaller the quantization offset is, the more bits there are, the lower the compression rate is but the higher the video quality is, thereby achieving the goal of controlling the video quality. The quantization offset includes the first offset Qa and the second offset Qb.

[0028] A computer device includes a memory and one or more processors. The memory stores computer code, and when the one or more processors execute the computer code, the processors perform a step of ROI encoding in an embodiment of the present invention.

[0029] Compared with the existing technology, the present invention provides a ROI-based encoding method. The present invention utilizes the characteristics of the human eye that there are regions of interest and regions of non-interest, and allocates more bitrate to the subjective region of interest, thereby achieving the goal of improving subjective quality at the same bitrate or reducing bitrate for the same experience, ultimately achieving the problem of optimizing the subjective quality of the video and saving bitrate. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the invention.

[0031] Figure 1 This is an overall flow chart based on the ROI area of the present invention; DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Obviously, the described embodiments are part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0033] Those skilled in the art will understand that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the wording "comprising" used in the description of the present invention refers to the presence of the stated features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0034] The technical terms involved in the embodiments of the present invention are defined as follows:

[0035] 1. ROI: Region of Interest (ROI); NROI:

[0036] 2. Quality adjustment parameters: used to adjust the quantization parameters of each MB / CU during the bit rate control process;

[0037] 3. Normal bitrate (NROI_bitrate): refers to the bitrate of a video containing ROI after encoding without using quality adjustment parameters;

[0038] 4. Bitrate change ratio (bt_change_ratio): refers to the ratio of the adjusted bitrate ROI_bitrate of the video under quality adjustment parameters to the normal bitrate NROI_bitrate of the video without quality adjustment parameters. The calculation formula is bt_change_ratio = (ROI_bitrate - NROI_bitrate) / NROI_bitrate * 100;

[0039] 5. ROI area ratio (ROI_ratio): refers to the area of interest ratio, and the range of ROI area ratio is 1% to 100%;

[0040] 6. Macroblock: It is the basic coding unit in the H.264 video coding standard.

[0041] First embodiment

[0042] like Figure 1 As shown, this embodiment provides a specific embodiment of a ROI-based encoding method, and the specific process is as follows:

[0043] S1: Obtain the ROI area ratio of the video to be processed, adjust the parameters according to the relationship between the quality of the video to be processed and the bit rate change, and pre-train a set of curves. The curves are the curves of the quality parameters of the video to be processed, reflecting the area ratio of different visual regions of interest (ROIs);

[0044] S2: Send the current frame of the video to be processed to the visual saliency model for detection to obtain the ROI region coordinates and ROI region area ratio;

[0045] S3: Obtaining ROI quality adjustment parameters for the current frame according to a preset target bit rate change rate and a quality parameter curve graph applicable to the current frame;

[0046] S4: Obtain the quality adjustment parameter of the non-ROI area by the distance between the non-ROI area and the center coordinate of the ROI area to prevent the obvious block effect caused by the large difference in encoding parameters between the ROI area and the non-ROI area;

[0047] S5: setting the coordinates of the ROI region and the ROI region quality adjustment parameters of the current frame, as well as the coordinates of the non-ROI region and the non-ROI region quality adjustment parameters of the current frame, and sending them to the encoder for encoding.

[0048] Preferably, step S1, the curve diagram of the video quality adjustment parameter to be processed, further comprises:

[0049] The quality adjustment parameters of the video to be processed are preset, and the adjusted bit rates of the video to be processed under different quality adjustment parameters of the video to be processed and the normal bit rates of the video to be processed without the quality adjustment parameters of the video to be processed are calculated, thereby determining the bit rate change rate of each video to be processed.

[0050] Preferably, step S1, the curve diagram of the video quality adjustment parameter to be processed, further comprises:

[0051] According to the adjusted bit rate and the normal bit rate, all bit rate change rates of the video to be processed are sorted out, the ROI area ratio, bit rate change rate and the quality adjustment parameters of the video to be processed are matched, and a quality parameter curve corresponding to the ROI area ratio of the video to be processed is constructed based on the matching results.

[0052] Preferably, a first quantization offset Qa for the current frame is obtained according to the current frame quality adjustment parameter, thereby obtaining a second quantization offset Qb for the non-ROI area in the current frame. Specifically, in this embodiment, the region of interest ROI of the i-th frame is determined, the ROI area includes at least one macroblock, feature information of the at least one ROI macroblock is extracted, the feature information includes the position coordinates of the at least one macroblock, and the first quantization offset Qa corresponding to each ROI macroblock in the at least one ROI macroblock is determined.

[0053] Preferably, the post-bit rate of the video to be processed under different quality adjustment parameters further includes:

[0054] According to the preset quality adjustment parameter of the video to be processed, the first quality adjustment parameter is the current quantization offset, the first quantization offset of the video to be processed is determined according to the current quantization offset, and the ROI of the video to be processed is encoded according to the first quantization offset of the video to be processed.

[0055] Preferably, the post-bit rate of the video to be processed under different quality adjustment parameters further includes:

[0056] The adjusted bit rate is obtained according to the encoding result of the video to be processed. When the first quantization offset of the video to be processed is not the last one, the next quality adjustment parameter of the video to be processed is selected as the current quantization offset, and the current quantization offset is re-performed to determine the first quantization offset.

[0057] Preferably, the second quantization offset Qb includes:

[0058] When the coordinates of the center point of the ROI area of the current frame are (Xa, Ya), the coordinates of the center point of the non-ROI area are (Xb, Yb), and the distance between the center point of the ROI area and the center point of the non-ROI area is D, the second quantization offset Qb of the non-ROI area of the current frame is calculated according to the following formula, where the calculation formula of the second quantization offset Qb is as follows:

[0059]

[0060] .

[0061] Preferably, during the encoding process, the first quantization offset Qa and the second quantization offset Qb are controlled by quality adjustment parameters, thereby achieving control over the quality of the video to be processed.

[0062] Preferably, controlling the quality of the video to be processed further includes:

[0063] The smaller the quantization offset, the more bits, the lower the compression rate but the higher the video quality, thereby achieving the goal of controlling the video quality. The quantization offset includes a first offset Qa and a second offset Qb. Specifically, in this embodiment, the bit rate of the current frame is certain. In the enhanced ROI mode, the first quantization offset is reduced to allocate more bits, and for the non-ROI area, the number of bits of the second quantization offset is increased to achieve the purpose of improving the video quality of the ROI area.

[0064] A computer device includes a memory and one or more processors. The memory stores computer code. When the computer code is executed by the one or more processors, the processors execute the steps of a ROI encoding method in an embodiment of the present invention.

[0065] Second embodiment

[0066] In this embodiment, a computer device is provided, including a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of an ROI encoding method in the first embodiment.

[0067] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this description.

[0068] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A ROI-based encoding method, characterized in that: The following steps are involved: S1: Obtain the ROI area ratio of the video to be processed, and pre-train a set of curves based on the relationship between the quality adjustment parameters and the bit rate change of the video to be processed. The curves are curves of the quality parameters of the video to be processed, reflecting the area ratios of different visual regions of interest (ROIs); S2: Send the current frame of the video to be processed to the visual saliency model for detection to obtain the ROI region coordinates and ROI region area ratio; S3: Obtaining an ROI quality adjustment parameter for the current frame according to a preset target bit rate change rate and a quality parameter curve graph applicable to the current frame; S4: obtaining a quality adjustment parameter of the non-ROI area according to the distance between the non-ROI area and the center coordinate of the ROI area, so as to prevent a significant blocking effect caused by a large difference in encoding parameters between the ROI area and the non-ROI area; S5: setting the coordinates of the ROI region and the ROI region quality adjustment parameters of the current frame, as well as the coordinates of the non-ROI region and the non-ROI region quality adjustment parameters of the current frame, and sending them to the encoder for encoding; wherein, according to the current frame quality adjustment parameter, a first quantization offset Qa for the current frame is obtained, thereby obtaining a second quantization offset Qb for the non-ROI area in the current frame; The second quantization offset Qb includes: When the coordinates of the center point of the ROI area of the current frame are (Xa, Ya), the coordinates of the center point of the non-ROI area are (Xb, Yb), and the distance between the center point of the ROI area and the center point of the non-ROI area is D, the second quantization offset Qb of the non-ROI area of the current frame is calculated according to the following formula, wherein the calculation formula of the second quantization offset Qb is as follows: 。 2. The ROI-based encoding method according to claim 1, characterized in that: The step S1, processing the relationship between the video quality adjustment parameter and the bit rate change, further includes: Preset the quality adjustment parameters of the video to be processed, calculate the adjusted bit rate of the video to be processed under different quality adjustment parameters of the video to be processed, and the normal bit rate of the video to be processed without the quality adjustment parameters of the video to be processed, so as to determine the bit rate change rate of each video to be processed.

3. The ROI-based encoding method according to claim 2, characterized in that: The step S1, pre-training a set of curves based on the relationship between the quality adjustment parameters and the bit rate change of the video to be processed, further includes: According to the adjusted bit rate and the normal bit rate, all the bit rate change rates of the video to be processed are sorted out, the ROI area ratio, the bit rate change rate and the quality adjustment parameter of the video to be processed are matched, and the quality parameter curve graph corresponding to the ROI area ratio of the video to be processed is constructed according to the matching result.

4. The ROI-based encoding method according to claim 3, characterized in that: According to the preset quality adjustment parameter of the video to be processed, the first quality adjustment parameter is the current quantization offset, the first quantization offset of the video to be processed is determined according to the current quantization offset, and the ROI of the video to be processed is encoded according to the first quantization offset of the video to be processed.

5. The ROI-based encoding method according to claim 4, characterized in that: The adjusted bit rate is obtained according to the encoding result of the video to be processed. When the first quantization offset of the video to be processed is not the last one, the next quality adjustment parameter of the video to be processed is selected as the current quantization offset, and the current quantization offset is re-performed to determine the first quantization offset.

6. The ROI-based encoding method according to claim 1, characterized in that: During the encoding process, the first quantization offset Qa and the second quantization offset Qb are controlled by the quality adjustment parameter, thereby achieving the goal of controlling the quality of the video to be processed.

7. The ROI-based encoding method according to claim 6, characterized in that: The smaller the quantization offset is, the more bits there are, the lower the compression rate is but the higher the video quality is, thereby achieving the goal of controlling the video quality.

8. A computer device comprising a memory and one or more processors, wherein the memory stores computer code, and when the computer code is executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1 to 7.

Images