A video intra-frame coding method, encoder and medium

By dividing video frames in the encoder and determining the reference pixel set, encoding using the most likely mode and interpolation processing method, the problem of splicing traces after panoramic video frame mapping is solved, and efficient coding and excellent visual effects are achieved.

CN115278237BActive Publication Date: 2024-11-26CHINA TELECOM CLOUD TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202210910672.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-29
Publication Date
2024-11-26
Estimated Expiration
2042-07-29

AI Technical Summary

Technical Problem

When the prior art maps panoramic video frames into flat video frames, there are many deformations and splicing traces, which affects the visual effect and is difficult to maintain encoding efficiency while weakening splicing traces.

Method used

By dividing video frames based on preset rules in the encoder, determining the target area and reference pixel set near the splicing boundary, and encoding using the most likely mode and interpolation processing method, ensuring that the pixel difference at the splicing does not affect the encoding efficiency.

Benefits of technology

It effectively reduces the visual splicing traces after panoramic video frame mapping, improves video quality, and maintains efficient coding efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115278237B_ABST
    Figure CN115278237B_ABST
Patent Text Reader

Abstract

The present invention discloses a video intra-frame encoding method, an encoder and a medium, which are applied to panoramic video, including: the encoder obtains multiple first CUs in the current video frame based on a first preset rule, CTU and the position information of the splicing boundary in the current video frame; divides the first CU according to a preset division method to obtain multiple first PUs; determines a first target PU adjacent to the splicing boundary in a preset coding direction, and determines multiple first target areas and a first reference pixel set corresponding to each first target area according to a first mapping relationship and the first target PU; for the first target area, determines a first MPM reference block corresponding to the first target area according to the preset coding direction and the first mapping relationship; encodes the first target PU according to the preset coding direction, the first reference pixel set and the first MPM reference block. The present invention can weaken the visually visible splicing boundary caused by excessive pixel differences and improve video quality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of video encoding technology, and in particular to a video intra-frame encoding method, an encoder and a medium. Background Art

[0002] Panoramic video is a video shot in 360 degrees with a 3D camera. It refers to all the scenes around an observation point in space, which is composed of all the light that can be received by this observation point. Therefore, panoramic video can be abstracted into a sphere with the observation point as the center. When watching the video, users can adjust the video up, down, left, and right at will.

[0003] When processing panoramic video frames, considering that the computer's memory is not suitable for storing spherical structure data, it is necessary to map the panoramic video into planar video frames, and then compress, encode, decode, and inversely map the mapped planar video frames before storing them in the computer.

[0004] In the prior art, a mapping method based on spherical equal-area expansion is usually used to map panoramic video frames into planar video frames. However, the planar video frames obtained using this mapping method will have many deformations and splicing traces. After subsequent encoding, decoding and inverse mapping, visible splicing traces will be produced visually, affecting viewing. At present, the commonly used processing method is to use the HEVC (High Efficiency Video Coding) coding method. This method has a high coding efficiency, but the processing result at the splicing boundary is not ideal, and there will still be visible splicing traces. The method of introducing redundant pixels to reduce the splicing traces will reduce the coding efficiency. This is a method that sacrifices coding efficiency in exchange for subjective experience.

[0005] Therefore, how to reduce the splicing marks after panoramic video frame mapping without affecting the coding efficiency is a technical problem that needs to be solved urgently. Summary of the invention

[0006] The present invention provides a video intra-frame coding method, an encoder and a medium, which are used to solve the problem in the prior art of how to reduce the splicing traces existing after panoramic video mapping without affecting the coding efficiency.

[0007] In a first aspect, an embodiment of the present invention provides a video intra-frame coding method, which is applied to panoramic video, comprising:

[0008] The encoder obtains a plurality of CUs (Coding Units, first coding units) in the current video frame based on a first preset rule, CTUs (Coding Tree Units, coding tree units) and position information of a splicing boundary in the current video frame;

[0009] The encoder divides the plurality of first CUs according to a preset division method to obtain a plurality of first PUs (Prediction Units);

[0010] The encoder determines, in a preset encoding direction, a first target PU adjacent to the splicing boundary, and determines, according to a first mapping relationship and the plurality of first target PUs, at least two first target regions and a first reference pixel set corresponding to each first target region;

[0011] For each first target area, the encoder determines a first MPM (Most Probable Mode) reference block corresponding to the first target area according to the preset encoding direction and the first mapping relationship;

[0012] The encoder encodes the first target PU according to the preset encoding direction, the first reference pixel set and the first MPM reference block.

[0013] An embodiment of the present invention provides a video intra-frame coding method, in which the encoder divides the current video frame into multiple first CUs, and divides the multiple first CUs into multiple first PUs. On this basis, the encoder determines the first target PU adjacent to the splicing boundary in a preset coding direction, and determines at least two first target areas and a first reference pixel set corresponding to each first target area according to a first mapping relationship and multiple first target PUs, determines the first MPM reference block corresponding to the first target area, and encodes the first target PU according to the preset coding direction, the first reference pixel set, and the first MPM reference block. The above-mentioned video intra-frame coding method determines the spherical pixels adjacent to the multiple first target areas as the first reference pixel set corresponding to the first target areas through the first mapping relationship between the current video frame of the panoramic spherical video and the corresponding planar video frame, and determines the spherical MPM reference blocks adjacent to the multiple first target areas as the first MPM reference blocks corresponding to the first target areas. Based on this, intra-frame prediction coding of the current video frame is performed, which can reduce the visually visible splicing boundary caused by the excessive pixel difference at the splicing on the current video frame without affecting the coding efficiency, thereby improving the quality of the video.

[0014] In an optional implementation, the encoder determines, according to the first mapping relationship and the plurality of first target PUs, at least two first target regions and a first reference pixel set corresponding to each first target region, including:

[0015] The encoder determines, according to the first mapping relationship, a reference area adjacent to each first target PU;

[0016] The encoder determines the category of each reference area according to the position information of each reference area;

[0017] The encoder forms the first target region with target boundary pixels in the first target PU corresponding to the reference region of the same category in the current video frame, and uses the target pixels in the reference region of the same category as the first reference pixel set.

[0018] In the above method, the encoder determines the reference area adjacent to each first target PU according to the first mapping relationship between the spherical video frame and the planar video frame, that is, the first target PU and the reference area corresponding to it are adjacent to the spherical video frame, and then determines its category according to the position information of each reference area. In the current video frame, the target boundary pixels in the first target PU corresponding to the reference area of ​​the same category form the first target area, and the target pixels in the reference area of ​​the same category are used as the first reference pixel set. The first target area in the current video frame where the reference pixels need to be modified and the first reference pixel set corresponding to the first target area are determined by the above method, and the reference pixels of the first target area are modified to the set of pixels adjacent to it on the spherical video frame, that is, the first reference pixel set, rather than the set of pixels adjacent to it on the planar video frame. Since the target boundary pixels in the first target area are strongly associated with the target pixels referenced during the prediction coding, the visual impact caused by the large difference in pixel values ​​can be reduced.

[0019] In an optional implementation, after the encoder uses the target pixel points in the reference area of ​​the same category as the first reference pixel set, the encoder further includes:

[0020] The encoder compares the number of target pixels in the first reference pixel set with the number of target boundary pixels in the first target region corresponding thereto, and if the comparison result is different, determines a second reference pixel set according to relevant information of each target pixel in the first reference pixel set and relevant information of each target boundary pixel in the first target region;

[0021] The relevant information includes location information and pixel values.

[0022] In the above method, if the number of target pixels in the first reference pixel set is different from the number of target boundary pixels in the first target region corresponding thereto, the first reference pixel set has no reference value for the first target region, and it is necessary to re-establish a new pixel reference set, i.e., the second reference pixel set, based on the interpolation processing method according to the relevant information of the target pixels and the relevant information of the target boundary pixels. Each target boundary pixel in the first target region is predictively encoded according to the second reference pixel set, thereby improving the accuracy of the predictive encoding result.

[0023] In an optional implementation, the encoder determines the second reference pixel set according to the relevant information of each target pixel point in the first reference pixel set and the relevant information of each target boundary pixel point in the first target area, including:

[0024] For each target boundary pixel point in the first target area, the encoder determines a target position in a reference area corresponding to the first target area according to position information of the target boundary pixel point in the first target area;

[0025] The encoder determines a first weight and a second weight according to position information of the target position, position information of a first target pixel point adjacent to the target position, and position information of a second target pixel point adjacent to the target position, wherein the first weight is used to characterize the distance between the target position and the first target pixel point, and the second weight is used to characterize the distance between the target position and the second target pixel point;

[0026] The encoder determines a pixel value corresponding to the target position according to the first weight, the second weight, a pixel value of the first target pixel point, and a pixel value of the second target pixel point;

[0027] The encoder uses the determined pixel values ​​corresponding to all the target positions as the second reference pixel set.

[0028] The above method is a specific process for the encoder to determine the second reference pixel set. The encoder first determines the target position in the reference area corresponding to the target boundary pixel point according to the position information of the target boundary pixel point in the first target area, and then determines the first weight used to characterize the distance between the target position and the first target pixel point according to the position information of the target position and the position information of the first target pixel point adjacent to the target position. According to the position information of the target position and the position information of the second target pixel point adjacent to the target position, the second weight used to characterize the distance between the target position and the second target pixel point is determined. Finally, the pixel value corresponding to the target position is determined according to the first weight, the second weight, the pixel value of the first target pixel point and the pixel value of the second target pixel point. The pixel values ​​corresponding to all target positions constitute the second reference pixel set. The reference pixels in the second reference pixel set are determined by the interpolation method so that the first target area can be predicted and encoded accordingly.

[0029] In an optional implementation, after the encoder determines at least two first target areas and a first reference pixel set corresponding to each first target area, for each first target area, before the encoder determines, according to the preset encoding direction and the first mapping relationship, an MPM reference block corresponding to the first target area, further comprising:

[0030] For each first target area, the encoder determines that the reference area has a reference value according to the position information of the first target area and the position information of the reference area corresponding to the first target area.

[0031] In the above method, the encoder determines that the reference area has reference value based on the position information of the first target area and the position information of the reference area corresponding to it, and then determines the first MPM reference block corresponding to the first target area according to the preset encoding direction and the first mapping relationship, so as to refer to the prediction mode in the first MPM reference block, and predictively encodes the first target area according to the first reference pixel set, thereby improving the encoding effect.

[0032] In an optional implementation, the video intra-frame encoding method further includes:

[0033] When the encoder determines that the reference area has no reference value, the video frame is rotated by a preset angle in a preset selected direction to obtain a rotated video frame;

[0034] The encoder obtains the CTU and the position information of the splicing boundary in the rotated video frame based on the first preset rule, and obtains a plurality of second CUs in the rotated video frame;

[0035] The encoder divides the multiple second CUs according to a preset division method to obtain multiple second PUs;

[0036] The encoder determines, in a preset encoding direction, a second target PU adjacent to the splicing boundary, and determines, according to a second mapping relationship and the plurality of second target PUs, at least two second target areas and a third reference pixel set corresponding to each second target area;

[0037] For each second target area, the encoder determines a second MPM reference block corresponding to the second target area according to the preset encoding direction and the second mapping relationship;

[0038] The encoder encodes the second target PU according to the preset encoding direction, the third reference pixel set and the second MPM reference block.

[0039] In the above method, the encoder determines that the reference area has no reference value based on the position information of the first target area and the position information of the reference area corresponding thereto, and then rotates the video frame by a preset angle in a preset selection direction to obtain a rotated video frame, and divides the rotated video frame to obtain a second CU, divides it into a second PU, determines the second target area and the third reference pixel set and the second MPM reference block corresponding thereto, and performs a predictive coding step. By rotating the video frame with the reference area having no reference value by a preset angle in a preset selection direction, all the reference areas corresponding to the second target areas have reference value. At this time, the rotated video frame is subjected to predictive coding operations, so that each second target area can be subjected to appropriate predictive coding operations, which greatly improves the effect of video coding and makes the visual viewing effect of the panoramic video after inverse mapping better.

[0040] In an optional implementation, the encoder obtains multiple coding units CU in the current video frame based on the first preset rule, the coding tree unit CTU and the position information of the splicing boundary in the current video frame, including:

[0041] The encoder divides the CTU based on the first preset rule to obtain a plurality of divided units;

[0042] For each of the divided units, the encoder determines, according to the position information of the divided unit and the position information of the splicing boundary, whether the divided unit includes the splicing boundary, and if so, divides the divided unit to obtain the first CU; otherwise, uses the divided unit as the first CU;

[0043] The first preset rule includes: the rate-distortion cost of the divided unit is less than the rate-distortion cost of the CTU.

[0044] In the above method, the encoder calculates the rate-distortion cost of each CU after division and the rate-distortion cost before division based on its own rate-distortion optimization algorithm, selects the division method with the smallest rate-distortion cost as the most reasonable division method, and divides the current video frame into CUs accordingly. The encoder performs encoding operations in CU units, which greatly improves the encoding efficiency.

[0045] In a second aspect, an embodiment of the present invention provides an encoder, applied to panoramic video, including:

[0046] A CU partitioning module, configured to obtain a plurality of first coding units CU in the current video frame based on a first preset rule, a coding tree unit CTU and position information of a splicing boundary in the current video frame;

[0047] A PU division module, used for dividing the multiple first CUs according to a preset division method to obtain multiple first prediction units PU;

[0048] a reference pixel set determination module, configured to determine a first target PU adjacent to the splicing boundary in a preset coding direction, and determine at least two first target areas and a first reference pixel set corresponding to each first target area according to a first mapping relationship and a plurality of the first target PUs;

[0049] An MPM reference block determination module, configured to determine, for each first target area, a first MPM reference block corresponding to the first target area according to the preset encoding direction and the first mapping relationship;

[0050] A prediction coding module is used to encode the first target PU according to the preset coding direction, the first reference pixel set and the first MPM reference block.

[0051] In an optional implementation, the reference pixel set determination module is specifically used to:

[0052] Determine a reference area adjacent to each first target PU according to the first mapping relationship;

[0053] Determine the category of each reference area according to the location information of each reference area;

[0054] In the current video frame, target boundary pixels in the first target PU corresponding to the reference area of ​​the same category are used to form the first target area, and target pixels in the reference area of ​​the same category are used as the first reference pixel set.

[0055] In an optional implementation, the reference pixel set determination module is further configured to:

[0056] Comparing the number of target pixel points in the first reference pixel set with the number of target boundary pixel points in the first target region corresponding thereto, and if the comparison result is different, determining a second reference pixel set according to relevant information of each target pixel point in the first reference pixel set and relevant information of each target boundary pixel point in the first target region;

[0057] The relevant information includes location information and pixel values.

[0058] In an optional implementation, the reference pixel set determination module is further configured to:

[0059] For each target boundary pixel point in the first target area, determining a target position in a reference area corresponding to the first target area according to position information of the target boundary pixel point in the first target area;

[0060] Determine a first weight and a second weight according to the position information of the target position, the position information of a first target pixel point adjacent to the target position, and the position information of a second target pixel point adjacent to the target position, wherein the first weight is used to characterize the distance between the target position and the first target pixel point, and the second weight is used to characterize the distance between the target position and the second target pixel point;

[0061] Determine a pixel value corresponding to the target position according to the first weight, the second weight, the pixel value of the first target pixel point, and the pixel value of the second target pixel point;

[0062] The determined pixel values ​​corresponding to all the target positions are used as the second reference pixel set.

[0063] In an optional implementation, the reference pixel set determination module is further configured to:

[0064] For each first target area, it is determined that the reference area has a reference value according to the position information of the first target area and the position information of the reference area corresponding to the first target area.

[0065] In an optional implementation, the encoder further includes:

[0066] A video rotation module, for determining that the reference area has no reference value, and then rotating the video frame by a preset angle in a preset selected direction to obtain a rotated video frame;

[0067] The CU division module is used to obtain the CTU and the position information of the splicing boundary in the rotated video frame based on the first preset rule, and obtain multiple second CUs in the rotated video frame;

[0068] The PU division module is used to divide the multiple second CUs according to a preset division method to obtain multiple second PUs;

[0069] The reference pixel set determination module is used to determine a second target PU adjacent to the splicing boundary in a preset coding direction, and determine at least two second target areas and a third reference pixel set corresponding to each second target area according to a second mapping relationship and a plurality of the second target PUs;

[0070] The MPM reference block determination module is used to determine, for each second target area, a second MPM reference block corresponding to the second target area according to the preset encoding direction and the second mapping relationship;

[0071] The prediction coding module is used to encode the second target PU according to the preset coding direction, the third reference pixel set and the second MPM reference block.

[0072] In an optional implementation manner, the CU division module is specifically used to:

[0073] Based on the first preset rule, the CTU is divided to obtain a plurality of divided units;

[0074] For each of the divided units, judging whether the divided unit includes the splicing boundary according to the position information of the divided unit and the position information of the splicing boundary, and if so, dividing the divided unit to obtain the first CU; otherwise, using the divided unit as the first CU;

[0075] The first preset rule includes: the rate-distortion cost of the divided unit is less than the rate-distortion cost of the CTU.

[0076] In a third aspect, an embodiment of the present invention provides an encoder for panoramic video, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the video intra-frame encoding method described in any one of the above embodiments when executing the computer program.

[0077] In a fourth aspect, an embodiment of the present invention provides a computer storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed on a computer, the computer executes the steps of the video intra-frame encoding method described in any one of the above embodiments.

[0078] For the technical effects that may be achieved by the encoder disclosed in the second aspect, the encoder disclosed in the third aspect, and the computer storage medium disclosed in the fourth aspect, please refer to the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0080] Figure 1 A schematic diagram of a flow chart of a video intra-frame encoding method provided by an embodiment of the present invention;

[0081] Figure 2A schematic diagram of a current video frame and its splicing boundary provided by an embodiment of the present invention;

[0082] Figure 3a A schematic diagram of CU division of a current video frame provided by an embodiment of the present invention;

[0083] Figure 3b A schematic diagram of a forced CU division of a current video frame provided by an embodiment of the present invention;

[0084] Figure 4 A schematic diagram of the correspondence between a target area and a reference area of ​​a current video frame provided by an embodiment of the present invention;

[0085] Figure 5 A schematic diagram of a method for calculating pixel values ​​of points to be interpolated provided by an embodiment of the present invention;

[0086] Figure 6 A schematic diagram of correspondence between a target area and a reference area of ​​a rotated video frame provided by an embodiment of the present invention;

[0087] Figure 7 A schematic diagram of the structure of an encoder provided by an embodiment of the present invention;

[0088] Figure 8 A schematic diagram of the module structure of an encoder provided by an embodiment of the present invention;

[0089] Fig. 9 A schematic diagram of the module structure of another encoder provided by an embodiment of the present invention;

[0090] Fig.10 A schematic diagram of a program product of a video intra-frame encoding method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0091] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0092] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0093] When storing panoramic videos, it is necessary to map the panoramic video frames into planar video frames, and then perform compression encoding, decoding, and inverse mapping on the mapped planar video frames, so that the panoramic video can be stored in a computer. At present, a mapping method based on spherical equal area expansion is usually used to map the panoramic video frames into planar video frames. However, the planar video obtained using this mapping method will have many deformation and splicing traces. After subsequent encoding, decoding, and inverse mapping, visible splicing traces will be produced visually, affecting viewing.

[0094] In order to solve the problem of reducing the splicing marks existing after panoramic video frame mapping without affecting the coding efficiency, the embodiments of the present invention provide a video intra-frame coding method, an encoder and a medium.

[0095] Example 1

[0096] The following is a description of a video intra-frame coding method provided by the present invention through a specific embodiment. The method is applied to panoramic video, such as Figure 1 As shown, including:

[0097] Step 101: The encoder obtains a plurality of first coding units CU in the current video frame based on a first preset rule, a coding tree unit CTU and position information of a splicing boundary in the current video frame;

[0098] Step 102: The encoder divides the multiple first CUs according to a preset division method to obtain multiple first prediction units PUs;

[0099] Step 103: the encoder determines a first target PU adjacent to the splicing boundary in a preset coding direction, and determines at least two first target areas and a first reference pixel set corresponding to each first target area according to the first mapping relationship and the multiple first target PUs;

[0100] Step 104: for each first target area, the encoder determines a first MPM reference block corresponding to the first target area according to a preset encoding direction and a first mapping relationship;

[0101] Step 105: The encoder encodes the first target PU according to a preset encoding direction, a first reference pixel set and a first MPM reference block.

[0102] An embodiment of the present invention provides a video intra-frame coding method, in which the encoder first obtains multiple first CUs in the current video frame based on a first preset rule, CTU and the position information of the splicing boundary in the current video frame, and then divides the multiple first CUs according to a preset division method to obtain multiple first PUs, and then determines the first target PU adjacent to the splicing boundary in the preset coding direction, and determines at least two first target areas and a first reference pixel set corresponding to each first target area according to a first mapping relationship and multiple first target PUs, determines the first MPM reference block corresponding to the first target area, and finally encodes the first target PU according to the preset coding direction, the first reference pixel set and the first MPM reference block. In the above-mentioned video intra-frame coding method, since the reference pixel point is not the pixel point adjacent to the first target PU when encoding the first target PU, but is determined according to the first mapping relationship and multiple first target PUs, the visually visible splicing boundary caused by the excessive pixel difference at the splicing point on the current video frame can be weakened, thereby improving the quality of the video.

[0103] The embodiment of the present invention is applied to a panoramic video. The video frame in the embodiment of the present invention is a plane video frame corresponding to the panoramic video. The plane video frame is obtained by mapping the panoramic spherical video frame.

[0104] In one embodiment, the encoder obtains multiple coding units CU in the current video frame based on the first preset rule, the coding tree unit CTU and the position information of the splicing boundary in the current video frame. The CTU can be divided based on the first preset rule to obtain multiple divided units, and then for each divided unit, according to the position information of the divided unit and the position information of the splicing boundary, it is determined whether the divided unit includes the splicing boundary. If included, the divided unit is divided to obtain the first CU, otherwise, the divided unit is used as the first CU, wherein the first preset rule may be that the rate-distortion cost of the divided unit is less than the rate-distortion cost of the CTU.

[0105] It should be noted that, in the embodiment of the present invention, when the encoder encodes the current video frame, the current video frame may be divided into slices or tiles, and the embodiment of the present invention does not impose any limitation on this.

[0106] In implementation, Figure 2, which is a schematic diagram of the structure of a current video frame with splicing boundaries. The current video frame 21 includes four splicing boundaries 22 and two slice boundaries 23. The encoder divides the current video frame into slices (it can also divide the current video frame into tiles, Figure 2 (not shown in the figure), two Slice boundaries 23 are obtained, and the current video frame is divided into three video frame areas, S2, S3, and S4. Each video frame area is independent, so multiple video frame areas can be processed in parallel, which improves the encoding efficiency of the encoder; because the current video frame is obtained by mapping the panoramic spherical video through a mapping method based on spherical equal area expansion, there is a stitching boundary 22 in the current video frame, and the stitching boundary 22 divides the current video frame into two video frame areas S1 and S5. The current video frame is divided into five video frame areas, S1, S2, S3, S4, and S5, based on the Slice boundary 23 and the stitching boundary 22. The stitching boundary in the embodiment of the present invention is 22.

[0107] Specifically, Figure 3a As shown, it is a schematic diagram of the division of CU in the current video frame. The current video frame will be divided into multiple CTUs 31, wherein CTU 31 is the basic unit for the encoder to perform prediction, transformation, quantization and entropy coding operations on the current video frame, and the size can be 64*64.

[0108] Specifically, the encoder can perform quadtree division on each CTU 31 according to the complexity of the current video frame content in a preset coding order based on the rate-distortion optimization algorithm to obtain multiple CUs 32, where CU 32 has four sizes, namely: 64*64, 32*32, 16*16, and 8*8. The encoder will use the rate-distortion optimization algorithm to calculate the first rate-distortion cost of the divided unit and the second rate-distortion cost of the unit before the division. If the first rate-distortion cost is less than the second rate-distortion cost, the current unit will be quadtree divided. If the first rate-distortion cost is greater than the second rate-distortion cost, the current unit will not be quadtree divided. After multiple recursive division processes, the final CU division result of the current video frame is obtained. For each CU located at the splicing boundary, the encoder determines whether the splicing boundary is included in the current CU. If it is included, the current CU is forced to perform quadtree division again, otherwise, the current CU is retained.

[0109] For example, Figure 3a As shown, CU 33 is a CU containing a splicing boundary, but CU 33 is not divided along the splicing boundary. At this time, the encoder forces CU 33 to perform another quadtree division, and the result is as follows Figure 3b shown.

[0110] It should be noted that Slice or Tile is located in the upper layer of CTU in the image structure. A Slice is composed of an integer number of CTUs arranged successively, and a Tile is also composed of an integer number of CTUs arranged successively.

[0111] In the embodiment of the present invention, the encoder calculates the rate-distortion cost of each CU after division and the rate-distortion cost before division based on its own rate-distortion optimization algorithm, selects the division method with the smallest rate-distortion cost as the most reasonable division method, and divides the current video frame into CUs accordingly. The encoder performs encoding operations in units of CUs, which greatly improves the encoding efficiency.

[0112] In implementation, the encoder re-divides the CU obtained by dividing the current video frame to obtain multiple PUs.

[0113] In one embodiment, the encoder determines at least two first target areas and a first reference pixel set corresponding to each first target area based on the first mapping relationship and multiple first target PUs, can determine a reference area adjacent to each first target PU based on the first mapping relationship, determine a category of each reference area based on the location information of each reference area, and in the current video frame, group the target boundary pixels in the first target PU corresponding to the reference areas of the same category into a first target area, and use the target pixels in the reference areas of the same category as the first reference pixel set.

[0114] It should be noted that in the embodiment of the present invention, the first target area is a row or a column of pixels, the reference area is also a row or a column of pixels, and the first target PU is allowed to cross the Slice or Tile boundary for reference.

[0115] In the specific implementation, Figure 4 As shown, it is a schematic diagram of the structure of the first target area of ​​the current video frame 21 and the reference area corresponding thereto, wherein the encoder can encode in a "Z"-shaped encoding order, and in this encoding direction, multiple first target PUs adjacent to the four splicing boundaries 22 are determined, and five first target areas are determined according to the first mapping relationship, the position information of the reference area, and the multiple first target PUs, namely: target area A, target area B, target area C, target area D, target area E, and five reference areas corresponding to the five first target areas, namely: reference area a, reference area b, reference area c, reference area d, and reference area e. Among them, the target boundary pixels in the first target PU corresponding to the reference area of ​​the same category in the current video frame 21 constitute the first target area, and the target pixels in the reference area of ​​the same category serve as the first reference pixel set.

[0116] In an embodiment of the present invention, the encoder determines a reference area adjacent to each first target PU according to a first mapping relationship between the spherical video and the planar video, that is, the first target PU and the reference area corresponding thereto are adjacent in the spherical video frame, and then determines the category of each reference area according to its position information. In the current video frame, the target boundary pixel points in the first target PU corresponding to the reference area of ​​the same category are combined into a first target area, and the target pixel points in the reference area of ​​the same category are used as the first reference pixel set.

[0117] In an embodiment of the present invention, a first target area in the current video frame whose reference pixels need to be modified and a first reference pixel set corresponding to the first target area are determined, and the reference pixels of the first target area are modified to a set of pixels of its adjacent parts on the spherical video frame, i.e., the first reference pixel set, rather than a set of pixels adjacent to it on the planar video frame. Since the target boundary pixels in the first target area are strongly correlated with the target pixels referenced during predictive encoding, the visual impact caused by the large difference in pixel values ​​can be reduced.

[0118] Specifically, after the encoder determines at least two first target areas and a first reference pixel set corresponding to each first target area, the encoder can also determine, for each first target area, that the reference area has a reference value based on position information of the first target area and position information of the reference area corresponding to the first target area before determining the MPM reference block corresponding to the first target area according to a preset encoding direction and a first mapping relationship.

[0119] For example, Figure 4 As shown, since the encoder encodes in a "Z"-shaped encoding order, when the target area A needs to be encoded, the pixels in the corresponding reference area a are all encoded pixels and can be used as reference pixels of the target area A in the prediction encoding process. Therefore, the reference area a corresponding to the target area A is actually of reference value.

[0120] It should be noted that the encoding order in the embodiment of the present invention is the encoding direction.

[0121] In one embodiment, after the encoder takes the target pixel points in the reference area of ​​the same category as the first reference pixel set, it can also compare the number of target pixel points in the first reference pixel set with the number of target boundary pixel points in the corresponding first target area. If the comparison result is different, the second reference pixel set is determined based on the relevant information of each target pixel point in the first reference pixel set and the relevant information of each target boundary pixel point in the first target area, wherein the relevant information includes position information and pixel values.

[0122] Specifically, the encoder needs to compare the number of target pixels in the first reference pixel set and the number of target boundary pixels in the corresponding first target area. If the comparison result is the same, the direct reference method is adopted, that is, the first reference pixel set is used as the reference pixel set for the target boundary pixels in the corresponding first target area; if the comparison result is different, the interpolation processing method is adopted, that is, according to the relevant information of each target pixel in the first reference pixel set and the relevant information of each target boundary pixel in the first target area, the second reference pixel set is determined, and the second reference pixel set is used as the reference pixel set for the target boundary pixels in the corresponding first target area.

[0123] For example, Figure 4 As shown, target areas A, D and E belong to the direct reference class, which does not require any interpolation processing; target areas B and C belong to the difference interpolation processing class.

[0124] In the embodiment of the present invention, if the number of target pixels in the first reference pixel set is different from the number of target boundary pixels in the first target region corresponding thereto, the first reference pixel set has no reference value for the first target region, and it is necessary to re-establish a new pixel reference set, i.e., a second reference pixel set, based on the interpolation processing method according to the relevant information of the target pixels and the relevant information of the target boundary pixels. Each target boundary pixel in the first target region is predictively encoded according to the second reference pixel set, thereby improving the accuracy of the predictive encoding result.

[0125] In one embodiment, the encoder determines the second reference pixel set based on relevant information of each target pixel in the first reference pixel set and relevant information of each target boundary pixel in the first target area. For each target boundary pixel in the first target area, the encoder can first determine the target position in the reference area corresponding to the first target area based on position information of the target boundary pixel in the first target area, and then determine the first weight and the second weight based on the position information of the target position, the position information of the first target pixel adjacent to the target position, and the position information of the second target pixel adjacent to the target position, wherein the first weight is used to characterize the distance between the target position and the first target pixel, and the second weight is used to characterize the distance between the target position and the second target pixel, and the pixel value corresponding to the target position is determined based on the first weight, the second weight, the pixel value of the first target pixel and the pixel value of the second target pixel, and finally the pixel values ​​corresponding to all the determined target positions are used as the second reference pixel set.

[0126] Specifically, for the first target area of ​​the interpolation processing class, taking target area A as an example, its vertical resolution is different from the vertical resolution of reference area a, that is, the number of target boundary pixels in target area A is different from the number of target pixels in reference area a, and it is necessary to use linear interpolation of surrounding pixels in reference area a to obtain the required pixel value.

[0127] like Figure 5 As shown in the figure, it is the principle diagram of weighted interpolation of the point to be interpolated. The distance between the position of the pixel to be interpolated and the positions of the adjacent pixels on both sides is used as the weight, and the interpolation is performed using weighted mixing. (Ri,0) and (Ri+1,0) respectively represent the reference pixel values ​​adjacent to the pixel to be interpolated Ri on both sides and existing in the first reference pixel set. Wi is the distance from Ri to (Ri,0). The larger the distance, the smaller the weight assigned, and vice versa. Finally, the interpolated Ri is used as the reference pixel. The calculation formula is as follows:

[0128] R i =(1-w i )R i,0 +w i R i+1,0

[0129] For example, target region A contains 4 target boundary pixels and reference region a contains 7 target pixels. The encoder predicts and encodes the third target boundary pixel in target region A. The position of the third target boundary pixel in target region A is at the 3 / 4 position. Then, the position of the third target boundary pixel in reference region a is at the 7*(3 / 4) position, and this position is used as the target position. Then, the target pixels adjacent to the target position are determined, that is, the second target pixel in reference region a is the first target pixel, and its pixel value is (Ri, 0), and the third target pixel in reference region a is the second target pixel, and its pixel value is (Ri+1, 0). The distance wi between the target position and the second target pixel in reference region a is determined and used as the first weight. The distance (1-wi) between the target position and the third target pixel in reference region a is determined and used as the second weight.

[0130] The above calculation formula is used to calculate the pixel value of the point to be interpolated, and all target boundary pixel points in target area A are traversed to perform interpolation processing in sequence, and the pixel values ​​of all the points to be interpolated obtained are used as the second reference pixel set.

[0131] In an embodiment of the present invention, the encoder determines the specific process of the second reference pixel set. The encoder first determines the target position in the reference area corresponding to the target boundary pixel point according to the position information of the target boundary pixel point in the first target area, and then determines the first weight used to characterize the distance between the target position and the first target pixel point according to the position information of the target position and the position information of the first target pixel point adjacent to the target position, and determines the second weight used to characterize the distance between the target position and the second target pixel point according to the position information of the target position and the position information of the second target pixel point adjacent to the target position. Finally, the pixel value corresponding to the target position is determined according to the first weight, the second weight, the pixel value of the first target pixel point and the pixel value of the second target pixel point, and the pixel values ​​corresponding to all the target positions constitute the second reference pixel set. The reference pixels in the second reference pixel set are determined by the interpolation processing method, so that the first target area can be predicted and encoded accordingly.

[0132] In one embodiment, if the encoder determines that the reference area has no reference value, the current video frame is rotated by a preset angle in a preset selected direction to obtain a rotated video frame, and based on the first preset rule, the CTU and the position information of the stitching boundary in the rotated video frame are obtained to obtain multiple second CUs in the rotated video frame, and the multiple second CUs are divided according to a preset division method to obtain multiple second PUs. In the preset coding direction, a second target PU adjacent to the stitching boundary is determined, and at least two second target areas and a third reference pixel set corresponding to each second target area are determined according to the second mapping relationship and the multiple second target PUs. For each second target area, the encoder determines the second MPM reference block corresponding to the second target area according to the preset coding direction and the second mapping relationship, and the encoder encodes the second target PU according to the preset coding direction, the third reference pixel set and the second MPM reference block.

[0133] Specifically, Figure 4 As shown, since the encoder encodes in a "Z"-shaped encoding order, when the D target area needs to be encoded, the pixels in the corresponding d reference area are all unencoded pixels and cannot be used as reference pixels of the D target area in the prediction encoding process. Therefore, the d reference area corresponding to the D target area is actually of no reference value. Similarly, the e reference area corresponding to the E target area is actually of no reference value.

[0134] In a specific implementation, if the reference area corresponding to the first target area has no reference value, the original video frame can be rotated 90° clockwise and then encoded. After decoding, the video frame can be rotated 90° counterclockwise to restore it.

[0135] Specifically, for the rotated video frame, after dividing it into a second CU and a second PU, the encoder determines a second target PU adjacent to the stitching boundary in a preset encoding direction, and determines at least two second target areas and a third reference pixel set corresponding to each second target area based on the second mapping relationship and multiple second target PUs.

[0136] For example, Figure 6 As shown, it is a schematic diagram of the correspondence between the second target area and the third reference pixel set in the rotated video frame. In the rotated video frame, according to the second mapping relationship, wherein the second mapping relationship is obtained by rotating the first mapping relationship, the multiple second target PUs at the four splicing boundaries 22 are divided into five types of second target areas: B, C, D, F, and G. That is, the second target area includes: B target area, C target area, D target area, F target area, and G target area. The five reference areas corresponding to the five target areas are b reference area, c reference area, d reference area, f reference area, and g reference area. Among them, the left area of ​​the part where the F target area and the G target area intersect refers to the f reference area, and the upper area refers to the g reference area.

[0137] According to the second mapping relationship, the encoder also determines two other reference areas with reference value, namely, reference area a and reference area e, so when processing the pixels of the above five second target areas, it can also process the pixels in target area A and target area E. In this way, the encoder can process more target areas and improve the accuracy of predictive coding.

[0138] In an embodiment of the present invention, the encoder determines that the reference area has no reference value based on the position information of the first target area and the position information of the reference area corresponding thereto, and then rotates the video frame by a preset angle in a preset selection direction to obtain a rotated video frame, and divides the rotated video frame to obtain a second CU, divides it into a second PU, determines the second target area and the third reference pixel set and the second MPM reference block corresponding thereto, and performs a predictive coding step. By rotating the video frame with the reference area having no reference value by a preset angle in a preset selection direction, all the reference areas corresponding to the second target areas have reference value. At this time, the rotated video frame is subjected to predictive coding operations, so that each second target area can be subjected to appropriate predictive coding operations, which greatly improves the effect of video coding and makes the visual viewing effect of the panoramic video after inverse mapping better.

[0139] In one embodiment, for each first target area, the encoder determines the first MPM reference block corresponding to the first target area according to the preset coding direction and the first mapping relationship. Since the PU has 53 prediction modes, the encoder needs to select the most likely mode and perform prediction coding accordingly. The encoder determines the adjacent first MPM reference block of the current first target area on the spherical video corresponding to the current video frame according to the first mapping relationship, and the prediction mode of each first target area refers to the prediction mode of the first MPM reference block corresponding to it.

[0140] Example 2

[0141] Based on the same concept, an embodiment of the present invention also provides an encoder applied to panoramic video. Since the encoder is the encoder in the method in the embodiment of the present invention, and the principle of solving the problem by the encoder is similar to that of the method, the implementation of the encoder can refer to the implementation of the method, and the repeated parts will not be repeated.

[0142] Refer to the following Figure 7 The encoder 70 according to this embodiment of the present invention will be described. Figure 7 The encoder 70 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0143] like Figure 7 As shown, the encoder 70 may be in the form of a general-purpose computing device, for example, it may be a terminal device. The components of the encoder 70 may include, but are not limited to: the at least one processor 71, the at least one memory 72 storing executable instructions of the processor 71, and a bus 73 connecting different system components (including the memory 72 and the processor 71), and the processor 71 is a processor of an intelligent device.

[0144] The processor 71 implements the following steps by running the executable instructions:

[0145] Based on a first preset rule, a coding tree unit CTU and position information of a splicing boundary in a current video frame, a plurality of first coding units CU in the current video frame are obtained;

[0146] Divide the multiple first CUs according to a preset division method to obtain multiple first prediction units PU;

[0147] In a preset coding direction, determine a first target PU adjacent to the splicing boundary, and determine at least two first target areas and a first reference pixel set corresponding to each first target area according to a first mapping relationship and the plurality of first target PUs;

[0148] For each first target area, determining a first MPM reference block corresponding to the first target area according to the preset encoding direction and the first mapping relationship;

[0149] The first target PU is encoded according to the preset encoding direction, the first reference pixel set and the first MPM reference block.

[0150] In one embodiment, the processor 71 is specifically configured to:

[0151] Determine a reference area adjacent to each first target PU according to the first mapping relationship;

[0152] Determine the category of each reference area according to the location information of each reference area;

[0153] In the current video frame, target boundary pixels in the first target PU corresponding to the reference area of ​​the same category are used to form the first target area, and target pixels in the reference area of ​​the same category are used as the first reference pixel set.

[0154] In one embodiment, the processor 71 is specifically configured to:

[0155] Comparing the number of target pixel points in the first reference pixel set with the number of target boundary pixel points in the first target region corresponding thereto, and if the comparison result is different, determining a second reference pixel set according to relevant information of each target pixel point in the first reference pixel set and relevant information of each target boundary pixel point in the first target region;

[0156] The relevant information includes location information and pixel values.

[0157] In one embodiment, the processor 71 is specifically configured to:

[0158] For each target boundary pixel point in the first target area, determining a target position in a reference area corresponding to the first target area according to position information of the target boundary pixel point in the first target area;

[0159] Determine a first weight and a second weight according to the position information of the target position, the position information of a first target pixel point adjacent to the target position, and the position information of a second target pixel point adjacent to the target position, wherein the first weight is used to characterize the distance between the target position and the first target pixel point, and the second weight is used to characterize the distance between the target position and the second target pixel point;

[0160] Determine a pixel value corresponding to the target position according to the first weight, the second weight, the pixel value of the first target pixel point, and the pixel value of the second target pixel point;

[0161] The determined pixel values ​​corresponding to all the target positions are used as the second reference pixel set.

[0162] In one embodiment, the processor 71 is specifically configured to:

[0163] For each first target area, it is determined that the reference area has a reference value according to the position information of the first target area and the position information of the reference area corresponding to the first target area.

[0164] In one embodiment, the processor 71 is specifically configured to:

[0165] For determining that the reference area has no reference value, the video frame is rotated by a preset angle in a preset selected direction to obtain a rotated video frame;

[0166] Based on the first preset rule, the CTU and the position information of the splicing boundary in the rotated video frame are obtained, and a plurality of second CUs in the rotated video frame are obtained;

[0167] Divide the multiple second CUs according to a preset division method to obtain multiple second PUs;

[0168] In a preset coding direction, determine a second target PU adjacent to the splicing boundary, and determine at least two second target areas and a third reference pixel set corresponding to each second target area according to a second mapping relationship and the plurality of second target PUs;

[0169] For each second target area, determining a second MPM reference block corresponding to the second target area according to the preset encoding direction and the second mapping relationship;

[0170] The second target PU is encoded according to the preset encoding direction, the third reference pixel set and the second MPM reference block.

[0171] In one embodiment, the processor 71 is specifically configured to:

[0172] Based on the first preset rule, the CTU is divided to obtain a plurality of divided units;

[0173] For each of the divided units, judging whether the divided unit includes the splicing boundary according to the position information of the divided unit and the position information of the splicing boundary, and if so, dividing the divided unit to obtain the first CU; otherwise, using the divided unit as the first CU;

[0174] The first preset rule includes: the rate-distortion cost of the divided unit is less than the rate-distortion cost of the CTU.

[0175] Bus 73 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor or local bus using any of a variety of bus architectures.

[0176] The memory 72 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 721 and / or a cache memory 722 , and may further include a read-only memory (ROM) 723 .

[0177] The memory 72 may also include a program / utility 725 having a set (at least one) of program modules 724, such program modules 724 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0178] The cluster resource planning device 70 may also communicate with one or more external devices 74 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the cluster resource planning device 70, and / or may communicate with any device that enables the cluster resource planning device 70 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 75. In addition, the cluster resource planning device 70 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 76. As shown, the network adapter 76 communicates with other modules of the electronic device 70 via a bus 73. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the cluster resource planning device 70, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0179] Example 3

[0180] Based on the same concept, an embodiment of the present invention also provides an encoder applied to panoramic video. Since the device is the device in the method in the embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0181] like Figure 8 As shown, the encoder includes the following modules:

[0182] The CU partitioning module 801 is used to obtain a plurality of first coding units CU in the current video frame based on a first preset rule, a coding tree unit CTU and position information of a splicing boundary in the current video frame;

[0183] A PU division module 802 is used to divide the multiple first CUs according to a preset division method to obtain multiple first prediction units PU;

[0184] A reference pixel set determination module 803 is used to determine a first target PU adjacent to the splicing boundary in a preset coding direction, and determine at least two first target areas and a first reference pixel set corresponding to each first target area according to a first mapping relationship and a plurality of the first target PUs;

[0185] An MPM reference block determining module 804 is configured to determine, for each first target area, a first MPM reference block corresponding to the first target area according to the preset encoding direction and the first mapping relationship;

[0186] The prediction coding module 805 is used to encode the first target PU according to the preset coding direction, the first reference pixel set and the first MPM reference block.

[0187] In an optional implementation manner, the reference pixel set determination module 803 is specifically configured to:

[0188] Determine a reference area adjacent to each first target PU according to the first mapping relationship;

[0189] Determine the category of each reference area according to the location information of each reference area;

[0190] In the current video frame, target boundary pixels in the first target PU corresponding to the reference area of ​​the same category are used to form the first target area, and target pixels in the reference area of ​​the same category are used as the first reference pixel set.

[0191] In an optional implementation, the reference pixel set determination module 803 is further configured to:

[0192] Comparing the number of target pixel points in the first reference pixel set with the number of target boundary pixel points in the first target region corresponding thereto, and if the comparison result is different, determining a second reference pixel set according to relevant information of each target pixel point in the first reference pixel set and relevant information of each target boundary pixel point in the first target region;

[0193] The relevant information includes location information and pixel values.

[0194] In an optional implementation manner, the reference pixel set determination module 803 is specifically configured to:

[0195] For each target boundary pixel point in the first target area, determining a target position in a reference area corresponding to the first target area according to position information of the target boundary pixel point in the first target area;

[0196] Determine a first weight and a second weight according to the position information of the target position, the position information of a first target pixel point adjacent to the target position, and the position information of a second target pixel point adjacent to the target position, wherein the first weight is used to characterize the distance between the target position and the first target pixel point, and the second weight is used to characterize the distance between the target position and the second target pixel point;

[0197] Determine a pixel value corresponding to the target position according to the first weight, the second weight, the pixel value of the first target pixel point, and the pixel value of the second target pixel point;

[0198] The determined pixel values ​​corresponding to all the target positions are used as the second reference pixel set.

[0199] In an optional implementation, the reference pixel set determination module 803 is further configured to:

[0200] For each first target area, it is determined that the reference area has a reference value according to the position information of the first target area and the position information of the reference area corresponding to the first target area.

[0201] In an optional embodiment, if Fig. 9 As shown, the encoder further includes:

[0202] The video rotation module 901 is used to determine that the reference area has no reference value, and then rotate the video frame by a preset angle in a preset selected direction to obtain a rotated video frame;

[0203] The CU division module 801 is used to obtain a plurality of second CUs in the rotated video frame based on a first preset rule, by obtaining the CTU and position information of the splicing boundary in the rotated video frame;

[0204] The PU division module 802 is used to divide the multiple second CUs according to a preset division method to obtain multiple second PUs;

[0205] The reference pixel set determination module 803 is used to determine a second target PU adjacent to the splicing boundary in a preset coding direction, and determine at least two second target areas and a third reference pixel set corresponding to each second target area according to a second mapping relationship and a plurality of the second target PUs;

[0206] The MPM reference block determination module 804 is used to determine, for each second target area, a second MPM reference block corresponding to the second target area according to the preset encoding direction and the second mapping relationship;

[0207] The prediction coding module 805 is used to encode the second target PU according to the preset coding direction, the third reference pixel set and the second MPM reference block.

[0208] In an optional implementation manner, the CU division module 801 is specifically configured to:

[0209] Based on the first preset rule, the CTU is divided to obtain a plurality of divided units;

[0210] For each of the divided units, judging whether the divided unit includes the splicing boundary according to the position information of the divided unit and the position information of the splicing boundary, and if so, dividing the divided unit to obtain the first CU; otherwise, using the divided unit as the first CU;

[0211] The first preset rule includes: the rate-distortion cost of the divided unit is less than the rate-distortion cost of the CTU.

[0212] Example 4

[0213] In some possible embodiments, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of each module in the encoder according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification. For example, the encoder can be used to obtain multiple first coding units CU in the current video frame based on a first preset rule, a coding tree unit CTU, and position information of a splicing boundary in a current video frame; divide the multiple first CUs according to a preset division method to obtain multiple first prediction units PU; determine a first target PU adjacent to the splicing boundary in a preset coding direction, and determine at least two first target areas and a first reference pixel set corresponding to each first target area according to a first mapping relationship and the multiple first target PUs; for each first target area, determine a first MPM reference block corresponding to the first target area according to the preset coding direction and the first mapping relationship; encode the first target PU according to the preset coding direction, the first reference pixel set and the first MPM reference block.

[0214] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0215] like Fig.10 As shown, a program product 100 for an encoder according to an embodiment of the present invention is described, which can be a portable compact disk read-only memory (CD-ROM) and includes program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0216] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein readable program code is carried. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0217] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0218] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0219] It should be noted that although several modules or submodules of the system are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules to be embodied.

[0220] The present application is described above with reference to the block diagrams and / or flow charts showing the methods, devices (systems) and / or computer program products according to the embodiments of the present application. It should be understood that a block of the block diagram and / or flow chart diagram and a combination of blocks of the block diagram and / or flow chart diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer and / or other programmable data processing device to produce a machine, so that the instructions executed by the computer processor and / or other programmable data processing device create a method for implementing the functions / actions specified in the block diagram and / or flow chart block.

[0221] Accordingly, the present application may also be implemented with hardware and / or software (including firmware, resident software, microcode, etc.). Furthermore, the present application may take the form of a computer program product on a computer-usable or computer-readable storage medium, which has a computer-usable or computer-readable program code implemented in the medium, for use by an instruction execution system or in conjunction with an instruction execution system. In the context of the present application, a computer-usable or computer-readable medium may be any medium that may contain, store, communicate, transmit, or convey a program for use by an instruction execution system, device, or apparatus, or in conjunction with an instruction execution system, device, or apparatus.

[0222] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A video intra-frame coding method, characterized in that: Applied to panoramic video, including: The encoder obtains a plurality of first coding units CU in the current video frame based on a first preset rule, a coding tree unit CTU, and position information of a splicing boundary in the current video frame; The encoder divides the multiple first CUs according to a preset division method to obtain multiple first prediction units PU; The encoder determines, in a preset encoding direction, a first target PU adjacent to the splicing boundary, and determines, according to a first mapping relationship and the plurality of first target PUs, at least two first target areas and a first reference pixel set corresponding to each first target area, wherein the first mapping relationship is used to characterize a mapping relationship between the current video frame and the panoramic video; For each first target area, the encoder determines a first MPM reference block corresponding to the first target area according to the preset encoding direction and the first mapping relationship; The encoder performs prediction encoding on the first target PU according to the preset encoding direction and with reference to the prediction mode of the first MPM reference block, using the pixel values ​​of the target pixels included in the first reference pixel set.

2. The method according to claim 1, characterized in that The encoder determines, according to the first mapping relationship and the plurality of first target PUs, at least two first target regions and a first reference pixel set corresponding to each first target region, including: The encoder determines, according to the first mapping relationship, a reference area adjacent to each first target PU; The encoder determines the category of each reference area according to the position information of each reference area; The encoder forms the first target region with target boundary pixels in the first target PU corresponding to the reference region of the same category in the current video frame, and uses the target pixels in the reference region of the same category as the first reference pixel set.

3. The method according to claim 2, characterized in that After the encoder uses the target pixel point in the reference area of ​​the same category as the first reference pixel set, the encoder further includes: The encoder compares the number of target pixels in the first reference pixel set with the number of target boundary pixels in the first target region corresponding thereto, and if the comparison result is different, determines a second reference pixel set according to relevant information of each target pixel in the first reference pixel set and relevant information of each target boundary pixel in the first target region; The relevant information includes location information and pixel values.

4. The method according to claim 3, characterized in that The encoder determines a second reference pixel set according to relevant information of each target pixel point in the first reference pixel set and relevant information of each target boundary pixel point in the first target area, including: For each target boundary pixel point in the first target area, the encoder determines a target position in a reference area corresponding to the first target area according to position information of the target boundary pixel point in the first target area; The encoder determines a first weight and a second weight according to position information of the target position, position information of a first target pixel point adjacent to the target position, and position information of a second target pixel point adjacent to the target position, wherein the first weight is used to characterize the distance between the target position and the first target pixel point, and the second weight is used to characterize the distance between the target position and the second target pixel point; The encoder determines a pixel value corresponding to the target position according to the first weight, the second weight, a pixel value of the first target pixel point, and a pixel value of the second target pixel point; The encoder uses the determined pixel values ​​corresponding to all the target positions as the second reference pixel set.

5. The method according to claim 1, characterized in that After the encoder determines at least two first target areas and a first reference pixel set corresponding to each first target area, and before the encoder determines, for each first target area, according to the preset encoding direction and the first mapping relationship, an MPM reference block corresponding to the first target area, the encoder further includes: For each first target area, the encoder determines that the reference area has a reference value according to the position information of the first target area and the position information of the reference area corresponding to the first target area.

6. The method according to claim 5, characterized in that The method further comprises: When the encoder determines that the reference area has no reference value, the video frame is rotated by a preset angle in a preset selected direction to obtain a rotated video frame; The encoder obtains the CTU and the position information of the splicing boundary in the rotated video frame based on the first preset rule, and obtains a plurality of second CUs in the rotated video frame; The encoder divides the multiple second CUs according to a preset division method to obtain multiple second PUs; The encoder determines, in a preset encoding direction, a second target PU adjacent to the splicing boundary, and determines, according to a second mapping relationship and the plurality of second target PUs, at least two second target areas and a third reference pixel set corresponding to each second target area; For each second target area, the encoder determines a second MPM reference block corresponding to the second target area according to the preset encoding direction and the second mapping relationship; The encoder encodes the second target PU according to the preset encoding direction, the third reference pixel set and the second MPM reference block.

7. The method according to any one of claims 1 to 6, characterized in that: The encoder obtains a plurality of coding units CU in the current video frame based on a first preset rule, a coding tree unit CTU, and position information of a splicing boundary in the current video frame, including: The encoder divides the CTU based on the first preset rule to obtain a plurality of divided units; For each of the divided units, the encoder determines, according to the position information of the divided unit and the position information of the splicing boundary, whether the divided unit includes the splicing boundary, and if so, divides the divided unit to obtain the first CU; otherwise, uses the divided unit as the first CU; The first preset rule includes: the rate-distortion cost of the divided unit is less than the rate-distortion cost of the CTU.

8. An encoder, characterized in that Applied to panoramic video, including: A CU partitioning module, configured to obtain a plurality of first coding units CU in the current video frame based on a first preset rule, a coding tree unit CTU and position information of a splicing boundary in the current video frame; A PU division module, used for dividing the multiple first CUs according to a preset division method to obtain multiple first prediction units PU; a reference pixel set determination module, configured to determine a first target PU adjacent to the splicing boundary in a preset coding direction, and determine at least two first target areas and a first reference pixel set corresponding to each first target area according to a first mapping relationship and a plurality of the first target PUs, wherein the first mapping relationship is used to characterize a mapping relationship between the current video frame and the panoramic video; An MPM reference block determination module, configured to determine, for each first target area, a first MPM reference block corresponding to the first target area according to the preset encoding direction and the first mapping relationship; A prediction coding module is used to perform prediction coding on the first target PU according to the preset coding direction and with reference to the prediction mode of the first MPM reference block, using the pixel values ​​of the target pixels included in the first reference pixel set.

9. An encoder, characterized in that: Applied to panoramic video, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the video intra-frame encoding method according to any one of claims 1 to 7 when executing the computer program.

10. A computer storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the steps of the video intra-frame encoding method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • 3D (three dimensional) video depth image intra-frame predicting mode selecting method and system

    CN104125473A

  • Improved reference pixel selection and filtering for intra coding of depth map

    CN106688238A

  • Intra-frame prediction method and device for video coding, storage medium and electronic equipment

    CN114630114A

  • Video coding methods and apparatus

    US20120114039A1

  • An Apparatus, A Method and A Computer Program for Video Coding and Decoding

    US20190297339A1