Fast encoding method and apparatus for 360-degree video

Abstract

The application discloses a fast encoding method and device for 360-degree video. The method comprises the following steps: acquiring a plurality of video frames in 360-degree video data; dividing each video frame to obtain a plurality of encoding units; for each encoding unit, encoding the encoding unit by using a preset encoding mode, and extracting features of the encoding unit and the encoded encoding unit to obtain a first feature vector and a second feature vector; analyzing the first feature vector and the second feature vector by using a decision tree model to obtain a probability value of terminating remaining encoding mode checking; and determining a coding mode with the lowest rate-distortion cost from the preset encoding mode or the plurality of encoding modes as a target encoding mode of the encoding unit according to the size relationship between the probability value and a preset threshold value. The application solves the technical problem that the resource cost and time cost are relatively high due to the traversal checking of a plurality of encoding modes by an encoder when determining the best encoding mode of an encoding unit.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video coding, in particular to a fast encoding method and device for 360-degree video. BACKGROUND

[0002] With the improvement of people's quality of life, the demand for video quality is also growing, and people are pursuing more real audio-visual experience. Immersive VR (Virtual Reality) video, as a video that can bring users an immersive experience, can exactly meet the needs of users. 360-degree video is a kind of immersive VR video, which is usually shot and spliced by multiple cameras, and the resolution is 2-8 times that of ordinary video. The increase in video data volume makes the traditional transmission method unable to meet the needs, so video coding technology becomes a key solution.

[0003] Currently, most of the 360-degree videos are based on the fast encoding algorithm of HEVC (High Efficiency Video Coding), and there is a lack of inter-frame fast encoding algorithm based on VVC (Versatile Video Coding). In addition, although there are some fast encoding algorithms based on VVC in traditional planar videos, the encoder usually needs to check all the coding units when determining the best encoding mode corresponding to each coding unit. The mode checking process is performed according to the detection order of the standard encoder, which generally checks the affine mode first, then checks the extended merge mode (including the normal Merge mode and MMVD mode and CIIP mode), then checks the geometric partition mode, then checks the advanced motion vector prediction mode, and finally checks the intra mode. This checking process will bring great resource consumption and time cost to the encoder.

[0004] At present, there is no effective solution to the above problems. SUMMARY

[0005] The embodiments of the present application provide a fast encoding method and device for 360-degree video, to at least solve the technical problem that the resource cost and time cost are high because the encoder needs to traverse and check multiple encoding modes when determining the best encoding mode of the coding unit.

[0006] According to an aspect of the embodiments of the present application, a method for fast encoding of 360-degree video is provided, including: obtaining a plurality of video frames in 360-degree video data to be encoded; dividing each video frame to obtain a plurality of coding units; encoding each coding unit using a preset coding mode, and extracting features of the coding unit and the coded coding unit respectively to obtain a first feature vector and a second feature vector; analyzing the first feature vector and the second feature vector using a pre-trained decision tree model to obtain a probability value of terminating remaining coding mode checking, wherein the preset coding mode is at least one coding mode in a plurality of coding modes, and the remaining coding mode is other coding modes in the plurality of coding modes except the preset coding mode; and determining a coding mode with the lowest rate-distortion cost from the preset coding mode or the plurality of coding modes as a target coding mode of the coding unit according to a size relationship between the probability value and a preset threshold value.

[0007] Optionally, dividing the video frame to obtain a plurality of coding units includes: performing equidistant azimuthal projection on the video frame to obtain a corresponding planar image; dividing the planar image to obtain a plurality of coding tree units; and recursively splitting each coding tree unit using a preset division method to obtain a plurality of coding units.

[0008] Optionally, the first feature vector includes at least one of the following: a size of the coding unit, a horizontal stretching degree of a coding tree unit to which the coding unit belongs, an average absolute error value between a video frame to which the coding unit belongs and a reference video frame, and a variance of a difference value between the coding unit and a homologous reference coding unit in the reference video frame, and the second feature vector includes at least one of the following: a rate-distortion cost of the coding unit encoded using the preset coding mode, and a variance of a residual error.

[0009] Optionally, the determination of the horizontal stretching degree of the coding tree unit to which the coding unit belongs includes: determining a height of a planar image corresponding to the video frame; determining a vertical coordinate of a center pixel point in a coding tree unit to which the coding unit belongs; and determining the horizontal stretching degree of the coding tree unit to which the coding unit belongs using the following formula according to the height of the planar image and the vertical coordinate of the center pixel point:

[0010]

[0011] In the formula, α represents the horizontal stretching degree of the coding tree unit to which the coding unit belongs, y represents the vertical coordinate of the center pixel point, H represents the height of the planar image, and sec(·) represents a secant function.

[0012] Optionally, the first feature vector and the second feature vector are analyzed by using the pre-trained decision tree model to obtain a probability value of terminating the remaining coding mode checking, including: determining a size of the coding unit, and determining a target decision tree in the decision tree model corresponding to the size of the coding unit, wherein the decision tree model includes a plurality of decision trees, and each decision tree is used to analyze the feature vector of a coding unit of a size; and the first feature vector and the second feature vector are analyzed by using the target decision tree to obtain the probability value of terminating the remaining coding mode checking.

[0013] Optionally, according to a size relationship between the probability value and a preset threshold value, a coding mode with a lowest rate-distortion cost is selected from the preset coding mode or the plurality of coding modes as the target coding mode of the coding unit, including: in a case where the probability value is not lower than the preset threshold value, rate-distortion costs of the coding unit coded by using each preset coding mode are determined, and the preset coding mode with the lowest rate-distortion cost is determined as the target coding mode of the coding unit; and in a case where the probability value is lower than the threshold value, rate-distortion costs of the coding unit coded by using each coding mode in the plurality of coding modes are determined, and the coding mode with the lowest rate-distortion cost is determined as the target coding mode of the coding unit.

[0014] Optionally, the preset coding mode includes an extended merge mode and an affine mode; and the remaining coding mode includes at least one of an advanced motion vector prediction mode, an intra prediction mode, and a geometric partition mode.

[0015] According to another aspect of the embodiments of the present application, a fast coding device for 360-degree video is also provided, including: an acquisition module configured to acquire a plurality of video frames in 360-degree video data to be coded; a division module configured to divide each video frame to obtain a plurality of coding units; a decision module configured to, for each coding unit, code the coding unit by using a preset coding mode, and extract features of the coding unit and the coded coding unit respectively to obtain corresponding first feature vectors and second feature vectors; analyze the first feature vectors and the second feature vectors by using a pre-trained decision tree model to obtain a probability value of terminating remaining coding mode checking, wherein the preset coding mode is at least one coding mode in a plurality of coding modes, and the remaining coding mode is other coding modes in the plurality of coding modes except the preset coding mode; and a determination module configured to, according to a size relationship between the probability value and a preset threshold value, determine a coding mode with a lowest rate-distortion cost from the preset coding mode or the plurality of coding modes as a target coding mode of the coding unit.

[0016] According to another aspect of the embodiments of the present application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein a device in which the non-volatile storage medium is located performs the fast encoding method of 360-degree video by running the computer program.

[0017] According to another aspect of the embodiments of the present application, a computer program product is also provided, which includes a stored computer program, wherein the computer program is executed by a processor to implement the fast encoding method of 360-degree video.

[0018] In the embodiments of the present application, a plurality of video frames in the 360-degree video data to be encoded are obtained; each video frame is divided to obtain a plurality of coding units; for each coding unit, the coding unit is encoded by using a preset coding mode, and feature extraction is respectively performed on the coding unit and the coded coding unit to obtain corresponding first feature vectors and second feature vectors; the first feature vectors and the second feature vectors are analyzed by using a pre-trained decision tree model to obtain a probability value of terminating remaining coding mode checking, wherein the preset coding mode is at least one coding mode in the plurality of coding modes, and the remaining coding mode is other coding mode in the plurality of coding modes except the preset coding mode; according to the size relationship between the probability value and a preset threshold value, a coding mode with the lowest rate-distortion cost is selected from the preset coding mode or the plurality of coding modes as the target coding mode of the coding unit. Through the above coding mode determination method, the purpose of quickly predicting and terminating the redundant coding mode detection is achieved, thereby realizing the technical effect of reducing the encoding complexity of 360-degree video under VVC, and further solving the technical problem that the encoder needs to consume a large amount of resource cost and time cost due to the need to sequentially traverse each coding mode when determining the optimal coding mode of the coding unit. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate the illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0020] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) according to an optional fast encoding method of 360-degree video according to the embodiments of the present application;

[0021] Figure 2 is a flowchart of an optional fast encoding method of 360-degree video according to the embodiments of the present application;

[0022] Figure 3 is a schematic diagram of a planar image according to the embodiments of the present application;

[0023] Figure 4 is a schematic diagram of an optional quad-tree partitioning according to an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of an optional coding tree unit partitioning according to an embodiment of the present application;

[0025] Figure 6 is a mapping diagram of an optional latitude region and horizontal stretch degree of a coding tree unit according to an embodiment of the present application;

[0026] Figure 7 is a schematic diagram of a structure of an optional fast encoding apparatus for 360-degree video according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to the process, method, product, or device.

[0029] In addition, the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties. For example, an interface is provided between the system and the relevant user or institution. Before obtaining the relevant information, the interface needs to send a request for obtaining to the aforementioned user or institution, and after receiving the consent information fed back by the aforementioned user or institution, the relevant information is obtained.

[0030] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0031] Virtual Reality (VR): As the name implies, it is to combine virtual and reality with each other (generate a virtual situation by using a specific technology, but give a real feeling). In theory, virtual reality technology is a computer simulation system that can create and experience virtual world, which uses computer to generate a simulated environment to make users immerse in the environment. Virtual reality technology is to use real life data, through computer technology to generate electronic signals, and combine with various output devices to make it into phenomena that can be felt by people. These phenomena can be real objects, or we can't see the matter with our eyes, which is expressed through a three-dimensional model.

[0032] Video coding technology: It refers to the way of converting the original video format file into another video format file through compression technology. The most important coding and decoding standards in video streaming are H.261, H.263 and H.264 of International Telecommunication Union.

[0033] HEVC (High Efficiency Video Coding): also known as H.265 and MPEG-H part 2, is one of the video compression standards, and is one of the potential successors of widely used AVC (Advanced Video Coding). Compared with AVC, HEVC provides about twice the data compression ratio at the same video quality level, or significantly improves the video quality at the same bit rate. It supports a resolution of up to 8192x4320, including 8K ultra-high definition (UHD).

[0034] VVC (Versatile Video Coding): It is a new generation of international video coding standard, which is named Versatile Video Coding, also known as H.266. The main goal is to improve the existing HEVC technology to provide higher compression performance, while optimizing for emerging applications, including 360°360-degree video and high dynamic range imaging HDR.

[0035] Coding Tree Unit (CTU): is the largest processing unit in the video encoder. The size of the coding tree unit is 64x64 in HEVC and 128x128 in VVC. Generally, the coding tree unit can contain multiple most basic processing units-coding units (CU). In HEVC, the CU is generally a square structure, such as 8x8, 16x16 and 32x32; and in VVC, due to the introduction of the quad-tree nested multi-type tree partitioning method in VVC, the CU not only exists in square structure, but also exists in other rectangular structures, such as 8x4, 64x8, etc.

[0036] Affine mode: is a new inter prediction technology introduced in VVC. By introducing the affine prediction mode, VVC can better handle video sequences containing rotation, scaling and shear motion to more accurately predict the motion information in the video sequence.

[0037] Merge mode: is a motion vector (MV) prediction technology proposed in HEVC, which can be regarded as a coding technology. This mode directly obtains the MV of the current block by using the MV of the adjacent blocks in the time domain and the spatial domain. In HEVC, the encoder will establish an MV candidate list for the current prediction unit, and the list includes 5 candidate MVs. By traversing the 5 candidate MVs and selecting the optimal MV index through the rate-distortion criterion, only the index needs to be transmitted to the decoder. In VVC, the MV candidate list includes 6 candidate MVs, in addition, the index of the best candidate MV for each CU in the merge mode is coded using unary binarization, wherein the first bin of the best index is coded using context coding, and the other bins are coded using bypass coding.

[0038] Merge mode with motion vector difference (MMVD) mode: after the determination of the merge mode candidate in H.265 / HEVC, the motion vector of the best candidate is directly selected from the candidate list as the motion vector of the current coding block, and there may be a certain error between the actual motion vector. Therefore, H.266 / VVC increases the motion vector difference (MVD) on the basis of the original MV, so that the motion vector is more accurate. That is, by fine-tuning the candidate motion vector, the actual motion vector can be better matched. This way can reduce the prediction error and improve the accuracy of the motion vector.

[0039] Combined Inter and Intra Prediction (CIIP) mode: refers to the use of both inter prediction and intra prediction in video encoding. This hybrid prediction method aims to improve the efficiency and quality of video encoding. In CIIP, the encoder can flexibly choose to use inter prediction or intra prediction based on the characteristics and scene content of the current frame, in order to better utilize the temporal and spatial correlation in the video sequence, thus achieving more efficient compression encoding.

[0040] Geometric Partitioning Mode (GPM): is a block partitioning mode for inter prediction, which divides the block into different geometric shapes such as square, parallelogram, triangle and curve, etc. to better adapt to complex motion and image content. Generally, in videos containing moving objects, complex motion may occur in the edge zone, and geometric partitioning mode can better capture the outline and motion direction of the object.

[0041] Advanced Motion Vector Prediction (AMVP) mode: is a new MV prediction technology proposed in H.265 / HEVC, and H.266 / VVC still uses this technology and improves it based on HEVC. AMVP uses the correlation of motion vectors in space and time to establish a candidate prediction MV list for the current prediction unit; and the encoding end can select the optimal prediction MVP from the list, and encode the MVP (i.e. obtain the real MV through motion search, and encode the motion vector difference); the decoding end only needs to calculate the current prediction MV by establishing the same list and obtaining the index of the prediction MVP in the list and the motion vector residual MVD.

[0042] Intra Mode: is a technology that uses the correlation of adjacent pixels in the same frame to predict the pixels in the current block using the reconstructed pixels in the adjacent region of the current block. The current CU can use the reconstructed pixels of the adjacent CU to predict the pixels in the current CU. After intra prediction, the prediction residual is processed through transformation, quantization, etc. to effectively remove the spatial redundancy of the video. Generally, intra prediction techniques generally include Planar mode (which is suitable for situations where pixel values change slowly), DC mode (which is suitable for large flat areas) and other 33 angle prediction modes; in VVC, the angle prediction mode is expanded to 65, and technologies such as Matrix Weighted Intra Prediction (MIP) and Intra Sub-Partitions (ISP) are introduced, which further improve the coding performance and image quality.

[0043] ERP (Equi-Rectangular Projection): A simple projection method that maps meridians as vertical lines of constant spacing and parallels as horizontal lines of constant spacing.

[0044] DT (Decision Tree): A decision analysis method that is used to evaluate the risk of a project and determine its feasibility by constructing a decision tree to find the probability that the expected value of net present value is greater than or equal to zero based on the probability of various situations occurring. In machine learning, a decision tree is a predictive model that represents a mapping relationship between object attributes and object values.

[0045] QP (Quantization Parameter): It is a key parameter for controlling the degree of video compression. The higher the QP value, the stronger the compression, and the smaller the file size, but the image quality will decrease; on the contrary, the lower the QP value, the weaker the compression, and the larger the file size, but the image quality is better. In the video encoding process, the quantization parameter directly affects the trade-off between the bit rate and the visual quality of the video.

[0046] Embodiment 1

[0047] According to the embodiments of the present application, an embodiment of a fast encoding method for 360-degree video is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] The method embodiments provided by the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a fast encoding method for 360-degree video is shown. As shown in FIG. 1, the computer terminal (or mobile device) includes a processor 1001, a memory 1002, a storage 1003, a communication interface 1004, a display 1005, a sensor 1006 and the like. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more (illustrated as 102a, 102b, ..., 102n) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0049] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0050] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the fast encoding method of 360-degree video in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implementing the fast encoding method of 360-degree video of the above-mentioned application. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0051] The transmission device 106 is configured to receive or send data via a network. The network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network interface controller (NIC) that can connect to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module that is configured to communicate with the Internet wirelessly.

[0052] The display can be a liquid crystal display (LCD) that is touch screen, for example, which can enable a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0053] In the above operating environment, Figure 2 is a flowchart of an optional fast encoding method of 360-degree video according to an embodiment of the present application, as shown in Figure 2 The method includes steps S202-S208, in which:

[0054] In step S202, a plurality of video frames in 360-degree video data to be encoded are obtained.

[0055] In the technical solution of step S202, the 360-degree video data is panoramic video stream data that meets the requirements of pre-processing (resolution, frame rate). The 360-degree video data can be obtained by a panoramic camera, can be generated by a series of continuously captured 360-degree panoramic images, or can be formed by splicing a plurality of traditional video clips onto a virtual sphere to form a panoramic video effect. Each video frame in the 360-degree video is a set of continuous or non-continuous image frames in a video sequence, and each image frame is a still image with a 360-degree panoramic view. Unlike traditional videos, each image frame of the 360-degree video contains a complete view of the surroundings.

[0056] In step S204, each video frame is divided to obtain a plurality of coding units.

[0057] In the technical solution of step S204, the encoder traverses each video frame in the 360-degree video data, and divides the currently traversed video frame to obtain a coding unit (CU). The coding unit is a basic spatial structure unit for processing and encoding image data in a video encoding process.

[0058] Step S206, for each coding unit, encoding the coding unit by using the preset coding mode, and respectively extracting features of the coding unit and the coded coding unit to obtain corresponding first feature vectors and second feature vectors; analyzing the first feature vectors and the second feature vectors by using the pre-trained decision tree model to obtain a probability value of terminating the remaining coding mode checking.

[0059] When the 360-degree video is encoded by using the H.266 / VVV inter-frame encoding mode, the multiple coding units divided from the video frames in the 360-degree video need to be sequentially subjected to the coding mode checking. However, the coding mode checking process has high complexity, and the number of coding units is huge. Therefore, it is very redundant for the encoder to perform the coding mode checking process for each coding unit, and the encoder also has great resource consumption and time overhead.

[0060] Therefore, in the embodiments of the present application, for each coding unit, the coding unit is first encoded by using a preset coding mode, and features of the coding unit and the coded coding unit are respectively extracted to obtain corresponding first feature vectors and second feature vectors, wherein the preset coding mode is at least one coding mode in multiple coding modes; then the first feature vectors and the second feature vectors of the coding unit are analyzed by using a pre-trained decision tree model, and a probability value of terminating the remaining coding mode checking is output. The probability value can reflect whether the simple coding mode is the preferred coding mode of the current coding unit, so as to stop the checking of the remaining coding modes. The remaining coding modes are other coding modes in the multiple coding modes corresponding to the H.266 / VVV inter-frame prediction except the preset coding mode.

[0061] Step S208, according to the size relationship between the probability value and the preset threshold value, determining a coding mode with the lowest rate-distortion cost from the preset coding mode or the multiple coding modes as the target coding mode of the coding unit.

[0062] In the technical solution provided in step S208, the encoder compares the probability value determined in step S206 with the preset threshold value, and judges whether to select the coding mode with the lowest rate-distortion cost from the preset coding mode as the target coding mode of the coding unit, or to select the coding mode with the lowest rate-distortion cost from the multiple coding modes as the target coding mode of the coding unit according to the comparison result.

[0063] The above method of the embodiment will be further introduced below.

[0064] As an optional implementation, in the technical solution provided in step S204, the method can include:

[0065] Step S2041: performing equirectangular projection on the video frame to obtain a corresponding planar image.

[0066] Specifically, in the technical solution provided in step S2041, generally, the video frame in the 360-degree video is usually a mapping of a spherical surface. Therefore, in order to map the spherical image corresponding to the video frame to a planar image, the encoder can use equirectangular projection to convert the longitude and latitude information of each pixel point of the spherical image corresponding to the video frame into coordinates on the planar image, to obtain a planar image as shown in FIG. 2B, so as to maintain the continuity and integrity of the spherical image corresponding to the video frame. Figure 3

[0067] Specifically, the equirectangular projection is performed by sampling each pixel point of the spherical image corresponding to the video frame, and directly mapping the longitude of the pixel point on the spherical image to the horizontal coordinate of the planar image, ranging from 0 to the width of the spherical image, and mapping the latitude to the vertical coordinate of the planar image, ranging from 0 to half the height of the spherical image, thereby obtaining a 2:1 rectangular planar image. Through this projection mode, the pixels on the top and bottom of the spherical image are stretched horizontally, while the pixels near the equator remain almost unchanged. This stretching processing ensures that the video frame can still represent the 360-degree panoramic information after projection.

[0068] It should be noted that after performing equirectangular projection on the video frame, in order to ensure the image quality, the resolution of the planar image can also be adjusted according to the subsequent encoding requirements. For example, the encoder can downsample the high-resolution video frame to 4432x2216 (such as the processing in the VVC standard encoder) according to the encoding needs, so as to process a more reasonable amount of data in the encoding process, and at the same time, the original resolution is restored by upsampling at the decoding end.

[0069] Step S2042: dividing the planar image to obtain a plurality of coding tree units.

[0070] In the technical solution provided in step S2042, the encoder unit can initialize the maximum size of the coding tree unit in advance, such as the maximum encoder unit size allowed by VVC is 128x128 pixels, but in some cases, the size of the maximum encoder unit can be set smaller to adapt to different encoding scenarios, which can vary according to the video resolution and encoding parameters. Then, the encoder can divide the planar image into one or more coding tree units with fixed size.

[0071] Step S2043: for each coding tree unit, performing recursive segmentation on the coding tree unit in a preset division mode to obtain a plurality of coding units.

[0072] ​In the technical solution provided in step S2043, the encoder can traverse each coding tree unit and use the Figure 4 The quad-tree (Quad-Tree-Splitting) method shown in the figure is used for recursive partitioning, where Figure 4 The quadtree shown contains multiple tree types nested within it, including quadtrees, horizontal binary trees, vertical binary trees, horizontal ternary trees, and vertical ternary trees. Specifically, the encoder can first analyze the current coding tree unit to determine whether further segmentation is necessary. This determination can be based on image content complexity, motion information, and an assessment of coding efficiency. If a segmentation decision is made, quadtree segmentation is performed to divide the current coding tree unit into four equal sub-units, each half the size of the original coding tree unit. These four sub-units can serve as candidates for coding tree units or coding units, depending on the subsequent segmentation decision. The recursive decision-making and quadtree segmentation process is then repeated for each newly generated sub-unit until a termination condition is met. This termination condition may include reaching a minimum coding unit size, reaching a predetermined tree depth, or further segmentation not resulting in a significant improvement in coding performance. Finally, at the end of the recursive segmentation, each final unsplit sub-unit is considered a coding unit, which serves as the smallest independent coding block in the encoding process and can be independently subjected to prediction, transform, quantization, and other operations.

[0073] Taking the size of the coding tree unit as 128*128 as an example, the coding tree unit can be Figure 5 The method shown is used to divide the image into multiple coding units of different sizes, including: 64*64, 16*16, 8*8, 8*32, 32*8, etc.

[0074] After the encoder obtains multiple coding units corresponding to each video frame through the technical solution provided by the above steps S2041-S2043, for each coding unit, the encoder can encode the coding unit using a preset mode, wherein the above preset coding mode can be: an extended merge mode (i.e., a merge mode with motion vector difference) and an affine mode.

[0075] Specifically, regarding the determination of the above-mentioned preset coding mode, the probability of different coding units selecting the coding mode can be obtained according to historical experience and sorted out as shown in Table 1 below.

[0076] Table 1

[0077]

[0078]

[0079] From the above table 1, it can be seen that, under different quantization parameters, the probabilities of selecting the encoding modes by the encoding units of different sizes are arranged in descending order, and the preferred order of the encoding modes can be roughly summarized as: extended merge mode, affine mode, advanced motion vector prediction mode, intra prediction mode and geometric partition mode.

[0080] Therefore, the "preset encoding mode" in the embodiments of the present application can be the extended merge mode and the affine mode, wherein the extended merge mode includes: normal merge mode, merge mode with motion vector difference and hybrid intra-inter prediction mode. The "remaining encoding mode" can be: advanced motion vector prediction mode, intra prediction mode, geometric partition mode, etc.

[0081] It should be noted that, for some specific scenarios, the preferred order of the encoding modes selected by the encoding units of different sizes is still different. For example: the possibility of selecting the affine mode for a block with a larger size is about 10% higher than that for a block with a smaller size; and in the block with a smaller size, the probabilities of selecting the intra prediction mode and the geometric partition mode are significantly increased, which can be due to the fact that the smaller block is usually used to encode the image region containing high details or complex texture; in addition, the intra encoding mode allows the region to utilize its own spatial redundancy for encoding instead of relying on the information of other frames, thereby better preserving the image details and quality. In order to further accurately determine the preset encoding mode of the encoding unit of each size, the embodiments of the present application can further perform time complexity calculation on the encoding units of different sizes by using different encoding modes in the same encoding environment, so as to determine the time complexity of the encoding units of different sizes according to different encoding modes, and then adjust the preset encoding mode of the encoding unit of the size according to the time complexity.

[0082] For example, taking the encoding unit with a size of 32*32 as the experimental object, the time complexity of the encoding unit using the five encoding modes of the extended merge mode, the affine mode, the advanced motion vector prediction mode, the intra prediction mode and the geometric partition mode is calculated respectively, and the results are shown in the following table 2.

[0083] Table 2

[0084]

[0085]

[0086] As can be seen from Table 2, for the coding unit with a size of 32*32, the complexity ratio of the coding unit using the geometric partition mode is the lowest, followed by the affine mode, the complexity of the extended merge mode is close to that of the intra prediction mode, and the complexity of the advanced motion vector prediction mode is the highest. Among them, the complexity of the extended merge mode is increased because it includes the ordinary merge mode, the merge mode with a motion vector difference, and the mixed intra-inter prediction, not because the complexity of the ordinary merge mode is high.

[0087] Further, after determining the preset coding mode through the above steps, the encoder can perform feature extraction on the coding unit and the coded coding unit respectively to obtain corresponding first feature vectors and second feature vectors. The first feature vectors include the size of the coding unit, the horizontal stretching degree of the coding tree unit to which the coding unit belongs, the average absolute error value (WholeSubAvg) between the video frame to which the coding unit belongs and the reference video frame, the variance (BlockSubDif) of the difference value between the coding unit and the homologous reference coding unit in the reference video frame, the quantization parameter (QP), and the parent mode (PMode). The second feature vectors include the rate-distortion cost (RDCost) of the coding unit using the preset coding mode for coding, and the variance (Resi_Differ) of the residual error.

[0088] For the horizontal stretching degree of the current coding tree unit to which the current coding unit belongs, the encoder can determine the horizontal stretching degree by the following method:

[0089] First step: determine the height of the planar image corresponding to the video frame;

[0090] Second step: determine the vertical coordinate of the center pixel point in the current coding tree unit to which the current coding unit belongs;

[0091] Third step: according to the height of the planar image and the vertical coordinate of the center pixel point, the horizontal stretching degree of the current coding tree unit to which the current coding unit belongs is determined by the following formula:

[0092]

[0093] In the formula, α represents the horizontal stretching degree of the current coding tree unit to which the current coding unit belongs, y represents the vertical coordinate of the center pixel point, H represents the height of the planar image, and sec(·) represents the secant function.

[0094] It should be noted that, in order to associate the stretching degree with the spherical image corresponding to the coding tree unit, the present embodiment can divide the latitude range of the spherical image corresponding to the video frame into multiple parts according to the size of the spherical image corresponding to the video frame, and define the corresponding relationship between each horizontal stretching degree and the latitude region, such as Figure 6The latitude region where the coding tree unit to which the coding unit belongs is located can be taken as the first feature vector.

[0095] For the average absolute error value between the current video frame to which the current coding unit belongs and the reference video frame, the encoder can determine the reference video frame of the current video frame to which the current coding unit belongs from the plurality of video frames, and determine the average absolute error value between the pixel value of each pixel point in the current video frame and the pixel value of each pixel point in the reference video frame. The average absolute error value reflects the motion degree of the current sequence to some extent. When the average absolute error value is small, it indicates that the sequence is simple in motion, and thus a simpler coding mode can be selected for encoding. Conversely, when the average absolute error value is large, it indicates that the sequence changes dramatically over time and is complex in motion, and thus a more complex coding mode can be selected for encoding.

[0096] For the variance of the difference value between the current coding unit and the homologous reference coding unit in the reference video frame, the encoder can determine the reference video frame of the current video frame to which the current coding unit belongs from the plurality of video frames, and divide the reference video frame to obtain a plurality of reference coding units; determine the homologous reference coding unit having the same position coordinates as the current coding unit in the plurality of reference coding units; and determine the variance of the difference value between the pixel value of each pixel point in the current coding unit and the pixel value of each pixel point in the homologous reference coding unit. The greater the difference value variance between the current coding unit and the homologous reference coding unit, the greater the difference between them, and thus more data is needed for encoding, and thus a complex coding mode can be selected for encoding. Conversely, if the difference value variance is small, it means that the difference between them is small, and thus a simpler coding mode can be selected for encoding to save the amount of encoded data. In short, the difference value variance can guide the encoder to select an appropriate mode to balance the encoding quality and the amount of data.

[0097] For the parent mode, it means that the current coding unit can continue to use the coding mode used by the previous generation unit. Therefore, by using this inheritance, the coding mode of the current coding unit can be roughly determined.

[0098] For the quantization parameter, the encoder can set it according to the actual application scenario, and the present application does not make specific limitations thereto.

[0099] For the rate-distortion cost of the current coding unit encoded by using the preset coding mode, the encoder can determine it by the following method: determining the difference value between the pixel value of each pixel point in the current coding unit and the pixel value of each pixel point in the current coding unit after encoding, and taking the difference value as the distortion degree; and determining the rate-distortion cost of the current coding unit encoded by using the preset coding mode by using the rate-distortion cost function according to the bit rate corresponding to the first coding mode, the distortion degree, and the preset trade-off factor.

[0100] For the variance of the residual, the encoder can determine by the following method: predicting the pixel value of each pixel point in the current coding unit by using a preset prediction model (such as an intra prediction model or an inter prediction model), and determining the difference between the original pixel value and the predicted pixel value of each pixel point in the current coding unit, taking the difference as the residual, and determining the variance of the residual. Wherein, if the residual is large, it means that there is a large motion change or scene content difference between the current frame and the reference frame, which usually requires more bits to encode the motion vector and residual data, thereby possibly leading to a higher rate-distortion cost; on the contrary, if the residual is small, it means that the current frame is similar to the reference frame, which can be effectively encoded by fewer bits to achieve a lower rate-distortion cost.

[0101] It should be noted that the above-mentioned reference frame can be determined by random access. For example, if the current frame is the second frame, the first frame can be used as the reference frame. In addition, since VVC supports bi-prediction, the current frame can have two reference frames (one forward reference frame and one backward reference frame). For example, if the current frame is the 20th frame, the 1st frame and the 40th frame can be used as the reference frame.

[0102] After obtaining the first feature vector and the second feature vector corresponding to the current coding by the above method, the encoder can determine the probability value of the termination of the remaining coding mode check corresponding to the current coding unit by the following method, comprising:

[0103] First, determine the size of the coding unit, and determine the target decision tree in the decision tree model corresponding to the size of the coding unit; then, analyze the first feature vector and the second feature vector by using the target decision tree to obtain the probability value of the termination of the remaining coding mode check.

[0104] Specifically, the pre-trained decision tree model in the embodiment of the present application includes a plurality of decision trees, and each decision tree is used to analyze the feature vector of a coding unit of a certain size.

[0105] This is because the distribution characteristics of the coding mode selected by the coding unit of different sizes are different, and specific reference is made to the above table 1, so it is very necessary to classify and train the coding units of different sizes. In the inter coding process of VVC, a total of 27 different sizes of coding units are generated, and if all sizes of coding units are discussed separately, 27 cases need to be discussed, which is a very complex and redundant process.

[0106] In order to reduce the workload as much as possible, the embodiments of the present application propose an efficient classification scheme, that is, 27 different sizes of coding units are divided into multiple categories. This classification is not only based on the physical size of the coding unit, but also considers their behavior patterns and selection tendencies in actual coding applications. The closer the size of the coding unit, the more similar their distribution rules are.

[0107] Specifically, in the embodiments of the present application, it is preferred to be divided into 5 categories, and the specific division results are as follows:

[0108] (1) 128x128, 128x64 and 64x128 CUs;

[0109] (2) 64x64, 32x32, 64x32, 32x64, 64x16 and 16x64 CUs;

[0110] (3) 64x8, 8x64, 16x32 and 32x16 CUs;

[0111] (4) 16x16, 16x8, 8x16, 8x8, 32x8 and 8x32 CUs;

[0112] (5) 64x4, 32x4, 16x4, 8x4, 4x64, 4x32, 4x16 and 4x8 CUs.

[0113] Through the above division strategy, the originally complex analysis process can be simplified to the discussion of 5 typical cases, which significantly reduces the complexity and workload of the research. Therefore, not only the efficiency of the coding decision is improved, but also a strong theoretical support is provided for further optimizing the VVC coding strategy and improving the coding performance.

[0114] Further, for each type of coding unit, historical coding data of the type of coding unit is obtained, wherein the historical coding data includes the first feature vector and the second feature vector (training samples) of the type of coding unit, and the optimal coding mode (sample label) of the type of coding unit; then, the training samples are taken as input, and the sample label is taken as output to train the decision tree corresponding to the type of coding unit, and the model parameters such as the depth of the tree and the minimum sample quantity of the leaf node are adjusted through cross-validation to optimize the prediction accuracy and the efficiency of early termination of the model. Therefore, the first feature vector and the second feature vector of the current coding unit are analyzed by the trained decision tree to determine the probability value of the remaining coding mode being selected as the optimal coding mode, and if the probability is lower than a certain threshold, it can be concluded that the remaining coding mode is unlikely to provide the optimal coding result, and therefore the encoder is suggested to terminate the check of the remaining coding mode, and therefore the "probability value of the remaining coding mode being selected as the optimal coding mode" is the "probability value of terminating the check of the remaining coding mode" in the embodiments of the present application. In addition, the best termination threshold (i.e. the threshold value) corresponding to the decision tree corresponding to the five types of coding units in the embodiments of the present application can be set to 0.75, 0.75, 0.80, 0.85, and 0.80, respectively.

[0115] Further, after the encoder obtains the probability value of the current coding unit terminating the check of the remaining coding mode through the above steps, the encoder can determine the coding mode of the current coding unit through the following method, which includes:

[0116] In the case where the probability value of terminating the check of the remaining coding mode is not lower than the preset threshold value, the rate-distortion cost of the coding unit using each preset coding mode is determined, and the preset coding mode with the lowest rate-distortion cost is taken as the target coding mode of the coding unit.

[0117] In the above case, it is indicated that the encoder has found a coding mode with sufficient performance, and the encoder terminates the check of the remaining coding mode at this time, which can greatly reduce the consumption of coding time and computing resources. At the same time, the rate-distortion cost of the coding unit using the preset coding mode is determined, and the preset coding mode with the lowest rate-distortion cost is taken as the target coding mode of the coding unit.

[0118] In the case where the probability value of terminating the check of the remaining coding mode is lower than the threshold value, the rate-distortion cost of the coding unit using each coding mode in the plurality of coding modes is determined, and the coding mode with the lowest rate-distortion cost is taken as the target coding mode of the coding unit.

[0119] In the above case, it is illustrated that the encoder, after checking the simple encoding modes, does not find that these simple encoding modes can provide the best encoding performance based on the current rate-distortion cost and image quality assessment, and then continues to check the remaining encoding modes, in other words, the simple encoding modes have not yet reached the expected standard of the encoder for encoding performance and quality. At this time, the encoder continues to check the remaining encoding modes in turn and determines the rate-distortion cost of the preset encoding mode and the remaining encoding mode for encoding the encoding unit, and takes the encoding mode with the lowest rate-distortion cost as the target encoding mode of the encoding unit.

[0120] The encoder determines the best encoding mode of the encoding unit by the above method, which not only can avoid redundant checking of complex encoding strategies that are not likely to be the optimal encoding mode, thereby significantly reducing the encoding time and improving the encoding efficiency, but also can dynamically determine the next step of the encoding process according to the characteristics of the encoding unit, rather than blindly following the fixed mode detection order, which helps the encoder to more efficiently process video content, especially when processing 360-degree video. In addition, the above method can reduce the encoding time as much as possible while ensuring that the video quality is not excessively sacrificed, thereby achieving a good balance between encoding quality and encoding speed.

[0121] In addition, in order to verify the beneficial effects of the above method, the RandomAccess is used to configure the VVC test model (VTM), and the QP values are set to 22, 27, 32 and 37 respectively. The general test video sequences required in the verification process are listed in Table 3 below, including resolution, encoding frame number, frame rate and bit depth and other information.

[0122] Table 3

[0123]

[0124] The 360-degree videos in Table 3 are encoded by using the fast encoding method provided in the embodiments of the present application, and the obtained encoding performance (including bit rate variation and time saving) is compared with the encoding performance obtained by using the standard encoder of the same version (i.e., the full mode check is performed once for each encoding unit). The BDBR (Bjontegaard Delta Bit Rate) % comparison result and the TS (Time Saving) % comparison result are shown in Table 4. The BDBR refers to the percentage of the variation between the bit rate after the encoding of the proposed algorithm and the bit rate after the encoding of the standard encoder in the video encoding process, under the condition that the quality of the compressed video is the same. Generally, if the value is negative, it means that the encoding efficiency is improved; if the value is positive, it means that the encoding efficiency is reduced. The TS refers to the improvement of the encoding speed, which is an important indicator for measuring the performance of the encoder. The TS can be calculated by comparing the time required for two encoders to complete the same video encoding. A higher TS value indicates that the new encoding method is faster than the reference encoding method and can complete the encoding task in a shorter time.

[0125] Table 4

[0126]

[0127]

[0128] As can be seen from Table 4, the BDBR comparison result obtained by using the encoding method provided in the present solution and the standard encoder of the same version increases by an average of 0.36% on different video sequences, and the TS comparison result decreases by an average of 21.35%.

[0129] Based on the solutions defined in steps S202 to S208, it can be known that, in the embodiments, the encoder extracts features before and after the encoding of each encoding unit by using the preset encoding mode, and analyzes the extracted feature vectors by using the decision tree model to decide whether to continue the mode check of the remaining encoding modes, so as to avoid the encoder blindly following the fixed mode detection order, resulting in a great increase in the encoding time and a great reduction in the encoding efficiency. As can be seen, by using the technical solution of the embodiments of the present application, the encoder can intelligently decide whether to continue the complex encoding mode detection, so as to achieve the purpose of fast prediction and early termination of the redundant encoding mode detection, thereby realizing the technical effect of reducing the encoding complexity of the VVC encoding process of the 360-degree video, and further solving the technical problem that the resource cost and time cost are high due to the traversal check of multiple encoding modes when the encoder determines the best encoding mode of the encoding unit.

[0130] Embodiment 2

[0131] Based on the embodiment 1 of the present application, an embodiment of a fast encoding device for 360-degree video is also provided, which executes the fast encoding method for 360-degree video of the above-mentioned embodiments when running. Wherein, Figure 7 is a structural schematic diagram of an optional fast encoding device for 360-degree video according to the embodiment of the present application, as Figure 7 shown, the fast encoding device for 360-degree video at least includes an acquisition module 72, a division module 74 and a decision module 76, and a determination module 78, wherein:

[0132] The acquisition module 72 is configured to acquire a plurality of video frames in the 360-degree video data to be encoded;

[0133] The division module 74 is configured to divide each video frame to obtain a plurality of coding units;

[0134] The decision module 76 is configured to encode each coding unit by using a preset encoding mode, and respectively extract features of the coding unit and the encoded coding unit to obtain a first feature vector and a second feature vector; analyze the first feature vector and the second feature vector by using a pre-trained decision tree model to obtain a probability value of terminating remaining encoding mode checking, wherein the preset encoding mode is at least one encoding mode in the plurality of encoding modes, and the remaining encoding mode is other encoding mode in the plurality of encoding modes except the preset encoding mode;

[0135] The determination module 78 is configured to determine the encoding mode with the lowest rate-distortion cost from the preset encoding mode or the plurality of encoding modes as the target encoding mode of the coding unit according to the size relationship between the probability value and a preset threshold value.

[0136] It should be noted that each module in the fast encoding device for 360-degree video described above can be a program module (for example, a set of program instructions that implement a certain specific function) or a hardware module. For the latter, it can be in the following form, but not limited to: the form of each module is a processor, or the functions of each module are implemented by a processor.

[0137] Embodiment 3

[0138] According to the embodiment of the present application, a non-volatile storage medium is also provided, which stores a program, wherein when the program runs, it controls the device where the non-volatile storage medium is located to execute the fast encoding method for 360-degree video in the embodiment 1.

[0139] Optionally, the device in which the non-volatile storage medium is located implements the following steps by running the program: obtaining a plurality of video frames in 360-degree video data to be encoded; for each video frame, dividing the video frame to obtain a plurality of coding units; for each coding unit, encoding the coding unit by using a preset coding mode, and respectively extracting features of the coding unit and the coded coding unit to obtain corresponding first feature vectors and second feature vectors; analyzing the first feature vectors and the second feature vectors by using a pre-trained decision tree model to obtain a probability value of terminating remaining coding mode checking, wherein the preset coding mode is at least one coding mode in a plurality of coding modes, and the remaining coding mode is other coding modes in the plurality of coding modes except the preset coding mode; and determining a coding mode with the lowest rate-distortion cost from the preset coding mode or the plurality of coding modes as a target coding mode of the coding unit according to a size relationship between the probability value and a preset threshold value.

[0140] According to an embodiment of the present application, a computer program product is also provided, which includes a stored computer program. When the computer program is executed by a processor, the computer program implements the fast encoding method for 360-degree video in the embodiment 1.

[0141] Optionally, the computer program implements the following steps: obtaining a plurality of video frames in 360-degree video data to be encoded; for each video frame, dividing the video frame to obtain a plurality of coding units; for each coding unit, encoding the coding unit by using a preset coding mode, and respectively extracting features of the coding unit and the coded coding unit to obtain corresponding first feature vectors and second feature vectors; analyzing the first feature vectors and the second feature vectors by using a pre-trained decision tree model to obtain a probability value of terminating remaining coding mode checking, wherein the preset coding mode is at least one coding mode in a plurality of coding modes, and the remaining coding mode is other coding modes in the plurality of coding modes except the preset coding mode; and determining a coding mode with the lowest rate-distortion cost from the preset coding mode or the plurality of coding modes as a target coding mode of the coding unit according to a size relationship between the probability value and a preset threshold value.

[0142] According to an embodiment of the present application, a processor is also provided, which is used to run a program. When the program is run, the processor implements the fast encoding method for 360-degree video in the embodiment 1.

[0143] Optionally, the program is configured to perform the following steps when executed: obtaining a plurality of video frames in 360-degree video data to be encoded; for each video frame, dividing the video frame to obtain a plurality of coding units; for each coding unit, encoding the coding unit using a preset coding mode, and respectively extracting features of the coding unit and the coded coding unit to obtain corresponding first feature vectors and second feature vectors; analyzing the first feature vectors and the second feature vectors using a pre-trained decision tree model to obtain a probability value of terminating remaining coding mode checking, wherein the preset coding mode is at least one coding mode in the plurality of coding modes, and the remaining coding mode is other coding modes in the plurality of coding modes except the preset coding mode; and determining a coding mode with the lowest rate-distortion cost from the preset coding mode or the plurality of coding modes as a target coding mode of the coding unit according to a size relationship between the probability value and a preset threshold value.

[0144] According to the embodiments of the present application, an electronic device is also provided, which includes one or more processors; a memory configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement a program for running, wherein the program is configured to perform the fast encoding method of 360-degree video in the above-mentioned embodiment 1 when running.

[0145] Optionally, the processor is configured to perform the following steps by means of the computer program: obtaining a plurality of video frames in 360-degree video data to be encoded; for each video frame, dividing the video frame to obtain a plurality of coding units; for each coding unit, encoding the coding unit using a preset coding mode, and respectively extracting features of the coding unit and the coded coding unit to obtain corresponding first feature vectors and second feature vectors; analyzing the first feature vectors and the second feature vectors using a pre-trained decision tree model to obtain a probability value of terminating remaining coding mode checking, wherein the preset coding mode is at least one coding mode in the plurality of coding modes, and the remaining coding mode is other coding modes in the plurality of coding modes except the preset coding mode; and determining a coding mode with the lowest rate-distortion cost from the preset coding mode or the plurality of coding modes as a target coding mode of the coding unit according to a size relationship between the probability value and a preset threshold value.

[0146] The above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0147] In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0148] In several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented by other ways. Among them, the above-described device embodiments are only schematic, for example, the division of units can be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0149] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e. they can be located in one place or distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0150] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0151] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the essential part or all or part of the related technology that make contributions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0152] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A method of fast encoding of a 360-degree video, characterized by, The method comprises: obtaining a plurality of video frames in 360-degree video data to be encoded; dividing each of the video frames to obtain a plurality of coding units; for each of the coding units, encoding the coding unit by using a preset coding mode, and respectively extracting features of the coding unit before encoding and the coding unit after encoding to obtain corresponding first feature vectors and second feature vectors; analyzing the first feature vectors and the second feature vectors by using a pre-trained decision tree model to obtain a probability value for terminating remaining coding mode checking, wherein the preset coding mode is at least one coding mode in a plurality of coding modes, and the remaining coding mode is other coding modes in the plurality of coding modes except the preset coding mode; determining, according to a size relationship between the probability value and a preset threshold value, whether to select a coding mode with a lowest rate-distortion cost from the preset coding mode as a target coding mode of the coding unit or to select a coding mode with a lowest rate-distortion cost from the plurality of coding modes as the target coding mode of the coding unit; wherein the first feature vector at least includes: a size of the coding unit, a horizontal stretching degree of a coding tree unit to which the coding unit belongs, an average absolute error value between a video frame to which the coding unit belongs and a reference video frame, and a variance of a difference between the coding unit and a homologous reference coding unit in the reference video frame; and the second feature vector at least includes: a variance of a residual determined by encoding the coding unit by using the preset coding mode, and a rate-distortion cost determined by encoding the coding unit by using the preset coding mode.

2. The method of claim 1, wherein, The method for dividing a video frame to obtain a plurality of coding units comprises: performing equirectangular projection on the video frame to obtain a corresponding planar image; dividing the planar image to obtain a plurality of coding tree units; for each of the coding tree units, performing recursive splitting on the coding tree unit by using a preset division manner to obtain the plurality of coding units.

3. The method of claim 1, wherein, The determination process of the horizontal stretching degree of the coding tree unit to which the coding unit belongs comprises: determining a height of the planar image corresponding to the video frame; determining a vertical coordinate of a center pixel point in the coding tree unit to which the coding unit belongs; determining the horizontal stretching degree of the coding tree unit to which the coding unit belongs by using the following formula according to the height of the planar image and the vertical coordinate of the center pixel point: ; In the formula, represents the horizontal stretching degree of the coding tree unit to which the coding unit belongs, represents the vertical coordinate of the center pixel point, and H represents the height of the planar image, represents the secant function.

4. The method of claim 1, wherein, The method for analyzing the first feature vectors and the second feature vectors by using the pre-trained decision tree model to obtain the probability value for terminating the remaining coding mode checking comprises: determining the size of the coding unit and determining a target decision tree corresponding to the size of the coding unit in the decision tree model, wherein the decision tree model comprises a plurality of decision trees, and each of the decision trees is used for analyzing feature vectors of a coding unit with a certain size; analyzing the first feature vectors and the second feature vectors by using the target decision tree to obtain the probability value for terminating the remaining coding mode checking.

5. The method of claim 1, wherein, The method comprises the following steps: In the case that the probability value is not lower than the preset threshold value, the rate-distortion cost of the coding unit coded by each preset coding mode is determined, and the preset coding mode with the lowest rate-distortion cost is determined as the target coding mode of the coding unit; In the case that the probability value is lower than the threshold value, the rate-distortion cost of the coding unit coded by each coding mode in the plurality of coding modes is determined, and the coding mode with the lowest rate-distortion cost is determined as the target coding mode of the coding unit.

6. The method of claim 1, wherein: The preset coding mode comprises an extended merge mode and an affine mode; The remaining coding mode comprises at least one of an advanced motion vector prediction mode, an intra mode, and a geometric partitioning mode (GPM).

7. An apparatus for fast encoding of a 360-degree video, the apparatus comprising: The method comprises the following steps: An acquisition module is configured to acquire a plurality of video frames in 360-degree video data to be coded; A division module is configured to divide each video frame to obtain a plurality of coding units; A decision module is configured to code each coding unit by using a preset coding mode, and extract features of the coding unit before coding and the coding unit after coding respectively to obtain a first feature vector and a second feature vector; analyze the first feature vector and the second feature vector by using a pre-trained decision tree model to obtain a probability value of terminating remaining coding mode checking, wherein the preset coding mode is at least one coding mode in a plurality of coding modes, and the remaining coding mode is other coding mode in the plurality of coding modes except the preset coding mode; A determination module is configured to determine whether to select the coding mode with the lowest rate-distortion cost from the preset coding mode as the target coding mode of the coding unit or to select the coding mode with the lowest rate-distortion cost from the plurality of coding modes as the target coding mode of the coding unit according to the size relationship between the probability value and a preset threshold value. The first feature vector at least comprises the size of the coding unit, the horizontal stretching degree of the coding tree unit to which the coding unit belongs, the average absolute error value between the video frame to which the coding unit belongs and a reference video frame, and the variance of the difference between the coding unit and a homologous reference coding unit in the reference video frame; and the second feature vector at least comprises the variance of the residual determined by coding the coding unit by using the preset coding mode, and the rate-distortion cost determined by coding the coding unit by using the preset coding mode.

8. A non-volatile storage medium, comprising: The non-volatile storage medium stores a computer program, and a device in which the non-volatile storage medium is located executes the fast encoding method of the 360-degree video according to any one of claims 1 to 6 by running the computer program.

9. A computer program product, characterised in that, The computer program is executed by a processor to implement the fast encoding method of the 360-degree video according to any one of claims 1 to 6. The computer program is executed by a processor to implement the fast encoding method of the 360-degree video according to any one of claims 1 to 6.

Images