360-degree video fast coding method and device

Abstract

The invention discloses a fast coding method and device for a 360-degree video. Comprising the following steps: acquiring a plurality of video frames in 360-degree video data; dividing each video frame to obtain a plurality of coding units; for each coding unit, coding the coding unit by using a preset coding mode, and performing feature extraction on the coding unit and the coded coding unit to obtain a first feature vector and a second feature vector; analyzing the first feature vector and the second feature vector by using a decision tree model to obtain a probability value for terminating the residual coding mode check; and according to a size relationship between the probability value and a preset threshold value, determining to select a coding mode with the lowest rate distortion cost from a preset coding mode or a plurality of coding modes as a target coding mode of the coding unit. According to the method and the device, the technical problem of high resource cost and time cost caused by the fact that a coder needs to perform traversal check on a plurality of coding modes when determining the optimal coding mode of the coding unit is solved.

Description

Technical Field

[0001] The present application relates to the field of video encoding technology, and in particular to a method and device for fast encoding of 360-degree videos. Background Art

[0002] As people's quality of life improves, their demand for video quality is also growing, and they are pursuing a more realistic audio-visual experience. As a video that can bring an immersive experience to users, immersive VR (Virtual Reality) video can just meet users' needs. 360-degree video is a type of immersive VR video, which is usually shot and spliced ​​by multiple cameras, and the resolution is 2-8 times that of ordinary videos. The increase in the amount of video data makes traditional transmission methods unable to cope with it, so video encoding technology has become a key solution.

[0003] Currently, most 360-degree videos are based on the fast encoding algorithm of HEVC (High Efficiency Video Coding), and lack the inter-frame fast encoding algorithm based on VVC (Versatile Video Coding, VVC). In addition, although there are some fast encoding algorithms based on VVC in traditional planar videos, when the fast encoding algorithm based on VVC determines the optimal encoding mode corresponding to each coding unit, the encoder usually needs to perform mode checks on all coding units, where the mode check process is carried out in the detection order of the standard encoder, generally checking the affine mode first; then checking the extended merge mode (including the normal Merge mode, MMVD mode and CIIP mode); then checking the geometric partition mode; then checking the advanced motion vector prediction mode; and finally checking the intra-frame mode. This checking process will bring great resource consumption and time expenditure to the encoder.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method and device for fast encoding of 360-degree videos, so as to at least solve the technical problem that a large amount of resource costs and time costs are consumed because the encoder needs to traverse and check multiple encoding modes when determining the optimal encoding mode of an encoding unit.

[0006] According to one aspect of an embodiment of the present application, a fast encoding method for 360-degree video is provided, comprising: obtaining multiple video frames in 360-degree video data to be encoded; for each video frame, dividing the video frame to obtain multiple encoding units; for each encoding unit, encoding the encoding unit using a preset encoding mode, and performing feature extraction on the encoding unit and the encoded encoding unit respectively to obtain corresponding first feature vectors and second feature vectors; analyzing the first feature vectors and the second feature vectors using a pre-trained decision tree model to obtain a probability value for terminating the remaining encoding mode check, wherein the preset encoding mode is at least one encoding mode among multiple encoding modes, and the remaining encoding modes are other encoding modes among multiple encoding modes except the preset encoding mode; and determining, based on the relationship between the probability value and a preset threshold value, to select a encoding mode with the lowest rate-distortion cost from the preset encoding mode or multiple encoding modes as the target encoding mode of the encoding unit.

[0007] Optionally, the video frame is divided to obtain multiple coding units, including: performing equidistant azimuthal projection on the video frame to obtain a corresponding plane image; dividing the plane image to obtain multiple coding tree units; for each coding tree unit, recursively splitting the coding tree unit using a preset partitioning method to obtain multiple coding units.

[0008] Optionally, the first feature vector includes at least one of the following: the size of the coding unit, the horizontal stretching degree of the coding tree unit to which the coding unit belongs, the mean absolute error value between the video frame to which the coding unit belongs and the reference video frame, and the variance of the difference between the coding unit and the co-located reference coding unit in the reference video frame; and the second feature vector includes at least one of the following: the rate-distortion cost of encoding the coding unit using a preset coding mode and the variance of the residual.

[0009] Optionally, the process of determining the horizontal stretching degree of the coding tree unit to which the coding unit belongs includes: determining the height of the plane image corresponding to the video frame; determining the vertical coordinate of the central pixel point in the coding tree unit to which the coding unit belongs; and determining the horizontal stretching degree of the coding tree unit to which the coding unit belongs by using the following formula according to the height of the plane image and the vertical coordinate of the central pixel point:

[0010]

[0011] Where α represents the horizontal stretching degree of the coding tree unit to which the coding unit belongs, y represents the ordinate of the center pixel, H represents the height of the plane image, and sec(·) represents the secant function.

[0012] Optionally, a pre-trained decision tree model is used to analyze the first eigenvector and the second eigenvector to obtain a probability value for terminating the remaining coding mode check, including: determining the size of the coding unit, and determining a target decision tree in the decision tree model that corresponds to the size of the coding unit, wherein the decision tree model includes multiple decision trees, and each decision tree is used to analyze the eigenvectors of coding units of a certain size; and using the target decision tree to analyze the first eigenvector and the second eigenvector to obtain a probability value for terminating the remaining coding mode check.

[0013] Optionally, based on the relationship between the probability value and a preset threshold value, determine to select a coding mode with the lowest rate-distortion cost from a preset coding mode or multiple coding modes as a target coding mode of the coding unit, including: when the probability value is not lower than the preset threshold value, determine the rate-distortion cost of encoding by the coding unit using each preset coding mode, and use the preset coding mode with the lowest rate-distortion cost as the target coding mode of the coding unit; when the probability value is lower than the threshold value, determine the rate-distortion cost of encoding by the coding unit using each coding mode within the multiple coding modes, and use the coding mode with the lowest rate-distortion cost as the target coding mode of the coding unit.

[0014] Optionally, the preset coding modes include: extended merge mode, affine mode; the remaining coding modes include at least one of the following: advanced motion vector prediction mode, intra-frame prediction mode, geometric partitioning mode.

[0015] According to another aspect of an embodiment of the present application, a fast encoding device for 360-degree video is also provided, including: an acquisition module, used to acquire multiple video frames in 360-degree video data to be encoded; a division module, used to divide the video frame for each video frame to obtain multiple encoding units; a decision module, used to encode the encoding unit for each encoding unit using a preset encoding mode, and extract features from the encoding unit and the encoded encoding unit respectively to obtain corresponding first feature vectors and second feature vectors; using a pre-trained decision tree model to analyze the first feature vector and the second feature vector to obtain a probability value for terminating the remaining encoding mode check, wherein the preset encoding mode is at least one encoding mode among multiple encoding modes, and the remaining encoding modes are other encoding modes among multiple encoding modes except the preset encoding mode; a determination module, used to determine, based on the relationship between the probability value and the preset threshold value, to select a coding mode with the lowest rate-distortion cost from the preset coding mode or multiple coding modes as the target coding mode of the coding unit.

[0016] According to another aspect of an embodiment of the present application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned 360-degree video fast encoding method by running the computer program.

[0017] According to another aspect of an embodiment of the present application, a computer program product is also provided, which includes a stored computer program, wherein when the computer program is executed by a processor, the above-mentioned method for fast encoding of 360-degree video is implemented.

[0018] In an embodiment of the present application, multiple video frames in the 360-degree video data to be encoded are obtained; each video frame is divided to obtain multiple encoding units; for each encoding unit, the encoding unit is encoded using a preset encoding mode, and the encoding unit and the encoded encoding unit are respectively feature extracted to obtain the corresponding first feature vector and second feature vector; the first feature vector and the second feature vector are analyzed using a pre-trained decision tree model to obtain a probability value for terminating the remaining encoding mode check, wherein the preset encoding mode is at least one encoding mode in multiple encoding modes, and the remaining encoding mode is other encoding modes in multiple encoding modes except the preset encoding mode; according to the relationship between the probability value and the preset threshold value, the encoding mode with the lowest rate distortion cost is determined from the preset encoding mode or multiple encoding modes as the target encoding mode of the encoding unit. Through the above-mentioned encoding mode determination method, the purpose of quickly predicting and terminating redundant encoding mode detection in advance is achieved, thereby achieving the technical effect of reducing the encoding complexity of 360-degree video under VVC, and further solving the technical problem that the encoder needs to consume a lot of resource costs and time costs because the related technology needs to traverse each encoding mode in turn when determining the optimal encoding mode of the encoding unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 It is a hardware structure block diagram of a computer terminal (or mobile device) for an optional fast encoding method for 360-degree video according to an embodiment of the present application;

[0021] Figure 2 is a flowchart of an optional method for fast encoding of 360-degree video according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of an optional plane image according to an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of an optional quadtree partitioning principle according to an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of an optional division of coding tree units according to an embodiment of the present application;

[0025] Figure 6 is an optional mapping diagram of the latitude region to which the coding tree unit belongs and the horizontal stretching degree according to an embodiment of the present application;

[0026] Figure 7 It is a schematic structural diagram of an optional 360-degree video fast encoding device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] In addition, the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set up between this system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0030] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0031] Virtual Reality (VR): As the name implies, it is the combination of virtuality and reality (using specific technology to generate a virtual situation, but giving people a sense of reality). Theoretically speaking, virtual reality technology is a computer simulation system that can create and experience a virtual world. It uses computers to generate a simulated environment and immerse users in it. Virtual reality technology uses real-life data, electronic signals generated by computer technology, and combines them with various output devices to transform them into phenomena that people can feel. These phenomena can be real objects in reality or substances that are invisible to our naked eyes, expressed through three-dimensional models.

[0032] Video coding technology: refers to the method of converting the original video format file into another video format file through compression technology. The most important codec standards in video streaming are ITU's H.261, H.263, and H.264.

[0033] HEVC (High Efficiency Video Coding): Also known as H.265 and MPEG-Hpart2, it is one of the video compression standards and one of the potential successors to the widely used AVC (Advanced Video Coding). Compared with AVC, HEVC provides about twice the data compression ratio at the same video quality level, or significantly improves video quality at the same bit rate. It supports resolutions up to 8192×4320, including 8K Ultra High Definition (UHD).

[0034] VVC (Versatile Video Coding) is a new generation of international video coding standard, named Versatile Video Coding, also known as H.266. Its main goal is to improve the existing HEVC technology to provide higher compression performance, while optimizing for emerging applications (including 360° 360-degree video and high dynamic range imaging HDR).

[0035] Coding Tree Unit (CTU): It is the largest processing unit in the video encoder. The size of the coding tree unit is 64×64 in HEVC and 128×128 in VVC. Generally, a coding tree unit can contain multiple most basic processing units - coding units (CU). In HEVC, CU is generally a square structure, such as 8×8, 16×16 and 32×32; in VVC, since VVC introduces the quadtree nested multi-type tree division method, CU not only has a square structure, but also other rectangular structures, such as 8×4, 64×8, etc.

[0036] Affine Mode: It is a new inter-frame prediction technology introduced by VVC. By introducing the affine prediction mode, VVC can better handle video sequences containing rotation, scaling, and miscutting movements, so as to more accurately predict the motion information in the video sequence.

[0037] Merge Mode: It is a motion vector (MV) prediction technology proposed in HEVC, which can be regarded as a coding technology. This mode directly predicts the MV of the current block using the MVs of adjacent blocks in the time domain and spatial domain. In HEVC, the encoder will establish an MV candidate list for the current prediction unit, and the list includes 5 candidate MVs. By traversing these 5 candidate MVs and selecting the index of the optimal MV through the rate-distortion criterion, it only needs to pass the index to the decoder. In VVC, the MV candidate list includes 6 candidate MVs. In addition, the index of the best candidate MV for each CU in the merge mode can be encoded using unary binarization, where the first bin of the best index uses context coding, and the other bins use bypass coding.

[0038] Merge Mode with Motion Vector Difference (MMVD) mode: After the merge mode candidates in H.265 / HEVC are determined, the best candidate motion vector is selected from the candidate list and directly used as the motion vector of the current coding block. There may be a certain error between the actual application vector and the motion vector. To this end, H.266 / VVC adds motion vector difference (MVD) on the basis of the original MV to make the motion vector more accurate. That is, by fine-tuning the candidate motion vector to better match the actual motion vector. This method can reduce the prediction error and improve the accuracy of the motion vector.

[0039] Combined Inter and Intra Prediction (CIIP) mode: refers to the use of both inter-frame prediction and intra-frame prediction in video coding. This hybrid prediction method aims to improve the efficiency and quality of video coding. In CIIP, the encoder can flexibly choose to use inter-frame prediction or intra-frame prediction according to the characteristics of the current frame and the scene content to better utilize the temporal and spatial correlation in the video sequence, thereby achieving more efficient compression coding.

[0040] Geometric Partitioning Mode (GPM): is a block partitioning mode for inter-frame prediction, which better adapts to complex motion situations and image content by dividing blocks into different geometric shapes, such as squares, parallelograms, triangles, and curves. Usually, in a video containing moving objects, complex motion situations may occur at the edges. In this case, the geometric partitioning mode can better capture the outline and motion direction of the object.

[0041] Advanced Motion Vector Prediction (AMVP) mode: It is a new MV prediction technology proposed in H.265 / HEVC, and H.266 / VVC still uses this technology and improves it on the basis of HEVC. AMVP uses the correlation between motion vectors in the spatial and temporal domains to establish a candidate prediction MV list for the current prediction unit; the encoder can select the best prediction MVP from it and perform differential encoding on the MVP (that is, obtain the real MV through motion search and encode the motion vector difference); the decoder establishes the same list and only needs to predict the index of the MVP in the list and the motion vector residual MVD to calculate the current predicted MV.

[0042] Intra Mode: It is a technology that uses the correlation of adjacent pixels in the same frame and the reconstructed pixels in the adjacent area of ​​the current block to predict the pixels in the current block. The current CU can use the reconstructed pixels of the adjacent CU to predict the pixels in the current CU. After intra prediction, the prediction residual is transformed, quantized, etc., which can effectively remove the spatial redundancy of the video. Usually, intra prediction technology generally includes Planar mode (this mode is suitable for situations where pixel values ​​change slowly), DC mode (suitable for large flat areas) and other 33 angle prediction modes; in VVC, the angle prediction mode is expanded to 65, and matrix weighted intra prediction (Matrix Weighted Intra Prediction, MIP) and intra sub-partition (Intra Sub-Partitions, ISP) and other technologies are introduced to further improve the coding performance and image quality.

[0043] ERP (Equi-Rectangular Projection): It is a simple projection method that maps longitude lines to vertical lines with constant spacing and latitude lines to horizontal lines with constant spacing.

[0044] DT (Decision Tree): Based on the known probability of occurrence of various situations, a decision analysis method is used to construct a decision tree to obtain the probability that the expected value of the net present value is greater than or equal to zero, evaluate project risks, and determine its feasibility. It is a graphical method that intuitively uses probability analysis. Since this decision branch is drawn into a graph that looks like a tree branch, it is called a decision tree. In machine learning, a decision tree is a prediction model that represents a mapping relationship between object attributes and object values.

[0045] QP (Quantization Parameter): It is a key parameter that controls the degree of video compression. The higher the QP value, the stronger the compression and the smaller the file size, but the image quality will decrease; conversely, the lower the QP value, the weaker the compression and the larger the file size, but the image quality is better. In the video encoding process, the quantization parameter directly affects the trade-off between the video bit rate and visual quality.

[0046] Example 1

[0047] According to an embodiment of the present application, an embodiment of a method for fast encoding of a 360-degree video is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing a fast encoding method for 360-degree video is shown. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0049] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0050] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the fast encoding method of 360-degree video in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the fast encoding method of 360-degree video of the above-mentioned application is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0051] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0052] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0053] In the above operating environment, Figure 2 is a flowchart of an optional method for fast encoding of 360-degree video according to an embodiment of the present application, such as Figure 2 As shown, the method at least includes steps S202-S208, wherein:

[0054] Step S202: Acquire multiple video frames in the 360-degree video data to be encoded.

[0055] In the technical solution provided in step S202, the above-mentioned 360-degree video data is a panoramic video stream data that meets the pre-processing requirements (resolution, frame rate), which can be obtained by shooting with a panoramic camera, or generated by a series of continuously captured 360-degree panoramic images, or by using professional software to splice multiple traditional video clips into a virtual sphere to form a panoramic video effect. Each video frame in the 360-degree video is a collection of continuous or non-continuous image frames in the video sequence, and each image frame is a still image of a 360-degree panoramic view. Unlike traditional videos, each image frame of a 360-degree video contains a complete field of view all around.

[0056] Step S204: divide each video frame into multiple coding units.

[0057] In the technical solution provided in step S204, the encoder traverses each video frame in the 360-degree video data, and divides the currently traversed video frame to obtain coding units (CUs), which are basic spatial structural units used to process and encode image data in the video encoding process.

[0058] Step S206, for each coding unit, encode the coding unit using a preset coding mode, and extract features from the coding unit and the encoded coding unit respectively to obtain corresponding first feature vectors and second feature vectors; use a pre-trained decision tree model to analyze the first feature vector and the second feature vector to obtain a probability value for terminating the remaining coding mode checks.

[0059] Considering that when encoding a 360-degree video using the H.266 / VVV inter-frame coding method, it is necessary to perform coding mode checks on multiple coding units obtained by dividing each video frame in the 360-degree video in turn. Due to the high complexity of the coding mode checking process and the large number of coding units, it is very redundant for the encoder to perform a coding mode checking process for each coding unit, and it will also bring great resource consumption and time overhead to the encoder.

[0060] To this end, it is proposed in an embodiment of the present application that, for each coding unit, the coding unit is first encoded using a preset coding mode, and features are extracted for the coding unit and the encoded coding unit, respectively, to obtain a corresponding first feature vector and a second feature vector, wherein the preset coding mode is at least one coding mode among multiple coding modes; then the first feature vector and the second feature vector of the coding unit are analyzed using a pre-trained decision tree model, and a probability value for terminating the check of the remaining coding modes is output, which can reflect whether the simple coding mode is the preferred coding mode for the current coding unit, thereby stopping the check of the remaining coding modes, wherein the remaining coding modes are other coding modes except the preset coding mode among the multiple coding modes corresponding to the H.266 / VVV inter-frame prediction.

[0061] Step S208: Determine, based on the relationship between the probability value and the preset threshold value, to select a coding mode with the lowest rate-distortion cost from a preset coding mode or multiple coding modes as a target coding mode for the coding unit.

[0062] In the technical solution provided in step S208, the encoder compares the probability value determined in the above step S206 with the preset threshold value, and determines based on the comparison result whether to select the coding mode with the lowest rate-distortion cost from the preset coding modes as the target coding mode of the coding unit, or to select the coding mode with the lowest rate-distortion cost from multiple coding modes as the target coding mode of the coding unit.

[0063] The above method of this embodiment is further introduced below.

[0064] As an optional implementation, in the technical solution provided in the above step S204, the method may include:

[0065] Step S2041, performing equidistant azimuthal projection on the video frame to obtain a corresponding plane image.

[0066] Specifically, in the technical solution provided in step S2041 above, generally, the video frame in the 360-degree video is usually a mapping of a spherical surface. Therefore, in order to map the spherical image corresponding to the video frame to the plane image, the encoder can use equidistant azimuthal projection to convert the longitude and latitude information of each pixel point on the spherical surface corresponding to the video frame into coordinates on the plane image, and obtain the following: Figure 3 The plane image shown is used to maintain the continuity and integrity of the spherical image corresponding to the video frame.

[0067] Specifically, the equidistant azimuthal projection samples each pixel point on the sphere corresponding to the video frame, and directly maps the longitude of the pixel point on the sphere to the horizontal coordinate of the plane image, ranging from 0 to the width of the spherical image, and the latitude is mapped to the vertical coordinate of the plane image, ranging from 0 to half the height of the spherical image, thereby obtaining a 2:1 rectangular plane. Through this projection method, the pixels on the top and bottom of the sphere are stretched horizontally, while the pixels near the equator remain almost intact. This stretching process ensures that the video frame can still fully represent the 360-degree panoramic information after projection.

[0068] It should be noted that after the video frame is equidistantly projected, the resolution of the planar image can be adjusted according to the subsequent encoding requirements to ensure image quality. For example, the encoder can downsample the high-resolution video frame to 4432×2216 (such as the processing in the VVC standard encoder) according to the encoding requirements, so as to process a more reasonable amount of data during the encoding process, and restore its original resolution by upsampling at the decoding end.

[0069] Step S2042, dividing the planar image into multiple coding tree units.

[0070] In the technical solution provided in step S2042, the encoder unit may initialize the maximum size of the coding tree unit in advance, such as the maximum size of the encoder unit allowed by VVC is 128×128 pixels, but in some cases, the size of the maximum encoder unit may be set smaller to adapt to different coding scenarios, which may vary according to the video resolution and coding parameters. Then, the encoder may divide the planar image into one or more coding tree units of fixed size.

[0071] Step S2043: for each coding tree unit, recursively split the coding tree unit using a preset partitioning method to obtain multiple coding units.

[0072] In the technical solution provided in step S2043, the encoder can traverse each coding tree unit and use Figure 4 The quad-tree (Quad-Tree-Splitting) method shown in the figure is used for recursive partitioning, where: Figure 4 The quadtree shown has multiple types of trees nested in it, including quadtrees, horizontal binary trees, vertical binary trees, horizontal ternary trees, and vertical ternary trees. That is, the encoder can first analyze the current coding tree unit to determine whether further segmentation is required, wherein the basis for the determination can be the complexity of the image content, motion information, and evaluation of coding efficiency; if the segmentation is determined, quadtree segmentation is performed to segment the current coding tree unit into four equal sub-units, each sub-unit size is half the size of the original coding tree unit, and these four sub-units can be used as candidates for coding tree units or coding units, depending on the subsequent segmentation decision. Then, the above recursive decision and quadtree segmentation process are repeated for each newly generated sub-unit until a certain termination condition is met, wherein the termination condition may include reaching the minimum coding unit size, reaching a predetermined tree depth, or further segmentation will not bring significant improvement in coding performance. Finally, when the recursive segmentation terminates, each final unsegmented sub-unit is regarded as a coding unit, and the coding unit is the smallest independent coding block in the encoding process, which can be predicted, transformed, quantized, etc. separately.

[0073] Taking the size of the coding tree unit as 128*128 as an example, the coding tree unit can be Figure 5 The method shown is used to divide the image into multiple coding units of different sizes, including: 64*64, 16*16, 8*8, 8*32, 32*8, etc.

[0074] After the encoder obtains multiple coding units corresponding to each video frame through the technical solution provided by the above steps S2041-S2043, for each coding unit, the encoder can encode the coding unit using a preset mode, wherein the above preset coding mode can be: an extended merge mode (i.e., a merge mode with motion vector difference) and an affine mode.

[0075] Specifically, regarding the determination of the above-mentioned preset coding mode, the probability of different coding units selecting the coding mode can be obtained according to historical experience and sorted into the following Table 1.

[0076] Table 1

[0077]

[0078]

[0079] It can be seen from Table 1 above that under different quantization parameters, the probability of coding units of different sizes selecting coding modes is arranged in order from large to small, and the preferred order of the coding modes can be roughly organized as: extended merge mode, affine mode, advanced motion vector prediction mode, intra-frame prediction mode and geometric partitioning mode.

[0080] Therefore, the "preset coding mode" in the embodiment of the present application can be an extended merge mode and an affine mode, wherein the extended merge mode includes: a normal merge mode, a merge mode with motion vector difference, and a hybrid intra-frame and inter-frame prediction mode. The "remaining coding modes" can be: an advanced motion vector prediction mode, an intra-frame prediction mode, a geometric partitioning mode, etc.

[0081] It should be noted that for some specific scenarios, the preferred order of selecting coding modes for coding units of different sizes is still different. For example, the possibility of selecting an affine mode for a larger block is about 10% higher than that for a smaller block on average; in a smaller block, the probability of the intra-frame prediction mode and the geometric partitioning mode being selected is significantly increased, which may be because smaller blocks are usually used to encode image areas containing high details or complex textures; in addition, the intra-frame coding mode allows these areas to use their own spatial redundancy for encoding, rather than relying on information from other frames, thereby better preserving image details and quality. In order to further accurately determine the preset coding mode of coding units of various sizes, the embodiment of the present application can also perform time complexity calculations on coding units of different sizes in the same coding environment based on the above Table 1, so as to calculate the time complexity of encoding coding units of different sizes according to different coding modes, thereby adjusting the preset coding mode of coding units of such sizes according to the time complexity.

[0082] For example, taking a coding unit of size 32*32 as the experimental object, the time complexity of encoding using five coding modes, namely, extended merge mode, affine mode, advanced motion vector prediction mode, intra-frame prediction mode, and geometric partitioning mode, is calculated respectively, and the results are shown in Table 2 below.

[0083] Table 2

[0084]

[0085]

[0086] It is not difficult to see from Table 2 that for 32*32 coding units, the complexity of the geometric partition mode is the lowest, followed by the affine mode, while the complexity of the extended merge mode is close to that of the intra-frame prediction mode, and the complexity of the advanced motion vector prediction mode is the highest. Among them, the overall mode complexity increases because the extended merge mode includes the normal merge mode, the merge mode with motion vector difference, and the mixed intra-frame and inter-frame prediction, not because the complexity of the normal merge mode is high.

[0087] Furthermore, after determining the preset coding mode through the above steps, the encoder can perform feature extraction on the coding unit and the encoded coding unit respectively to obtain the corresponding first feature vector and second feature vector respectively. The first feature vector includes: the size of the coding unit, the horizontal stretching degree of the coding tree unit to which the coding unit belongs, the mean absolute error value (WholeSubAvg) between the video frame to which the coding unit belongs and the reference video frame, the variance of the difference between the coding unit and the co-located reference coding unit in the reference video frame (BlockSubDif), the quantization parameter (QP), and the parent mode (PMode); and the second feature vector includes: the rate-distortion cost (RDCost) of the coding unit encoded using the preset coding mode, and the variance of the residual (Resi_Differ).

[0088] For the horizontal stretching degree of the current coding tree unit to which the current coding unit belongs, the encoder may determine the horizontal stretching degree by:

[0089] Step 1: Determine the height of the plane image corresponding to the video frame;

[0090] Step 2: determine the vertical coordinate of the center pixel point in the current coding tree unit to which the current coding unit belongs;

[0091] Step 3: According to the height of the plane image and the vertical coordinate of the central pixel, the horizontal stretching degree of the current coding tree unit to which the current coding unit belongs is determined by the following formula:

[0092]

[0093] Wherein, α represents the horizontal stretching degree of the current coding tree unit to which the current coding unit belongs, y represents the ordinate of the center pixel, H represents the height of the plane image, and sec(·) represents the secant function.

[0094] It should be noted that, in order to associate the stretching degree with the spherical image corresponding to the coding tree unit, the embodiment of the present application can divide the latitude range of the spherical image corresponding to the video frame into multiple parts according to the size of the spherical image corresponding to the video frame, and define the corresponding relationship between each horizontal stretching degree and the latitude area, such as Figure 6As shown, the latitude region where the coding tree unit to which the coding unit belongs is located can be used as the first feature vector.

[0095] The encoder can determine the mean absolute error value between the current video frame to which the current coding unit belongs and the reference video frame by the following method: determine the reference video frame of the current video frame to which the current coding unit belongs from multiple video frames, and determine the mean absolute error value between the pixel value of each pixel point in the current video frame and the pixel value of each pixel point in the reference video frame. The mean absolute error value reflects the degree of motion of the current sequence to a certain extent. Generally, when the mean absolute error value is small, it indicates that the sequence motion is relatively simple, so a simpler encoding mode can be selected for encoding; conversely, if the mean absolute error value is large, it indicates that the sequence changes violently over time and the motion is complex, so a more complex encoding mode can be selected for encoding.

[0096] For the variance of the difference between the current coding unit and the co-located reference coding unit in the reference video frame, the encoder can determine it by the following method: determine the reference video frame of the current video frame to which the current coding unit belongs in multiple video frames, and divide the reference video frame to obtain multiple reference coding units; determine the co-located reference coding unit with the same position coordinates as the current coding unit in multiple reference coding units; determine the variance of the difference between the pixel value of each pixel point in the current coding unit and the pixel value of each pixel point in the co-located reference coding unit. Among them, the larger the variance of the difference between the current coding unit and the co-located reference coding unit, the greater the difference between the two, so more data is needed to encode, so a complex encoding mode can be selected for encoding; conversely, if the variance of the difference is small, it means that the difference between the two is small, so a simpler encoding mode can be selected for encoding to save the amount of encoded data. In short, the variance of the difference can guide the encoder to select a suitable mode to balance the encoding quality and the amount of data.

[0097] For the parent mode, it means that the current coding unit may continue to use the coding mode used by its previous generation unit. Therefore, by utilizing this inheritance, the coding mode of the current coding unit can be roughly determined.

[0098] For the quantization parameters, the encoder can set them according to the actual application scenario, and this application does not impose any specific restrictions on this.

[0099] For the rate-distortion cost of encoding the current coding unit using the preset coding mode, the encoder can determine it by the following method: determine the difference between the pixel value of each pixel in the current coding unit and the pixel value of each pixel in the current coding unit after encoding, and use the difference as the distortion; based on the bit rate, distortion and preset trade-off factor corresponding to the first coding method, use the rate-distortion cost function to determine the rate-distortion cost of encoding the coding unit using the preset coding mode.

[0100] The variance of the residual can be determined by the encoder in the following way: using a preset prediction model (such as an intra-frame prediction model or an inter-frame prediction model) to predict the pixel value of each pixel in the current coding unit, and determining the difference between the original pixel value and the predicted pixel value of each pixel in the current coding unit, using the difference as the residual, and determining the variance of the residual. If the residual is large, it means that there may be a large motion change or scene content difference between the current frame and the reference frame, which usually requires more bits to encode the motion vector and residual data, which may result in a higher rate-distortion cost; on the contrary, if the residual is small, it means that the current frame is highly similar to the reference frame, and can be effectively encoded with fewer bits to achieve a lower rate-distortion cost.

[0101] It should be noted that the above reference frames can be determined by random access. For example, if the current frame is the 2nd frame, the 1st frame can be used as a reference frame; in addition, since VVC supports bidirectional prediction, the current frame can have two reference frames (a forward reference frame and a backward reference frame). For example, if the current frame is the 20th frame, the 1st frame and the 40th frame can be used as reference frames.

[0102] After obtaining the first eigenvector and the second eigenvector corresponding to the current encoding by the above method, the encoder can determine the termination residual encoding mode check probability value corresponding to the current encoding unit by the following method, including:

[0103] First, the size of the coding unit is determined, and a target decision tree corresponding to the size of the coding unit in the decision tree model is determined; then, the first eigenvector and the second eigenvector are analyzed using the target decision tree to obtain a probability value for terminating the remaining coding mode check.

[0104] Specifically, the pre-trained decision tree model in the embodiment of the present application includes multiple decision trees, and each decision tree is used to analyze the feature vectors of coding units of a certain size.

[0105] This is because the distribution characteristics of the coding modes selected for coding units of different sizes are different. For details, refer to Table 1 above. Therefore, it is very necessary to classify and train coding units of different sizes. In the inter-frame coding process of VVC, a total of 27 coding units of different sizes are generated. If all coding units of different sizes are discussed separately, 27 situations need to be discussed, which is a very complex and redundant process.

[0106] In order to reduce the workload as much as possible, the embodiment of the present application proposes an efficient classification scheme, that is, 27 coding units of different sizes are divided into multiple categories. This classification is not only based on the physical size of the coding unit, but also takes into account their behavior patterns and selection tendencies in actual coding applications. The closer the size of the coding unit, the more similar their distribution patterns are.

[0107] Specifically, the embodiments of the present application are preferably divided into 5 categories, and the specific classification results are as follows:

[0108] (1) 128×128, 128×64 and 64×128 CU;

[0109] (2) 64×64, 32×32, 64×32, 32×64, 64×16 and 16×64 CU;

[0110] (3) 64×8, 8×64, 16×32 and 32×16 CU;

[0111] (4) 16×16, 16×8, 8×16, 8×8, 32×8 and 8×32 CU;

[0112] (5) 64×4, 32×4, 16×4, 8×4, 4×64, 4×32, 4×16 and 4×8 CU.

[0113] Through the above division strategy, the originally complicated analysis process can be simplified into discussions of five typical cases, significantly reducing the complexity and workload of the research. This not only improves the efficiency of coding decisions, but also provides strong theoretical support for further optimizing VVC coding strategies and improving coding performance.

[0114] Further, for each type of coding unit, the historical coding data of the coding unit is obtained, wherein the historical coding data includes: the first eigenvector and the second eigenvector (training sample) of the coding unit, and the optimal coding mode (sample label) of the coding unit; then, the training sample is used as input and the sample label is used as output to train the decision tree corresponding to the coding unit, and the model parameters such as the depth of the tree, the minimum number of samples of the leaf node, etc. are adjusted through cross-validation to optimize the prediction accuracy of the model and the efficiency of early termination. Therefore, the first eigenvector and the second eigenvector of the current coding unit are analyzed by the trained decision tree to determine the probability value of the remaining coding mode being selected as the optimal coding mode. If this probability is lower than a certain threshold, it can be concluded that the remaining coding mode is unlikely to provide the optimal coding result, so it is recommended that the encoder terminate the check of the remaining coding mode in advance, so the "probability value of the remaining coding mode being selected as the optimal coding mode" is the "probability value of terminating the check of the remaining coding mode" in the embodiment of the present application. In addition, in the embodiment of the present application, the optimal termination thresholds (i.e., threshold values) corresponding to the decision trees corresponding to the above five types of coding units can be set to: 0.75, 0.75, 0.80, 0.85, and 0.80, respectively.

[0115] Furthermore, after the encoder obtains the probability value of terminating the remaining coding mode check of the current coding unit through the above steps, the encoder may decide the coding mode of the current coding unit by the following method, including:

[0116] When the probability value of terminating the remaining coding mode check is not lower than the preset threshold value, the rate-distortion cost of each preset coding mode used by the coding unit is determined, and the preset coding mode with the lowest rate-distortion cost is used as the target coding mode of the coding unit.

[0117] In the above case, it means that the encoder has found a coding mode with good enough performance. At this time, the encoder terminates the check of the remaining coding modes in advance, which can greatly reduce the encoding time and computing resource consumption. At the same time, the rate-distortion cost of the encoder encoding the coding unit using the preset coding mode is determined, and the preset coding mode with the lowest rate-distortion cost is used as the target coding mode of the coding unit.

[0118] When the probability value of terminating the remaining coding mode check is lower than the threshold value, the rate-distortion cost of encoding by the coding unit using each coding mode in the multiple coding modes is determined, and the coding mode with the lowest rate-distortion is used as the target coding mode of the coding unit.

[0119] In the above case, it means that after checking the simple coding modes, the encoder does not find that these simple coding modes can provide the best coding performance based on the current rate-distortion cost and image quality evaluation, and will continue to check the remaining coding modes. In other words, the simple coding modes have not yet reached the encoder's expected standards for coding performance and quality. At this time, the encoder continues to check the remaining coding modes in turn, and determines the rate-distortion cost of the preset coding mode and the remaining coding modes for encoding the coding unit, and uses the coding mode with the lowest rate-distortion cost as the target coding mode for the coding unit.

[0120] The encoder determines the best encoding mode of the encoding unit through the above method, which can not only avoid redundant checks on complex encoding strategies that are unlikely to be the optimal encoding mode, thereby significantly reducing encoding time and improving encoding efficiency; it can also dynamically determine the next step of the encoding process based on the characteristics of the encoding unit, rather than blindly following a fixed mode detection order, which helps the encoder process video content more efficiently, especially when processing 360-degree videos. In addition, the above method can minimize encoding time while ensuring that video quality is not excessively sacrificed, thereby achieving a good balance between encoding quality and encoding speed.

[0121] In addition, in order to verify the beneficial effects of the above method, the embodiment of the present application uses RandomAccess to configure the VVC test model (VTM), where the QP values ​​are set to 22, 27, 32 and 37. The following Table 3 lists the general test video sequences required for the verification process, including information such as resolution, number of encoded frames, frame rate and bit depth.

[0122] Table 3

[0123]

[0124] The fast encoding method provided in the embodiment of the present application is used to encode each 360-degree video in the above Table 3, and the obtained encoding performance (including bit rate change, time saving) is compared with the encoding performance obtained by encoding with the same version of the standard encoder (i.e., a full mode check is performed for each encoding unit), and the BDBR (Bjontegaard Delta Bit Rate, bit rate change) % comparison results and TS (Time Saving, time saving) % comparison results shown in Table 4 are obtained. Among them, the above BDBR refers to the percentage of change between the bit rate after the proposed algorithm encoding is completed and the bit rate after the standard encoder is completed while ensuring the same quality of the compressed video during the video encoding process. Generally, if this value is negative, it means that the encoding efficiency has been improved; if this value is positive, it means that the encoding efficiency has been reduced. TS refers to the improvement in encoding speed, which is an important indicator for measuring encoder performance. TS can be calculated by comparing the time required for two encoders to complete the same video encoding, wherein a higher TS value indicates that the new encoding method is faster than the reference encoding method and can complete the encoding task in a shorter time.

[0125] Table 4

[0126]

[0127]

[0128] It is not difficult to see from the above Table 4 that the BDBR comparison results obtained by encoding using the encoding method provided by this solution and the standard encoder of the same version on different video sequences increased by an average of 0.36%, while the TS comparison results decreased by an average of 21.35%.

[0129] Based on the scheme defined by the above steps S202 to S208, it can be known that in an embodiment, the encoder extracts features from each coding unit before and after encoding using a preset coding mode, and uses a decision tree model to analyze the extracted feature vectors to decide whether to continue to perform mode checks on the remaining coding modes, thereby avoiding the encoder blindly following a fixed mode detection order, which greatly increases the encoding time and greatly reduces the encoding efficiency. It can be seen that through the technical solution of the embodiment of the present application, the encoder can intelligently decide whether to continue complex coding mode detection, achieving the purpose of quickly predicting and terminating redundant coding mode detection in advance, thereby achieving the technical effect of reducing the encoding complexity of the VVC encoding process for 360-degree video, and further solving the technical problem that the encoder needs to traverse and check multiple coding modes when determining the optimal coding mode of the coding unit, resulting in a large amount of resource cost and time cost.

[0130] Example 2

[0131] Based on Embodiment 1 of the present application, an embodiment of a 360-degree video fast encoding device is also provided, and the device executes the 360-degree video fast encoding method of the above embodiment when running. Figure 7 is a schematic structural diagram of an optional 360-degree video fast encoding device according to an embodiment of the present application, such as Figure 7 As shown, the 360-degree video fast encoding device at least includes an acquisition module 72, a division module 74, a decision module 76, and a determination module 78, wherein:

[0132] An acquisition module 72, configured to acquire a plurality of video frames in the 360-degree video data to be encoded;

[0133] A division module 74 is used for dividing each video frame into multiple coding units;

[0134] A decision module 76 is used to encode each coding unit using a preset coding mode, and perform feature extraction on the coding unit and the encoded coding unit respectively to obtain a corresponding first feature vector and a second feature vector; analyze the first feature vector and the second feature vector using a pre-trained decision tree model to obtain a probability value for terminating the remaining coding mode check, wherein the preset coding mode is at least one coding mode among the multiple coding modes, and the remaining coding modes are other coding modes among the multiple coding modes except the preset coding mode;

[0135] The determination module 78 is used to determine, based on the relationship between the probability value and the preset threshold value, to select a coding mode with the lowest rate-distortion cost from a preset coding mode or multiple coding modes as a target coding mode for the coding unit.

[0136] It should be noted that the various modules in the above-mentioned 360-degree video fast encoding device can be program modules (for example, a set of program instructions for implementing a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0137] Example 3

[0138] According to an embodiment of the present application, a non-volatile storage medium is also provided, in which a program is stored, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the fast encoding method of 360-degree video in Example 1.

[0139] Optionally, the device where the non-volatile storage medium is located implements the following steps by running the program: obtaining multiple video frames in the 360-degree video data to be encoded; for each video frame, dividing the video frame to obtain multiple encoding units; for each encoding unit, encoding the encoding unit using a preset encoding mode, and extracting features from the encoding unit and the encoded encoding unit respectively to obtain corresponding first feature vectors and second feature vectors; using a pre-trained decision tree model to analyze the first feature vector and the second feature vector to obtain a probability value for terminating the remaining encoding mode check, wherein the preset encoding mode is at least one encoding mode among multiple encoding modes, and the remaining encoding modes are other encoding modes among multiple encoding modes except the preset encoding mode; based on the relationship between the probability value and the preset threshold value, determining to select the encoding mode with the lowest rate-distortion cost from the preset encoding mode or multiple encoding modes as the target encoding mode of the encoding unit.

[0140] According to an embodiment of the present application, a computer program product is also provided, which includes a stored computer program, wherein when the computer program is executed by a processor, the method for fast encoding of 360-degree video in embodiment 1 is implemented.

[0141] Optionally, the computer program executes the following steps: obtaining multiple video frames in the 360-degree video data to be encoded; for each video frame, dividing the video frame to obtain multiple encoding units; for each encoding unit, encoding the encoding unit using a preset encoding mode, and extracting features from the encoding unit and the encoded encoding unit respectively to obtain corresponding first feature vectors and second feature vectors; using a pre-trained decision tree model to analyze the first feature vector and the second feature vector to obtain a probability value for terminating the remaining encoding mode check, wherein the preset encoding mode is at least one encoding mode among the multiple encoding modes, and the remaining encoding modes are other encoding modes among the multiple encoding modes except the preset encoding mode; based on the relationship between the probability value and the preset threshold value, determining to select the encoding mode with the lowest rate-distortion cost from the preset encoding mode or the multiple encoding modes as the target encoding mode of the encoding unit.

[0142] According to an embodiment of the present application, a processor is also provided, which is used to run a program, wherein the fast encoding method of 360-degree video in Example 1 is executed when the program is running.

[0143] Optionally, the following steps are executed when the program is running: multiple video frames in the 360-degree video data to be encoded are obtained; for each video frame, the video frame is divided to obtain multiple encoding units; for each encoding unit, the encoding unit is encoded using a preset encoding mode, and features are extracted for the encoding unit and the encoded encoding unit respectively to obtain corresponding first feature vectors and second feature vectors; the first feature vectors and the second feature vectors are analyzed using a pre-trained decision tree model to obtain a probability value for terminating the remaining encoding mode check, wherein the preset encoding mode is at least one encoding mode among the multiple encoding modes, and the remaining encoding modes are other encoding modes among the multiple encoding modes except the preset encoding mode; based on the relationship between the probability value and the preset threshold value, the encoding mode with the lowest rate-distortion cost is determined to be selected from the preset encoding mode or multiple encoding modes as the target encoding mode of the encoding unit.

[0144] According to an embodiment of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors; a memory for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are implemented to run the programs, wherein the program is configured to execute the fast encoding method of 360-degree video in the above-mentioned embodiment 1 when running.

[0145] Optionally, the processor is configured to execute the following steps through a computer program: obtaining multiple video frames in the 360-degree video data to be encoded; for each video frame, dividing the video frame to obtain multiple encoding units; for each encoding unit, encoding the encoding unit using a preset encoding mode, and extracting features from the encoding unit and the encoded encoding unit respectively to obtain corresponding first feature vectors and second feature vectors; analyzing the first feature vectors and the second feature vectors using a pre-trained decision tree model to obtain a probability value for terminating the remaining encoding mode check, wherein the preset encoding mode is at least one encoding mode among multiple encoding modes, and the remaining encoding modes are other encoding modes among multiple encoding modes except the preset encoding mode; based on the relationship between the probability value and the preset threshold value, determining to select the encoding mode with the lowest rate-distortion cost from the preset encoding mode or multiple encoding modes as the target encoding mode of the encoding unit.

[0146] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0147] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0148] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0149] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0150] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0151] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0152] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A fast encoding method for 360-degree video, characterized in that: include: Obtain multiple video frames in the 360-degree video data to be encoded; For each of the video frames, dividing the video frame to obtain a plurality of coding units; For each of the coding units, the coding unit is encoded using a preset coding mode, and features are extracted for the coding unit and the encoded coding unit respectively to obtain a corresponding first feature vector and a second feature vector; the first feature vector and the second feature vector are analyzed using a pre-trained decision tree model to obtain a probability value for terminating the remaining coding mode check, wherein the preset coding mode is at least one coding mode among multiple coding modes, and the remaining coding modes are other coding modes among the multiple coding modes except the preset coding mode; According to the relationship between the probability value and the preset threshold value, it is determined to select a coding mode with the lowest rate-distortion cost from the preset coding mode or the multiple coding modes as the target coding mode of the coding unit.

2. The method according to claim 1, characterized in that The video frame is divided into multiple coding units, including: Performing equidistant azimuthal projection on the video frame to obtain a corresponding plane image; Dividing the planar image to obtain a plurality of coding tree units; For each of the coding tree units, the coding tree unit is recursively divided using a preset division method to obtain the multiple coding units.

3. The method according to claim 2, characterized in that The first feature vector at least includes: the size of the coding unit, the horizontal stretching degree of the coding tree unit to which the coding unit belongs, the mean absolute error value between the video frame to which the coding unit belongs and the reference video frame, and the variance of the difference between the coding unit and the co-located reference coding unit in the reference video frame; The second feature vector at least includes: a rate-distortion cost and a variance of a residual when the encoding unit uses the preset encoding mode for encoding.

4. The method according to claim 3, characterized in that The process of determining the horizontal stretching degree of the coding tree unit to which the coding unit belongs includes: Determine the height of the plane image corresponding to the video frame; Determine the ordinate of a central pixel point in a coding tree unit to which the coding unit belongs; According to the height of the plane image and the vertical coordinate of the central pixel point, the horizontal stretching degree of the coding tree unit to which the coding unit belongs is determined by using the following formula: Wherein, α represents the horizontal stretching degree of the coding tree unit to which the coding unit belongs, y represents the ordinate of the central pixel point, H represents the height of the plane image, and sec(·) represents the secant function.

5. The method according to claim 1, characterized in that Analyzing the first feature vector and the second feature vector using a pre-trained decision tree model to obtain a probability value of terminating the remaining coding mode check includes: Determine the size of the coding unit, and determine a target decision tree corresponding to the size of the coding unit in the decision tree model, wherein the decision tree model includes a plurality of decision trees, and each decision tree is used to analyze feature vectors of coding units of a certain size; The first feature vector and the second feature vector are analyzed using the target decision tree to obtain a probability value for terminating the residual coding mode check.

6. The method according to claim 1, characterized in that Determining, according to a magnitude relationship between the probability value and a preset threshold value, to select a coding mode with the lowest rate-distortion cost from the preset coding mode or the multiple coding modes as a target coding mode of the coding unit includes: When the probability value is not lower than a preset threshold value, determining the rate-distortion cost of encoding by the encoding unit using each of the preset encoding modes, and taking the preset encoding mode with the lowest rate-distortion cost as the target encoding mode of the encoding unit; When the probability value is lower than the threshold value, the rate-distortion cost of encoding by the encoding unit using each encoding mode in the multiple encoding modes is determined, and the encoding mode with the lowest rate-distortion is used as the target encoding mode of the encoding unit.

7. The method according to claim 1, characterized in that The preset encoding modes include: extended merge mode and affine mode; The remaining coding modes include at least one of the following: advanced motion vector prediction mode, intra mode, and geometric partitioning mode.

8. A fast encoding device for 360-degree video, characterized in that: include: An acquisition module, used for acquiring multiple video frames in the 360-degree video data to be encoded; A division module, used for dividing each of the video frames into multiple coding units; A decision module is used to encode each of the coding units using a preset coding mode, and perform feature extraction on the coding unit and the encoded coding unit respectively to obtain a corresponding first feature vector and a second feature vector; analyze the first feature vector and the second feature vector using a pre-trained decision tree model to obtain a probability value for terminating the remaining coding mode check, wherein the preset coding mode is at least one coding mode among multiple coding modes, and the remaining coding modes are other coding modes among the multiple coding modes except the preset coding mode; A determination module is used to determine, based on the size relationship between the probability value and a preset threshold value, to select a coding mode with the lowest rate-distortion cost from the preset coding mode or the multiple coding modes as the target coding mode of the coding unit.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, wherein the device where the non-volatile storage medium is located executes the fast encoding method of 360-degree video according to any one of claims 1 to 7 by running the computer program.

10. A computer program product, characterized in that include: A computer program, wherein when the computer program is executed by a processor, the method for fast encoding of 360-degree video according to any one of claims 1 to 7 is implemented.

Images