A video encoding optimization method and system

Abstract

The present invention relates to video coding technology, and discloses a video coding optimization method and system. A video coding optimization method is applied to a NUMA architecture, and the method includes: detecting the CPU hardware of a server, and obtaining the total number of CPU nodes and the logical cores included in the CPU nodes, where the total number of CPU nodes is M; creating coding threads, and allocating the original frame memory and the reconstructed frame memory according to the number of CPU nodes; encoding the video on an encoder according to the gop id of the current video frame; for the encoded video outputting a video bitstream, and sequentially splicing the bitstreams into a complete bitstream according to the display order of I frames in the bitstream from low to high. The present invention improves the coding parallelism while avoiding remote memory access existing in conventional software coding, and at the same time reduces the consumption of computing resources by the encoder.

Description

Technical Field

[0001] The present invention relates to video encoding technology, and particularly to a video encoding optimization method and system applied to a NUMA architecture. Background Art

[0002] Currently, for software encoders of ultra-high-definition videos, a combination of multiple parallel encoding strategies is adopted to achieve ultra-high-definition real-time transcoding. Commonly used parallel encoding strategies include:

[0003] (1) Intra-frame line-level parallel encoding (realizing simultaneous encoding of multiple lines within a frame by using multiple threads);

[0004] (2) Frame-level parallel encoding (realizing simultaneous encoding of multiple frames by using multiple threads);

[0005] (3) GOP-level parallel encoding (realizing simultaneous encoding of multiple GOPs by using multiple threads).

[0006] Through parallel encoding based on multiple threads, the computing resources of a multi-core CPU are fully mobilized as much as possible. At the same time, to meet the computing requirements of ultra-high-definition video real-time encoding, hardware generally adopts a server with an ultra-multi-core CPU.

[0007] Currently, multi-core CPUs generally adopt the NUMA architecture, that is, "Non-Uniform Memory Access". The NUMA architecture solves the performance bottleneck problem caused by accessing memory through the traditional north bridge on a multi-core CPU. In the NUMA architecture, a server is divided into several nodes (SOCKETS), and each node has a separate CPU and memory. The CPU directly accesses the local address through the memory controller, with fast speed and short latency; it remotely accesses the physical addresses of other nodes through the QPI LINK.

[0008] For multi-threaded software running on a CPU with a NUMA architecture, memory remote access should be minimized as much as possible. Normal multi-threaded encoding, especially 4K / 8K ultra-high-definition encoding, requires accessing a large amount of memory. Among them, the reconstructed frame data and the original frame data are the parts that occupy the largest amount of memory.

[0009] The original frame is mainly used in the pre-analysis stage to calculate image characteristics and encoding complexity, so as to determine encoding quantization parameters and other parameters; the reconstructed frame mainly provides reference pixels for intra-frame and inter-frame prediction. In addition, in the encoding mode selection stage, it is necessary to use the reconstructed image and the original image to calculate encoding distortion. Therefore, the memory storing the reconstructed frame and the original frame is undoubtedly the part with the highest access frequency in encoding.

[0010] When running 4K / 8K ultra-high-definition encoding on a multi-core CPU with a NUMA architecture, since there are dependencies between encoding threads and the dependency conditions do not always hold, threads often enter the WAIT state. When the conditions are met, the threads enter the RUN state again. The CPU schedules threads to implement the switching of thread states. During this process, a thread may switch from one NUMA node to another NUMA node.

[0011] Therefore, the reconstructed frame memory, the original frame memory accessed by the thread, and the encoding thread itself cannot always be guaranteed to be within the same node, and there will inevitably be a large number of remote memory accesses, resulting in the inability to maximize the computing power of the multi-core CPU.

[0012] In video coding standards, there are generally three types of coding frames: I-frames, P-frames, and B-frames. In video coding, GOP (Group of picture) is commonly used to represent a group of images between two I-frames in a video coding sequence. The GOP length represents the number of frames within the GOP and is an important parameter of the encoder.

[0013] GOP parallel encoding is a common means for ultra-high-definition real-time encoding. The Close gop frame structure is adopted, which can ensure that adjacent GOPs can be encoded independently, and the parallelism can be doubled. The encoder repackages the GOP bitstreams in the order of GOPs to form the final complete bitstream. The encoding kernel refers to a software module with complete encoding capabilities; the input of the encoding kernel is the original video image sequence, and the output is the encoded video bitstream.

[0014] For example, in the prior art, the patent application number is: CN202011644043.5201910600394.7; the patent name is a method and system for real-time encoding of 8K ultra-high-definition video, and the patent application date is December 31, 2020. On a NUMA architecture CPU, it limits the encoding task within one node and cannot maximize the computing power of the multi-node CPU. Summary of the Invention

[0015] In view of the problem in the prior art that in video coding on a NUMA architecture CPU, it limits the encoding task within one node and cannot maximize the computing power of the multi-node CPU, the present invention provides a video coding optimization method and system.

[0016] To solve the above technical problems, the present invention is solved by the following technical solutions:

[0017] A video coding optimization method, applied in a NUMA architecture, the method includes:

[0018] Detection of the server, detecting the CPU hardware of the server, and obtaining the number of CPU nodes and the logical cores included in the CPU nodes, where the total number of CPU nodes is M;

[0019] Creation of the encoder, creating encoding threads, and allocating the original frame memory and the reconstructed frame memory according to the number of CPU nodes;

[0020] Encoding of the video, encoding the video on the encoder according to the gop id of the current video frame;

[0021] Composition of the bitstream, for the encoded video, outputting the video bitstream, and splicing the bitstream into a complete bitstream according to the display order of the I frames in the bitstream from low to high.

[0022] Preferably, the method for creating the encoder includes:

[0023] Step 1, initializing and setting the node index variable i = 0;

[0024] Step 2, calling pthread_setaffinity_np to set the thread of the encoder to run on the logical cores of node i;

[0025] Step 3, creating N(i) encoding threads, and allocating P(i) original frames and Q(i) reconstructed frame memories according to the number of CPU nodes, where:

[0026]

[0027]

[0028]

[0029] where i is the number of nodes, taking a positive integer and less than M, N is the total number of threads, P is the total number of original frame memories, and Q is the total number of reconstructed frame memories;

[0030] Step 4, incrementing the node index variable by 1, that is, i = i + 1. When i is less than M, continue with Step 2, otherwise the encoder creation is complete.

[0031] Preferably, the method for encoding the video includes:

[0032] Obtaining the gop id of the current video frame, and updating the gop id, where the gop id is initialized to 0 and the maximum value is M - 1;

[0033] For the video frame with the current gop id of i, obtaining the frame memory pointers from the original frame memory and the reconstructed frame memory of node i and assigning them to the current frame, and copying the original frame image into the original frame memory;

[0034] The encoding threads on the scheduling node i encode the current video frame.

[0035] Preferably, N(i) encoding threads are created, and P(i) original frame memories and Q(i) reconstructed frame memories are allocated according to the number of nodes of the CPU. The allocation method is as follows:

[0036] N(i) = N / M, P(i) = P / M, Q(i) = Q / M;

[0037] where i is the number of nodes, taking positive integers and less than M; N is the total number of threads, P is the total number of original frame memories, and Q is the total number of reconstructed frame memories.

[0038] Preferably, the gop id update method is as follows: after the first frame, every time an I frame or an IDR frame is received, the gop id is incremented by 1. If the gop id is equal to M, the gop id is assigned 0.

[0039] To solve the above technical problems, the present invention also provides a video encoding optimization system, which is applied to the NUMA architecture and characterized by including a server detection module, an encoder creation module, a video encoding module, and a bitstream synthesis module.

[0040] The server detection module is used to detect the CPU hardware of the server and obtain the number of nodes of the CPU and the logical cores included in the nodes of the CPU, where the total number of nodes of the CPU is M.

[0041] The encoder creation module is used to create encoding threads and allocate original frame memories and reconstructed frame memories according to the number of nodes of the CPU.

[0042] The video encoding module encodes the video on the encoder according to the gop id of the current video frame.

[0043] The bitstream synthesis module is used to output the video bitstream of the encoded video and splice the bitstream into a complete bitstream according to the ascending display order of the I frames in the bitstream.

[0044] Preferably, the encoder creation module includes:

[0045] The initialization module is used to initialize and set the node index variable i = 0.

[0046] The thread running module runs the threads of the encoder on the logical cores of node i by calling pthread_setaffinity_np.

[0047] The encoding thread creation module is used to create N(i) encoding threads and allocate the memory for P(i) original frames and Q(i) reconstructed frames according to the number of nodes of the CPU, where:

[0048]

[0049]

[0050]

[0051] where i is the number of nodes, taking positive integers and less than M, N is the total number of threads, P is the total memory of original frames, and Q is the total memory of reconstructed frames;

[0052] The node index judgment module is used to judge the node index variable.

[0053] Preferably, the video encoding module includes:

[0054] The video frame acquisition module is used to acquire the GOP id of the current video frame and update the GOP id, where the GOP id is initialized to 0 and the maximum value is M - 1;

[0055] The video frame encoding module is used to, for the video frame with the current GOP id being i, obtain the frame memory pointer from the original frame memory and the reconstructed frame memory of node i and assign it to the current frame, copy the original frame image to the original frame memory; schedule the encoding threads on node i to encode the current video frame.

[0056] To solve the above technical problems, the present invention also provides an electronic device, which includes: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the above-mentioned video encoding optimization method.

[0057] To solve the above technical problems, the present invention also provides a computer-readable storage medium, in which computer execution instructions are stored, and when the processor executes the computer execution instructions, the above-mentioned video encoding optimization method is implemented.

[0058] Due to the adoption of the above technical solutions, the present invention has significant technical effects:

[0059] First, while improving the encoding parallelism, the present invention avoids the remote memory access existing in conventional software encoding.

[0060] Second, the present invention is applied to the HEVC and AVS3 encoders of ArcVideo, and the transcoding speed of ultra-high-definition videos such as 4K / 8K is comprehensively increased by about 10%, and there is no loss in the coding quality. In the case of real-time coding, the consumption of computing resources by the ArcVideo encoder is reduced. Description of the Drawings

[0061] Figure 1 It is a flowchart of the present invention.

[0062] Figure 2 It is a creation flowchart of the encoder of the present invention.

[0063] Figure 3 It is a schematic diagram of the coding framework of the present invention.

[0064] Wherein:

[0065] NUMA is the abbreviation of Non-Uniform Memory Access, that is, Non-Uniform Memory Access;

[0066] I-frame: Intra-coded picture, an intra-frame coded image frame;

[0067] P-frame: Predictive-coded Picture, a predictive coded image frame;

[0068] B-frame: Bidirectionally predicted picture, a bidirectionally predicted coded image frame;

[0069] GOP: Group of picture (image group), a group of images between two I-frames. Detailed Embodiments

[0070] The present invention will be further described in detail below with reference to the drawings and embodiments.

[0071] Embodiment 1

[0072] A video coding optimization method, applied to a NUMA architecture, the method includes:

[0073] Detection of the server, detecting the CPU hardware of the server, and obtaining the number of CPU nodes and the logical cores included in the CPU nodes, where the total number of CPU nodes is M;

[0074] Creation of the encoder, creating coding threads, and allocating raw frame memory and reconstructed frame memory according to the number of CPU nodes;

[0075] Encoding of the video, encoding the video on the encoder according to the gop id of the current video frame;

[0076] Composition of the bitstream: For the encoded video output video bitstream, the bitstream is spliced into a complete bitstream according to the display order of the I-frames in the bitstream from low to high.

[0077] Preferably, the method for creating the encoder includes:

[0078] Step 1: Initialize the node index variable i = 0;

[0079] Step 2: Call pthread_setaffinity_np to set the thread of the encoder to run on the logical core of node i;

[0080] Step 3: Create N(i) encoding threads, and allocate memory for P(i) original frames and Q(i) reconstructed frames according to the number of nodes of the CPU, where:

[0081]

[0082]

[0083]

[0084] where i is the number of nodes, taking a positive integer and less than M, N is the total number of threads, P is the total memory of the original frames, and Q is the total memory of the reconstructed frames;

[0085] Step 4: Increment the node index variable by 1, i.e., i = i + 1. If i is less than M, continue with Step 2; otherwise, the encoder creation is complete.

[0086] The video encoding method includes:

[0087] Obtain the gop id of the current video frame and update the gop id, where the gop id is initialized to 0 and the maximum value is M - 1;

[0088] For the video frame with the current gop id of i, obtain the frame memory pointers from the original frame memory and the reconstructed frame memory of node i and assign them to the current frame, and copy the original frame image into the original frame memory;

[0089] Schedule the encoding threads on node i to encode the current video frame.

[0090] Create N(i) encoding threads, and allocate P(i) original frames and Q(i) reconstructed frames according to the number of nodes of the CPU. The allocation method is:

[0091] N(i) = N / M, P(i) = P / M, Q(i) = Q / M;

[0092] Wherein, i is the number of nodes, taking a positive integer and less than M; N is the total number of threads, P is the total number of original frame memories, and Q is the total number of reconstructed frame memories.

[0093] The gop id is updated as follows: after the first frame, every time an I frame or an IDR frame is received, the gop id is incremented by 1. If the gop id is equal to M, the gop id is assigned 0.

[0094] Embodiment 2

[0095] Based on Embodiment 1, for the detection of the server CPU hardware situation, for example, in the Linux system, by reading the information in the / proc / cpuinfo file and filtering the keywords "processor" and "physical id", the cpu id and physical id information can be obtained. The cpu id is the id of the logical processor, and the physical id is the NUMA node where the logical processor is located. For example, on an Intel(R) Xeon(R) Gold 6258R CPU, there are 112 logical processors and 2 nodes. Among them, Node 0 (physical id 0) contains logical processors (0 to 27, 56 to 83), and Node 1 (physical id 1) contains logical processors (28 to 55, 84 to 111).

[0096] Assume that the number of encoding threads to be created is N, the number of CPU nodes is M, the number of original frame memories is P frames, and the number of reconstructed frame memories is Q frames.

[0097] The threads, original frames, and reconstructed frame memories are divided into M parts. Considering the computational load balance of the nodes, in this embodiment, an average distribution is adopted. N(i), P(i), and Q(i) can be freely allocated according to the actual application, as long as they satisfy the three equations of N, P, and Q in Embodiment 1; that is, N / M threads, P / M original frame memories, and Q / M reconstructed frame memories are allocated to each node. Here, taking the Linux system and a dual-node Intel(R) Xeon(R) Gold 6258R CPU (M = 2) as an example, the pthread_setaffinity_np function is used to set the affinity between the threads and the logical processors, that is, to set on which logical processors the threads run.

[0098] First, call pthread_setaffinity_np to set the thread to run on all logical processors of node 0. Then, create N / 2 encoding threads and complete the thread initialization work, including the allocation of memory variables used within the threads. Create a memory pool and allocate memory for P / 2 original frames and Q / 2 reconstructed frames. Thereafter, these threads will run on all logical processors of node 0, and the memory accessed within the threads and half of the memory for the reconstructed frames and original frames are also within the physical address of node 0. Similarly, call pthread_setaffinity_np again to set the thread to run on all logical processors of node 1, create the remaining N / 2 encoding threads, complete the thread initialization work, including the allocation of memory variables used within the threads, create a memory pool and allocate memory for P / 2 original frames and Q / 2 reconstructed frames. These threads will run on all logical processors of node 1, and the memory accessed within the threads and half of the memory for the reconstructed frames and original frames are also on the physical address of node 1.

[0099] Embodiment 3

[0100] Based on the above embodiments, this embodiment provides a video encoding optimization system applied to a NUMA architecture, which includes a server detection module, an encoder creation module, a video encoding module, and a bitstream synthesis module.

[0101] Server detection module, which is used to detect the CPU hardware of the server and obtain the number of nodes of the CPU and the logical cores included in the nodes of the CPU, where the total number of nodes of the CPU is M.

[0102] Encoder creation module, which is used to create encoding threads and allocate memory for original frames and reconstructed frames according to the number of nodes of the CPU.

[0103] Video encoding module, which encodes the video on the encoder according to the gop id of the current video frame.

[0104] Bitstream synthesis module, which is used to output the video bitstream after encoding and splice the bitstream into a complete bitstream according to the display order of I frames in the bitstream from low to high.

[0105] The encoder creation module includes:

[0106] Initialization module, which is used to initialize and set the node index variable i = 0.

[0107] Thread running module, which sets the threads of the encoder to run on the logical cores of node i by calling pthread_setaffinity_np.

[0108] An encoding thread creation module, which is used to create N(i) encoding threads and allocate memory for P(i) original frames and Q(i) reconstructed frames according to the number of nodes of the CPU, where:

[0109]

[0110]

[0111]

[0112] where i is the number of nodes, taking positive integers and less than M, N is the total number of threads, P is the total original frame memory number, and Q is the total reconstructed frame memory number;

[0113] A node index judgment module, which is used to judge the node index variable.

[0114] The video encoding module includes:

[0115] A video frame acquisition module, which is used to acquire the gop id of the current video frame and update the gop id, where the gop id is initialized to 0 and the maximum value is M - 1;

[0116] A video frame encoding module, which is used to, for the video frame with the current gop id of i, obtain the frame memory pointer from the original frame memory and the reconstructed frame memory of node i and assign it to the current frame, copy the original frame image into the original frame memory; schedule the encoding threads on node i to encode the current video frame.

[0117] Embodiment 4

[0118] Based on the above embodiments, this embodiment provides an electronic device, which includes:

[0119] At least one processor and a memory;

[0120] The memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the described video encoding optimization method.

[0121] Embodiment 5

[0122] Based on the above embodiments, this embodiment provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the described video encoding optimization method is implemented.

Claims

1. A video encoding optimization method, applied in a NUMA architecture, characterized in that, The method includes: Detecting the server, detecting the CPU hardware of the server, and obtaining the number of CPU nodes and the logical cores included in the CPU nodes, where the total number of CPU nodes is M; Creating an encoder, creating encoding threads, and allocating raw frame memory and reconstructed frame memory according to the number of CPU nodes; Encoding the video, encoding the video on the encoder according to the gopid of the current video frame; Synthesizing the bitstream, outputting a video bitstream for the encoded video, and splicing the bitstream into a complete bitstream according to the display order of I frames in the bitstream from low to high; The method for creating an encoder includes, Step 1, initialize and set the node index variable i = 0; Step 2, call pthread_setaffinity_np to set the thread of the encoder to run on the logical cores of node i; Step 3, create N(i) encoding threads, allocate P(i) raw frames and Q(i) reconstructed frame memories, where: where, i is the node index, taking a positive integer and less than M, N is the total number of threads, P is the total number of raw frame memories, and Q is the total number of reconstructed frame memories; Step 4, increment the node index variable by 1, that is, i = i + 1. If i is less than M, continue with Step 2, otherwise the encoder creation is complete; Obtain the gopid of the current video frame and update the gopid, where the gopid is initialized to 0 and the maximum value is M - 1; For the video frame with the current gopid of i, obtain the frame memory pointer from the raw frame memory and reconstructed frame memory of node i and assign it to the current frame, and copy the raw frame image to the raw frame memory; Schedule the encoding threads on node i to encode the current video frame.

2. The video coding optimization method according to claim 1, wherein Create N(i) encoding threads, allocate P(i) raw frames and Q(i) reconstructed frame memories, and the allocation method is: N(i) = N / M, P(i) = P / M, Q(i) = Q / M; where, i is the number of nodes, taking a positive integer and less than M; N is the total number of threads, P is the total number of raw frame memories, and Q is the total number of reconstructed frame memories.

3. The video coding optimization method according to claim 1, wherein The gopid update method is: after the first frame, every time an I frame or an IDR frame is received, the gopid is incremented by 1. If the gopid is equal to M, the gopid is assigned 0.

4. A video encoding optimization system is applied in a NUMA architecture, and is characterized in that, It includes a server detection module, an encoder creation module, a video encoding module, and a bitstream synthesis module, Server detection module, the server detection module is used to detect the CPU hardware of the server, and obtain the number of CPU nodes and the logical cores included in the CPU nodes, where the total number of CPU nodes is M; Encoder creation module, the encoder creation module is used to create encoding threads, and allocate raw frame memory and reconstructed frame memory according to the number of CPU nodes; Video encoding module, the video encoding module encodes the video on the encoder according to the gopid of the current video frame; Bitstream synthesis module, the bitstream synthesis module is used to output a video bitstream for the encoded video, and splice the bitstream into a complete bitstream according to the display order of I frames in the bitstream from low to high; The encoder creation module includes: Initialization module, the initialization module is used to initialize and set the node index variable i = 0; Thread running module, which runs the thread of the encoder on the logical core of node i by calling pthread_setaffinity_np; Encoding thread creation module, which is used to create N(i) encoding threads, allocate memory for P(i) original frames and Q(i) reconstructed frames, where: where i is the number of nodes, taking positive integers and less than M, N is the total number of threads, P is the total memory number of original frames, and Q is the total memory number of reconstructed frames; Node index judgment module, which is used to judge the node index variable; The video encoding module includes: Video frame acquisition module, which is used to acquire the gopid of the current video frame and update the gopid, where the gopid is initialized to 0 and the maximum value is M - 1; Video frame encoding module, which is used to obtain the frame memory pointer of the current video frame with gopid i from the original frame memory and the reconstructed frame memory of node i, assign it to the current frame, copy the original frame image to the original frame memory; Schedule the encoding threads on node i to encode the current video frame.

5. An electronic device, characterized in that, Including: At least one processor and a memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes a video encoding optimization method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the processor executes the computer-executable instructions, a video encoding optimization method according to any one of claims 1 to 3 is implemented.

Images