A method and system for implementing real-time encoding optimization at 8K 120fps
By using memory pools, buffers, multi-threading mechanisms, and NUMA core binding technology, real-time encoding of 8K 120fps was achieved, solving the problem of excessively long memory copy time at high frame rates, improving encoding efficiency and stability, and ensuring the continuity and synchronization of the video stream.
Patent Information
- Application Number
- CN202311043125.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-18
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-08-18
AI Technical Summary
Existing encoding technologies are insufficient to meet the real-time encoding requirements of 8K 120fps, especially at high frame rates where memory copying time is too long, leading to difficulties in video processing. Common hardware such as GPUs and CPUs are also unable to achieve real-time encoding.
By reducing memory copy time through a memory pool mechanism, using a buffer and multi-threading mechanism to perform parallel encoding of multiple encoding instances, and utilizing NUMA and core binding mechanisms to make reasonable use of CPU and memory resources, real-time encoding of 8K 120fps is achieved.
It significantly improves 8K 120fps encoding performance, enhances encoding efficiency and stability, reduces the risk of data loss, ensures the continuity and synchronization of video streams, and provides technical scalability.
Smart Images

Figure CN117061765B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of ultra-high-definition video encoding technology, and in particular to a method and system for achieving real-time encoding optimization of 8K 120fps. Background Technology
[0002] In recent years, video technology has seen remarkable leaps and advancements, evolving from initial standard definition and high definition to 4K and 8K ultra-high definition. Currently, more and more large-scale events and professional industries support and adopt 8K ultra-high definition video output, leading to a growing demand for 8K ultra-high definition encoding. For 8K ultra-high definition encoding technology, 8K offers four times the pixel resolution of 4K, resulting in more detailed image quality. Especially in this digital age, audiences and professional producers have increasingly higher demands for visual experience. In certain applications, such as virtual reality and real-time playback of large events, traditional frame rates of 50fps and 60fps are gradually becoming insufficient; high resolution and high frame rates have become basic requirements. However, high frame rates of 100fps and 120fps significantly increase the technical difficulty of real-time video encoding.
[0003] While the high frame rate of 120fps provides a detailed and vivid visual experience, it also presents challenges to video processing technology. Each frame at 120fps has approximately 8.33ms of processing time, making real-time video decoding and output somewhat difficult. Specifically, for 8K video, the memory copy time alone can exceed 8.33ms, not to mention the challenges of high-quality encoding. Existing encoding technologies, although relatively efficient, struggle to meet such real-time demands, and most hardware on the market, such as common GPUs and CPUs, faces a significant challenge in achieving real-time processing of 8K 120fps. Summary of the Invention
[0004] The purpose of this invention is to address the aforementioned technical problems by proposing a method and system for optimizing real-time encoding of 8K 120fps. This method reduces memory copying time through a memory pool mechanism, enables multiple encoding instances to encode different GOP segments in parallel through buffering and multi-threading mechanisms, and utilizes NUMA and core-binding mechanisms to more rationally utilize the physical machine's CPU and memory resources, ultimately achieving real-time encoding of 8K 120fps. The objective of this invention can be achieved through the following technical solutions:
[0005] This invention provides a method for optimizing real-time encoding at 8K 120fps. Through buffering and multi-threading mechanisms, multiple encoding instances encode different GOP fragments in parallel, including the following steps:
[0006] Step S1: Create an input memory pool to store frame addresses, and initialize the encoder used to receive and process input frames.
[0007] Step S2: The encoder receives the frame address of the input frame through the GOP node in the GOP memory buffer pool and puts the frame address into the frame list of the GOP node until the length of the frame list reaches the preset GOP length.
[0008] Step S3: The GOP buffer creates X parallel instance sub-threads according to the configured number of NUMA nodes X, and each instance sub-thread is bound to a NUMA node. It encodes the input frame corresponding to the frame address in the GOP node and writes the encoded frame to the output buffer.
[0009] Step S4: For the encoded frames in the output buffer, the DTS values of the encoded frames are corrected and sorted by the sub-thread, while the main thread retrieves the encoded frames from the output buffer.
[0010] Step S5: Encapsulate and output the extracted encoded frame.
[0011] Preferably, in step S2, the encoder receives the frame address of the input frame through the GOP node in the GOP memory buffer pool, and puts the frame address into the frame list of the GOP node until the length of the frame list reaches the preset GOP length, further including...
[0012] Step S201: Extract a frame address from the input memory pool, store the video frame to be encoded in the memory location associated with the frame address, and encode it as an input frame using the encoder;
[0013] In step S202, a free GOP node is extracted from the GOP buffer pool in the encoder, and the frame address of the input frame is directly put into the frame list of the GOP node until the predetermined length of the GOP node is reached. The GOP node enters the GOP buffer for encoding processing, and a free GOP node is extracted from the GOP buffer pool for use by the next input frame.
[0014] Preferably, in step S3, the GOP buffer starts several parallel instance sub-threads to encode the input frames corresponding to the frame addresses in the GOP nodes, forming encoded frames which are then written to the output buffer. This further includes...
[0015] Step S301: When a GOP node enters the GOP buffer for encoding, the GOP buffer wakes up the instance child thread to perform parallel encoding on the GOP node.
[0016] In step S302, each instance sub-thread extracts a GOP node from the GOP buffer and begins encoding;
[0017] Step S303: When the instance sub-thread finishes encoding an input frame, the frame address corresponding to the input frame is put back into the input memory pool and marked as idle;
[0018] Step S304: After the instance sub-thread completes the encoding of the input frames corresponding to all frame addresses in the entire GOP node, it writes the encoded frames to the output buffer and puts the GOP node back into the GOP buffer pool as an idle GOP node for continued use.
[0019] Preferably, in step S4, for the encoded frames in the output buffer, the DTS values of the encoded frames are corrected and sorted by a sub-thread, further including...
[0020] Step S411: The child thread detects the encoded frames in the output buffer. If there is no data in the output buffer, it waits for a new encoded frame to enter.
[0021] Step S412: When there is a new encoded frame in the output buffer, DTS correction is performed on the encoded frame, and the frame is sorted in ascending order according to the corrected DTS to ensure that the DTS of the encoded frame in the header is minimized.
[0022] Step S413: Continue to check the output buffer, and repeat steps S411 and S413.
[0023] Preferably, in step S4, the main thread simultaneously retrieves the encoded frame from the output buffer, further including...
[0024] Step S421: Use the DTS of the encoded frame in the processed header as the initial expected DTS;
[0025] Step S422: Retrieve the encoded frames in the header to obtain the DTS;
[0026] Step S423: Compare the DTS of the encoded frame in the header with the expected DTS:
[0027] If the DTS of the encoded frame is equal to the expected DTS, the frame is directly extracted and the next step is performed; if the DTS of the encoded frame is greater than the expected DTS, the frame is not extracted, and the process waits for more than 2 GOP lengths without finding a matching video frame DTS, then the video frame is directly extracted and the expected DTS is updated to the DTS of this encoded frame; if the DTS of the encoded frame is less than the expected DTS, the expected DTS is updated to the DTS of this encoded frame and the frame is directly extracted.
[0028] Step S424: After the video frame is extracted, the expected DTS = DTS of the video frame + 1; and return to step S422 to continue the operation.
[0029] Based on the same inventive concept, the present invention also provides a system for implementing 8K 120fps real-time encoding optimization, which executes the above-described method for implementing 8K 120fps real-time encoding optimization, including:
[0030] The frame receiving and management module receives the video frame to be encoded, extracts a frame address from the input memory pool, stores the video frame in the memory location associated with the frame address, and uses it as an input frame for encoding by the encoding processing module.
[0031] The encoding processing module receives the frame address of the input frame through the GOP memory buffer pool and stores it in the frame list of the GOP node. When the length of the frame list reaches the preset GOP length, the GOP buffer wakes up multiple instance sub-threads to perform parallel encoding to form encoded frames, and stores the encoded frames in the output buffer. The sub-threads then correct and sort the DTS of the encoded frames, and finally the main thread retrieves the encoded frames from the output buffer.
[0032] The encapsulation output module encapsulates the extracted encoded frames according to a preset format and structure before outputting the video stream.
[0033] Preferably, the frame receiving management module includes a video frame receiving unit and a frame address allocation unit;
[0034] The video frame receiving unit is used to receive video frames to be encoded;
[0035] The frame address allocation unit is used to extract a frame address from the input memory pool, store the video frame to be encoded in the memory location associated with the frame address, and encode it as an input frame through the encoder.
[0036] Preferably, the encoding processing module includes an input frame buffer allocation unit, an instance sub-thread startup unit, an instance sub-thread encoding unit, and a frame output sorting adjustment unit;
[0037] The input frame buffer allocation unit retrieves a free GOP node from the GOP buffer pool and directly puts the frame address of the input frame into the frame list of the GOP node.
[0038] In the instance sub-thread startup unit, when the frame list length of the GOP node reaches the predetermined length of the GOP node, the GOP node enters the GOP buffer. The GOP buffer wakes up the instance sub-thread to perform parallel encoding of the GOP node and retrieves a free GOP node from the GOP buffer pool for use in the next input frame.
[0039] Each instance sub-thread encoding unit extracts a GOP node from the GOP buffer and begins encoding. When an instance sub-thread finishes encoding an input frame, it puts the corresponding frame address back into the input memory pool and marks it as idle. When an instance sub-thread finishes encoding all the input frames corresponding to all frame addresses in the entire GOP node, it writes the encoded frame to the output buffer and puts the GOP node back into the GOP buffer pool as an idle GOP node for continued use.
[0040] The frame output sorting adjustment unit, for the encoded frames in the output buffer, corrects and sorts the DTS values of the encoded frames through a sub-thread, while the main thread retrieves the encoded frames from the output buffer.
[0041] Based on the same inventive concept, the present invention also provides a device for implementing real-time encoding optimization for 8K 120fps, comprising:
[0042] Memory, which stores computer-readable instructions;
[0043] When the processor executes computer-readable instructions, it implements the above-described method for achieving real-time encoding optimization at 8K 120fps.
[0044] Based on the same inventive concept, the present invention also provides a readable storage medium for implementing 8K 120fps real-time encoding optimization, comprising: a processing program stored on the readable storage medium, wherein when the processing program is executed by a processor, the above-described method for implementing 8K 120fps real-time encoding optimization is implemented.
[0045] Compared with the prior art, the present invention has at least one of the following technical advantages:
[0046] This invention reduces the time spent on memory copying by using an input memory pool mechanism, enables multiple encoding instances to encode different GOP fragments in parallel through a GOP buffer and a multi-threading mechanism, and makes more rational use of the physical machine's CPU and memory resources by utilizing NUMA and core binding mechanisms, thus significantly improving the encoding performance of 8K 120fps. Under suitable hardware conditions, it can achieve real-time encoding output of 8K 120fps.
[0047] (1) Improved efficiency and performance optimization. By using the input memory pool mechanism, the frequent memory allocation and release were reduced, thereby improving memory utilization efficiency. At the same time, multiple parallel instance sub-threads and NUMA core binding mechanism were used to process different GOPs in parallel, enabling multiple coding tasks to be performed simultaneously, which greatly accelerated the overall coding efficiency.
[0048] (2) Improved stability and reliability. The use of a buffer management mechanism ensured the stability and continuity of data and reduced the risk of data loss and corruption. At the same time, it provided a DTS correction scheme to prevent the problem of inconsistent timestamps in the video stream and ensured the continuity and synchronization of the video stream.
[0049] (3) Scalability: Due to the adoption of multi-threading, buffer management and NUMA technologies, each part of the method can be optimized or extended independently, which facilitates future technology upgrades or expansions. Attached Figure Description
[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below:
[0051] Figure 1 This is a flowchart of the steps in the method for implementing real-time encoding optimization at 8K 120fps according to the present invention;
[0052] Figure 2 This is a flowchart illustrating the process of the method for achieving real-time encoding optimization at 8K 120fps according to the present invention.
[0053] Figure 3 This is a flowchart of the DTS correction steps in the method for achieving 8K 120fps real-time encoding optimization according to the present invention;
[0054] Figure 4 This is a system architecture diagram of the system used in this invention to achieve real-time encoding optimization for 8K 120fps. Detailed Implementation
[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0056] Example 1
[0057] The purpose of this invention is to address the aforementioned technical problems by proposing a method and system for optimizing real-time encoding of 8K 120fps. This method reduces memory copying time through a memory pool mechanism, enables multiple encoding instances to encode different GOP segments in parallel through buffering and multi-threading mechanisms, and utilizes NUMA and core-binding mechanisms to more rationally utilize the physical machine's CPU and memory resources, ultimately achieving real-time encoding of 8K 120fps. The objective of this invention can be achieved through the following technical solutions:
[0058] This invention provides a method for optimizing real-time encoding at 8K 120fps. Through buffering and multi-threading mechanisms, multiple encoding instances encode different GOP fragments in parallel. (See also...) Figure 1 and Figure 2 This includes the following steps:
[0059] Step S1: Create an input memory pool to store frame addresses to solve the problem of excessive time consumption caused by memory copying of uncompressed input frames. The size of the input memory pool is generally set to the number of parallel instances of GOP (group of pictures) N * the number of frames in a GOP * the amount of YUV data in a frame * 2. At the same time, the encoder used to receive and process input frames is initialized.
[0060] Step S2: The encoder receives the frame address of the input frame through the GOP node in the GOP memory buffer pool and puts the frame address into the frame list of the GOP node until the length of the frame list reaches the preset GOP length.
[0061] Preferably, in step S2, the encoder receives the frame address of the input frame through the GOP node in the GOP memory buffer pool, and puts the frame address into the frame list of the GOP node until the length of the frame list reaches the preset GOP length, further including...
[0062] Step S201: Extract a frame address from the input memory pool, store the video frame to be encoded in the memory location associated with the frame address, and encode it as an input frame using the encoder;
[0063] In step S202, a free GOP node is extracted from the GOP buffer pool in the encoder, and the frame address of the input frame is directly put into the frame list of the GOP node until the predetermined length of the GOP node is reached. The GOP node enters the GOP buffer for encoding processing, and a free GOP node is extracted from the GOP buffer pool for use by the next input frame.
[0064] Step S3: The GOP buffer creates X parallel instance sub-threads according to the configured number of NUMA nodes X, and each instance sub-thread is bound to a NUMA node. It encodes the input frame corresponding to the frame address in the GOP node and writes the encoded frame to the output buffer.
[0065] Preferably, in step S3, the GOP buffer starts several parallel instance sub-threads to encode the input frames corresponding to the frame addresses in the GOP nodes, forming encoded frames which are then written to the output buffer. This further includes...
[0066] Step S301: When a GOP node enters the GOP buffer for encoding, the GOP buffer wakes up the instance child thread to perform parallel encoding on the GOP node.
[0067] In step S302, each instance sub-thread extracts a GOP node from the GOP buffer and begins encoding;
[0068] Step S303: When the instance sub-thread finishes encoding an input frame, the frame address corresponding to the input frame is put back into the input memory pool and marked as idle;
[0069] Step S304: After the instance sub-thread completes the encoding of the input frames corresponding to all frame addresses in the entire GOP node, it writes the encoded frames to the output buffer and puts the GOP node back into the GOP buffer pool as an idle GOP node for continued use.
[0070] Step S4: For the encoded frames in the output buffer, the DTS values of the encoded frames are corrected and sorted by a sub-thread, while the main thread retrieves the encoded frames from the output buffer. The PTS of this encoded frame is returned based on the PTS of the input frame and does not require processing. The DTS needs to be recalculated and corrected. The returned DTS is recalculated based on the current PTS value, the initial PTS value, and the initial DTS value before use.
[0071] Preferably, please refer to Figure 3 In step S4, for the encoded frames in the output buffer, the DTS values of the encoded frames are corrected and sorted by a sub-thread, further including...
[0072] Step S411: The child thread detects the encoded frames in the output buffer. If there is no data in the output buffer, it waits for a new encoded frame to enter.
[0073] Step S412: When there is a new encoded frame in the output buffer, DTS correction is performed on the encoded frame, and the frame is sorted in ascending order according to the corrected DTS to ensure that the DTS of the encoded frame in the header is minimized.
[0074] Since encoding may include B-frames, the PTS output by the GOP node after encoding may not be increasing, so it cannot be used as a sorting criterion. DTS, however, is increasing, and DTS is used as the sorting criterion. Furthermore, because frames from adjacent GOP nodes input by different encoding instances are not necessarily adjacent, the DTS order returned by encoding cannot be directly used as a sorting criterion; DTS correction needs to be performed on the returned frames after GOP node encoding.
[0075] The correction process is as follows: Record the timestamps (PTS) and DTS of the first returned encoded frame as the initial PTS (start_PTS) and initial DTS (start_DTS). Record the timestamp (PTS) of the first frame returned by the current GOP as start_PTS_gop. The DTS offset of the first frame returned by the current GOP is DTS_delta = start_PTS_gop - start_PTS. The DTS_delta of non-first frames returned by the current GOP is the previous frame's DTS_delta + 1. This ensures the DTS order of different GOPs as well as the DTS order of the same GOP. Finally, the corrected DTS of the current returned frame is: DTS = start_DTS + DTS_delta.
[0076] Note: PTS: Presentation Timestamp. DTS: Decoding Timestamp.
[0077] Step S413: Continue to check the output buffer, and repeat steps S411 and S413.
[0078] Preferably, in step S4, the main thread simultaneously retrieves the encoded frame from the output buffer, further including...
[0079] Step S421: Use the DTS of the encoded frame in the processed header as the initial expected DTS;
[0080] Step S422: Retrieve the encoded frames in the header to obtain the DTS;
[0081] Step S423: Compare the DTS of the encoded frame in the header with the expected DTS:
[0082] If the DTS of the encoded frame is equal to the expected DTS, the frame is directly extracted and the next step is performed; if the DTS of the encoded frame is greater than the expected DTS, the frame is not extracted, and the process waits for more than 2 GOP lengths without finding a matching video frame DTS, then the video frame is directly extracted and the expected DTS is updated to the DTS of this encoded frame; if the DTS of the encoded frame is less than the expected DTS, the expected DTS is updated to the DTS of this encoded frame and the frame is directly extracted.
[0083] Step S424: After the video frame is extracted, the expected DTS = DTS of the video frame + 1; and return to step S422 to continue the operation.
[0084] Step S5: Encapsulate and output the extracted encoded frame.
[0085] Example 2
[0086] This invention also provides a system for implementing real-time encoding optimization for 8K 120fps. For the method described above for implementing real-time encoding optimization for 8K 120fps, please refer to [link / reference]. Figure 4 ,include,
[0087] The frame receiving and management module receives the video frame to be encoded, extracts a frame address from the input memory pool, stores the video frame in the memory location associated with the frame address, and uses it as an input frame for encoding by the encoding processing module.
[0088] The encoding processing module receives the frame address of the input frame through the GOP memory buffer pool and stores it in the frame list of the GOP node. When the length of the frame list reaches the preset GOP length, the GOP buffer wakes up multiple instance sub-threads to perform parallel encoding to form encoded frames, and stores the encoded frames in the output buffer. The sub-threads then correct and sort the DTS of the encoded frames, and finally the main thread retrieves the encoded frames from the output buffer.
[0089] In video coding, a GOP (Group of Pictures) is a sequence of frames that begins with an I-frame (keyframe) and is followed by a series of P-frames (predictive coded frames) and B-frames (bidirectional predictive coded frames). The length of a GOP is typically defined as the number of frames between one I-frame and the next.
[0090] Therefore, "whether the GOP length has been reached" refers to whether the number of frames currently added to the GOP node's frame list has reached the predetermined GOP length. If this length has been reached or exceeded, the GOP can be considered complete and proceed to the next step. If this length has not yet been reached, then it is necessary to continue waiting for more frames to be added to the GOP.
[0091] The encapsulation output module encapsulates the extracted encoded frames according to a preset format and structure before outputting the video stream.
[0092] Preferably, the frame receiving management module includes a video frame receiving unit and a frame address allocation unit;
[0093] The video frame receiving unit is used to receive video frames to be encoded;
[0094] The frame address allocation unit is used to extract a frame address from the input memory pool, store the video frame to be encoded in the memory location associated with the frame address, and encode it as an input frame through the encoder.
[0095] Preferably, the encoding processing module includes an input frame buffer allocation unit, an instance sub-thread startup unit, an instance sub-thread encoding unit, and a frame output sorting adjustment unit;
[0096] The input frame buffer allocation unit retrieves a free GOP node from the GOP buffer pool and directly puts the frame address of the input frame into the frame list of the GOP node.
[0097] In the instance sub-thread startup unit, when the frame list length of the GOP node reaches the predetermined length of the GOP node, the GOP node enters the GOP buffer. The GOP buffer wakes up the instance sub-thread to perform parallel encoding of the GOP node and retrieves a free GOP node from the GOP buffer pool for use in the next input frame.
[0098] Each instance sub-thread encoding unit extracts a GOP node from the GOP buffer and begins encoding. When an instance sub-thread finishes encoding an input frame, it puts the corresponding frame address back into the input memory pool and marks it as idle. When an instance sub-thread finishes encoding all the input frames corresponding to all frame addresses in the entire GOP node, it writes the encoded frame to the output buffer and puts the GOP node back into the GOP buffer pool as an idle GOP node for continued use.
[0099] The frame output sorting adjustment unit, for the encoded frames in the output buffer, corrects and sorts the DTS values of the encoded frames through a sub-thread, while the main thread retrieves the encoded frames from the output buffer.
[0100] Based on the same inventive concept, the present invention also provides a device for implementing real-time encoding optimization for 8K 120fps, comprising:
[0101] Memory, which stores computer-readable instructions;
[0102] When the processor executes computer-readable instructions, it implements the above-described method for achieving real-time encoding optimization at 8K 120fps.
[0103] Based on the same inventive concept, the present invention also provides a readable storage medium for implementing 8K 120fps real-time encoding optimization, comprising: a processing program stored on the readable storage medium, wherein when the processing program is executed by a processor, the above-described method for implementing 8K 120fps real-time encoding optimization is implemented.
[0104] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.
Claims
1. A method for implementing 8K 120fps real-time encoding optimization, a plurality of encoding instances encode different GOP slices in parallel through a buffer and a multi-thread mechanism, characterized in that, Includes the following steps: Step S1: Create an input memory pool to store frame addresses, and initialize the encoder used to receive and process input frames. In step S2, the encoder receives the frame address of the input frame through the GOP node in the GOP memory buffer pool, and puts the frame address into the frame list of the GOP node until the length of the frame list reaches the preset GOP length. Step S3: The GOP buffer creates X parallel instance sub-threads according to the configured number of NUMA nodes X, and each instance sub-thread is bound to a NUMA node. The input frame corresponding to the frame address in the GOP node is encoded to form an encoded frame and written to the output buffer. Step S4: For the encoded frames in the output buffer, the DTS values of the encoded frames are corrected and sorted by a sub-thread, while the main thread retrieves the encoded frames from the output buffer; wherein, in step S4, the DTS values of the encoded frames in the output buffer are corrected and sorted by a sub-thread, including... Step S411: The sub-thread detects the encoded frame in the output buffer, and waits for a new encoded frame to enter when there is no data in the output buffer; Step S412: When there is a new encoded frame in the output buffer, the encoded frame is DTS corrected and sorted in ascending order according to the corrected DTS to ensure that the DTS of the encoded frame in the header is minimized. Step S413: Continue to detect the output buffer, and repeat steps S411 and S413. In step S4, the simultaneous extraction of the encoded frame from the output buffer by the main thread includes: Step S421: Use the DTS of the encoded frame of the processed header as the initial expected DTS; Step S422: Retrieve the encoded frame in the header to obtain the DTS; Step S423: Compare the DTS of the encoded frame in the header with the expected DTS: If the DTS of the encoded frame is equal to the expected DTS, the frame is directly extracted and the next step is performed; if the DTS of the encoded frame is greater than the expected DTS, the frame is not extracted, and the process waits for more than two GOP lengths without finding a matching DTS for the video frame, then the video frame is directly extracted and the expected DTS is updated to the DTS of this encoded frame; if the DTS of the encoded frame is less than the expected DTS, the expected DTS is updated to the DTS of this encoded frame, and the frame is directly extracted. Step S424: After the video frame is extracted, the expected DTS = the DTS of the video frame + 1; and return to step S422 to continue the operation; Step S5: Encapsulate and output the extracted encoded frame.
2. The method for implementing 8K 120fps real-time encoding optimization according to claim 1, characterized in that, In step S2, the encoder receives the frame address of the input frame through GOP nodes in the GOP memory buffer pool, and puts the frame address into the frame list of the GOP node until the length of the frame list reaches a preset GOP length, further including... Step S201: Extract a frame address from the input memory pool, store the video frame to be encoded in the memory location associated with the frame address, and encode it as the input frame through the encoder; Step S202: Extract an idle GOP node from the GOP buffer pool in the encoder, and directly put the frame address of the input frame into the frame list of the GOP node until the predetermined length of the GOP node is reached. The GOP node enters the GOP buffer for encoding processing, and an idle GOP node is extracted from the GOP buffer pool for use by the next input frame.
3. The method for implementing 8K 120fps real-time encoding optimization according to claim 2, characterized in that, In step S3, the GOP buffer creates X parallel instance sub-threads based on the configured number of NUMA nodes X, and each instance sub-thread is bound to a NUMA node. The input frame corresponding to the frame address in the GOP node is encoded to form an encoded frame, which is then written to the output buffer. Further steps include... Step S301: When the GOP node enters the GOP buffer for encoding processing, the GOP buffer wakes up the instance sub-thread to perform parallel encoding on the GOP node; Step S302: Each instance sub-thread extracts one GOP node from the GOP buffer and begins encoding; Step S303: After the instance sub-thread finishes encoding an input frame, the frame address corresponding to the input frame is put back into the input memory pool and marked as idle; Step S304: After the instance sub-thread completes encoding the input frames corresponding to all the frame addresses in the entire GOP node, the encoded frames are written to the output buffer, and the GOP node is returned to the GOP buffer pool as an idle GOP node for continued use.
4. A system for enabling 8K 120fps real-time encoding optimization, performing the method for enabling 8K 120fps real-time encoding optimization according to any one of claims 1 to 3, characterized in that, include, The frame receiving and management module receives the video frame to be encoded, extracts a frame address from the input memory pool, stores the video frame in the memory location associated with the frame address, and encodes it as the input frame through the encoding processing module. The encoding processing module receives the frame address of the input frame through the GOP memory buffer pool and stores it in the frame list of the GOP node. When the length of the frame list reaches the preset GOP length, the GOP buffer wakes up multiple instance sub-threads to perform parallel encoding to form an encoded frame, stores the encoded frame in the output buffer, and corrects and sorts its DTS through the sub-threads. Finally, the main thread retrieves the encoded frame from the output buffer. The encapsulation output module encapsulates the extracted encoded frames according to a preset format and structure before outputting the video stream.
5. The system for implementing real-time encoding optimization at 8K 120fps according to claim 4, characterized in that, The frame receiving management module includes a video frame receiving unit and a frame address allocation unit; The video frame receiving unit is used to receive the video frame to be encoded; The frame address allocation unit is used to extract a frame address from the input memory pool, store the video frame to be encoded in the memory location associated with the frame address, and encode it as the input frame through the encoder.
6. The system for enabling 8K 120fps real-time encoding optimization according to claim 5, wherein, The encoding processing module includes an input frame buffer allocation unit, an instance sub-thread startup unit, an instance sub-thread encoding unit, and a frame output sorting adjustment unit. The input frame buffer allocation unit extracts an idle GOP node from the GOP buffer pool and directly puts the frame address of the input frame into the frame list of the GOP node. When the frame list length of the GOP node reaches the predetermined length of the GOP node, the instance sub-thread startup unit initiates the GOP node into the GOP buffer. The GOP buffer then wakes up the instance sub-thread to perform parallel encoding on the GOP node and retrieves idle GOP nodes from the GOP buffer pool for use in the next input frame. Each instance sub-thread encoding unit extracts a GOP node from the GOP buffer and begins encoding; When the instance sub-thread finishes encoding an input frame, it puts the frame address corresponding to the input frame back into the input memory pool and marks it as idle; After the instance sub-thread completes encoding the input frames corresponding to all frame addresses in the entire GOP node, it writes the encoded frames into the output buffer and puts the GOP node back into the GOP buffer pool as an idle GOP node for continued use. The frame output sorting adjustment unit, for the encoded frames in the output buffer, corrects and sorts the DTS values of the encoded frames through a sub-thread, while the main thread retrieves the encoded frames from the output buffer.
7. An apparatus for enabling 8K 120fps real-time encoding optimization, comprising: include: A memory that stores computer-readable instructions; A processor, when the computer-readable instructions are executed by the processor, implements the method for implementing 8K 120fps real-time encoding optimization as described in any one of claims 1 to 3.
8. A readable storage medium for implementing 8K 120fps real-time encoding optimization, characterized in that, include: The readable storage medium stores a processing program, which, when executed by a processor, implements the method for implementing 8K 120fps real-time encoding optimization as described in any one of claims 1 to 3.
Citation Information
Patent Citations
Method and apparatus for converting RTSP / RTP audio and video flows into TS flows and outputting TS flows through ASI
CN105491401A
Video coding optimization method and system
CN114374848A
