Method for saving memory occupation of video coding code stream

Abstract

The application provides a method for saving memory occupation of a video coding code stream, comprising the following steps: S1, applying for memory and configuring an encoder; S2, starting the encoder and waiting for the memory to be filled; S3, interrupting processing and reusing the memory; and S4, completing the encoding of one frame of image. The method supports one frame of image to be divided into multiple times to be completed by improving the video encoder. Thus, the data output each time can be reduced, the memory occupation is reduced, and an optimization method for reducing the memory occupation of the video coding is realized.

Description

Technical Field

[0001] This invention belongs to the field of embedded system optimization technology, and specifically relates to a method for saving memory usage in video encoding bitstreams. Background Technology

[0002] In modern electronic devices, especially embedded systems, memory is typically configured to be relatively small in order to reduce costs. Therefore, when developing products for such systems with limited memory, it is necessary to save memory resources in various aspects.

[0003] In existing technologies, video encoders inside chips, such as H.265, H.264, and JPEG, encode one frame of image and output one frame of bitstream data, generally without interruption. During this process, the software needs to pre-allocate a segment of physical memory for the encoder to store the encoded bitstream data.

[0004] However, the shortcomings of existing technology are:

[0005] As video resolution increases, the bitrate of the video encoding output also increases, consequently requiring more memory. This can lead to memory shortages during product development.

[0006] Furthermore, the terminology used in this art includes:

[0007] H264 / H265 / JPEG: Three commonly used video encoding formats.

[0008] DMA: Direct Memory Access refers to a data transfer method in which I / O devices and memory directly exchange data without the intervention of a processor (or CPU) program. Summary of the Invention

[0009] To address the aforementioned issues, the purpose of this application is to improve the video encoder by enabling the processing of a single frame in multiple steps. This reduces the amount of data output each time, thereby decreasing memory usage and achieving an optimized method for reducing video encoding memory consumption.

[0010] Specifically, the present invention provides a method for saving memory usage in video encoding bitstreams, the method comprising:

[0011] S1: Allocate memory and configure the encoder; including:

[0012] S1.1, Call the operating system's memory allocation interface to request a memory block of a specified size. This memory block will be used to store the data output by the encoder via DMA;

[0013] S1.2 After acquiring memory, configure the memory address and size information to the encoder; including: writing the memory address into the encoder's DMA output address register so that the encoder can confirm the output address of the data;

[0014] At the same time, the memory size is written to the DMA memory size register of another encoder so that the encoder can confirm when the memory is full and thus stop outputting data;

[0015] S2: Start the encoder and wait for memory to fill; including:

[0016] After configuring the encoder, start the encoder to begin working; the encoder will encode the input image data according to the H264 / H265 protocol, and output the data to the memory allocated in step S1.1 via DMA. At this time, the first segment of segmented encoding is being performed.

[0017] During the encoder's data output process, the encoder simultaneously outputs data and counts the amount of data output. This logic is implemented in the encoder's hardware. When the amount of output data reaches the size configured in the register, the encoder considers the memory to be full and stops outputting data. At this time, the encoder generates a system software interrupt to notify the encoding software that the encoder hardware has completed one segment of the segmented encoding. The encoding software here is the application processing software that works with the encoder hardware and is responsible for receiving and processing the encoded data.

[0018] S3: Interrupt handling and memory reuse; including:

[0019] When the encoder generates an interrupt signal, it notifies the encoding software that a segment of encoding is complete and outputs data. The encoder software copies the completed data into the encoding frame buffer in the application layer in sequence. The encoding frame buffer is used to concatenate the data of a frame of image after multiple segment encodings into a single frame of data.

[0020] S4: Complete the encoding of one frame of image; including:

[0021] Repeat steps S2 and S3 above to encode the data of a frame of image into segments and output them to DMA memory; and copy the DMA segmented data to the frame buffer of the application layer in sequence; when all the data of a frame of image is encoded, the complete encoded bitstream data of a frame of image is obtained in the frame buffer of the application layer.

[0022] In step S1.1, the memory allocation interface of the operating system is called, which is the get_free_pages function in the Linux kernel; the size of the memory block to be allocated is 50KB, or 51200 bytes.

[0023] In step S1.2, after allocating memory, the address and size information of the memory are configured to the encoder by writing to the relevant registers of the encoder.

[0024] Step S3 further includes: performing the following operations in the encoding software:

[0025] S3.1, Copy the data in the DMA memory to the frame buffer allocated during the initialization of the encoding software for subsequent processing or storage;

[0026] S3.2 After the copy is completed, the memory previously allocated to the encoder is invalid or useless and can be reused by the encoder. Assume that the encoder memory is 50KB. At this time, reconfigure the encoder registers, start the encoder to continue encoding, and output the new encoded data to the same DMA memory.

[0027] In step S3, before reconfiguring and starting the encoder, it is necessary to ensure that the data in the DMA memory has been completely copied away; otherwise, the new encoded data will overwrite or conflict with the old data.

[0028] In step S4, the method of merging multiple data segments includes:

[0029] During the initialization of the encoding software, a frame buffer is requested using the malloc interface, and data is written into this buffer sequentially each time data is copied. Once all the data of a frame image has been copied, the complete encoded bitstream data can be read from the frame buffer for subsequent processing or storage.

[0030] Following the improved process described above, the DMA memory usage for encoding will be reduced compared to the original process. The improved process only requires the memory size of a segment of encoded data.

[0031] The size of the encoded data segment is 50KB.

[0032] Therefore, the advantage of this application is:

[0033] This method can save encoder memory resources. Attached Figure Description

[0034] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0035] Figure 1 This is a schematic diagram of the coding process in existing technology.

[0036] Figure 2 This is a schematic diagram of the coding process of the method in this application.

[0037] Figure 3 This is a flowchart of the method described in this application. Detailed Implementation

[0038] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0039] like Figure 1 The diagram shows the encoding process before the improvement. After the encoder finishes encoding a frame of image, it outputs a complete frame bitstream data to the driver's DMA memory, and then copies this frame bitstream data to the application layer memory.

[0040] The improved process is as follows: Figure 2 As shown, after starting frame encoding, the encoder divides the image frame into multiple parts, encoding a small segment each time. The detailed improved process is as follows: Figure 3 As shown, it includes: Step S1: Allocate memory and configure the encoder; further includes:

[0041] S1.1 First, the operating system's memory allocation interface (such as the get_free_pages function in the Linux kernel) is called to allocate a memory block of a specified size, assuming the size here is 50KB (i.e., 51200 bytes); this memory block will be used to store the data output by the encoder via DMA;

[0042] S1.2 After allocating memory, the address and size information of the memory need to be configured to the encoder. This is usually achieved by writing to the relevant registers of the encoder; specifically, the memory address is written to the encoder's DMA output address register so that the encoder can confirm the output address of the data; at the same time, the memory size is written to another encoder's DMA memory size register so that the encoder can confirm when the memory is full and thus stop outputting data.

[0043] Step S2: Start the encoder and wait for the memory to fill up; including:

[0044] After configuring the encoder, you can start the encoder to work; the encoder will encode the input image data according to the H264 / H265 protocol and output the data to the memory allocated in step S1.1 via DMA. At this time, the first segment of segmented encoding is being performed.

[0045] During the process of the encoder outputting data, the encoder will count the amount of data being output while outputting the data. This part of the logic is implemented in the hardware of the encoder. When the amount of output data reaches the size configured in the previous register (i.e., 50KB), the encoder will consider the memory to be full and stop outputting data; at this time, the encoder will generate a system software interrupt to notify the encoding software that a segment of the segmented encoding by the encoder hardware has been completed; the encoding software here is the application processing software supporting the encoder hardware and is responsible for receiving and processing the encoded data; Step S3: Interrupt handling and memory reuse; including:

[0046] When the encoder generates an interrupt signal, it notifies the encoding software that a segment of encoding is completed and outputs the data; the encoding software of the encoder copies the completed data to the encoding frame buffer in the application layer in sequence; the encoding frame buffer is used to splice the data after multiple segment encodings of a frame of image into a frame of data; in the encoding software, the following operations are performed:

[0047] S3.1, Copy the data in the DMA memory to the frame buffer applied for during the initialization of the encoding software for subsequent processing or storage;

[0048] S3.2, After the copying is completed, the previously allocated 50KB memory for the encoder becomes invalid or useless and can be reused by the encoder; at this time, reconfigure the register of the encoder, start the encoder to continue encoding, and output the new encoded data to the same DMA memory.

[0049] It should be noted that before reconfiguring and starting the encoder, it is necessary to ensure that the data in the DMA memory has been completely copied away, otherwise, the new encoded data may overwrite or conflict with the old data.

[0050] Step S4: Complete the encoding of a frame of image; including

[0051] By repeating the above Step S2 and Step S3, the data of a frame of image can be segmented and encoded and output to the DMA memory. When all the data of a frame of image has been encoded, the multiple segments of data previously copied to the application layer can be merged together in sequence to obtain the complete encoded bitstream data of a frame of image.

[0052] The method of merging multiple segments of data is very simple, including: during the initialization of the encoding software, use the malloc interface to apply for and establish a frame buffer area, and write the data to this buffer area in sequence each time data is copied; when all the data of a frame of image has been copied, the complete encoded bitstream data can be read from the frame buffer area for subsequent processing or storage.

[0053] Following the improved workflow described above, the DMA memory usage for encoding will be significantly less than that of the original workflow. The improved workflow requires the memory size of a block of encoded data, for example, 50KB. The previous workflow required encoding one frame at a time, necessitating approximately 1MB or more of memory.

[0054] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for saving memory usage in video encoding bitstreams, characterized in that, The method includes: S1: Allocate memory and configure the encoder; including: S1.1, Call the operating system's memory allocation interface to request a memory block of a specified size. This memory block will be used to store the data output by the encoder via DMA; S1.2 After acquiring memory, configure the memory address and size information to the encoder; including: writing the memory address into the encoder's DMA output address register so that the encoder can confirm the output address of the data; At the same time, the memory size is written to the DMA memory size register of another encoder so that the encoder can confirm when the memory is full and thus stop outputting data; S2: Start the encoder and wait for memory to fill; including: After configuring the encoder, start the encoder to begin working; the encoder will encode the input image data according to the H264 / H265 protocol, and output the data to the memory allocated in step S1.1 via DMA. At this time, the first segment of segmented encoding is being performed. During the encoder's data output process, the encoder simultaneously outputs data and counts the amount of data output. This logic is implemented in the encoder's hardware. When the amount of output data reaches the size configured in the register, the encoder considers the memory to be full and stops outputting data. At this time, the encoder generates a system software interrupt to notify the encoding software that the encoder hardware has completed one segment of the segmented encoding. The encoding software here is the application processing software that works with the encoder hardware and is responsible for receiving and processing the encoded data. S3: Interrupt handling and memory reuse; including: When the encoder generates an interrupt signal, it notifies the encoding software that a segment of encoding is complete and outputs data. The encoder software copies the completed data into the encoding frame buffer in the application layer in sequence. The encoding frame buffer is used to concatenate the data of a frame of image after multiple segment encodings into a single frame of data. S4: Complete the encoding of one frame of image; including: Repeat steps S2 and S3 above to encode the data of a frame of image into segments and output them to DMA memory; and copy the DMA segmented data to the frame buffer of the application layer in sequence; when all the data of a frame of image is encoded, the complete encoded bitstream data of a frame of image is obtained in the frame buffer of the application layer.

2. The method for saving video encoding bitstream memory usage according to claim 1, characterized in that, In step S1.1, the memory allocation interface of the operating system is called, which is the get_free_pages function in the Linux kernel; the size of the memory block to be allocated is 50KB, or 51200 bytes.

3. The method for saving video encoding bitstream memory usage according to claim 1, characterized in that, In step S1.2, after allocating memory, the address and size information of the memory are configured to the encoder by writing to the relevant registers of the encoder.

4. The method for saving video encoding bitstream memory usage according to claim 1, characterized in that, Step S3 further includes: in the encoding software, performing the following operations: S3.1, copying the data in the DMA memory to the frame buffer allocated during the initialization of the encoding software for subsequent processing or storage; S3.2 After the copy is completed, the memory previously allocated to the encoder is invalid or useless and can be reused by the encoder. Assume that the encoder memory is 50KB. At this time, reconfigure the encoder registers, start the encoder to continue encoding, and output the new encoded data to the same DMA memory.

5. The method for saving video encoding bitstream memory usage according to claim 4, characterized in that, In step S3, before reconfiguring and starting the encoder, it is necessary to ensure that the data in the DMA memory has been completely copied away; otherwise, the new encoded data will overwrite or conflict with the old data.

6. The method for saving video encoding bitstream memory usage according to claim 5, characterized in that, In step S4, the method of merging multiple data segments includes: During the initialization of the encoding software, a frame buffer is requested using the malloc interface, and data is written into this buffer sequentially each time data is copied. Once all the data of a frame image has been copied, the complete encoded bitstream data can be read from the frame buffer for subsequent processing or storage.

7. The method for saving video encoding bitstream memory usage according to claim 1, characterized in that, Following the improved process described above, the DMA memory usage for encoding will be reduced compared to the original process. The improved process only requires the memory size of a segment of encoded data.

8. The method for saving video encoding bitstream memory usage according to claim 7, characterized in that, The size of the encoded data segment is 50KB.

Images