Coding method, decoding method and device based on display stream compression, computer equipment, readable storage medium and program product

Abstract

The invention relates to a display stream compression-based encoding method, a display stream compression-based decoding method, a display stream compression-based decoding device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a to-be-coded video frame image and image parameter information; segmenting the video frame image to be coded into a plurality of slices based on the image parameter information; determining the column number of a slice corresponding to the to-be-coded video frame image, and determining a corresponding number of coding cores for parallel operation based on the column number; respectively distributing each column of the slices to the corresponding coding cores, wherein one column of the slices corresponds to one coding core; and coding the slices in parallel through the coding cores. By adopting the method, the coding and decoding efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to an encoding method, a decoding method, an apparatus, a computer device, a computer-readable storage medium, and a computer program product based on display stream compression. Background Art

[0002] With the development of video technology, the demand for ultra-high-resolution 4K and 8K videos is increasing. Display link bandwidth and cost issues have become one of the main bottlenecks. Display Stream Compression (DSC) is an effective technology that saves bandwidth by compressing display images.

[0003] Traditional display stream compression codecs use a single encoding core mode, which has low encoding and decoding efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a coding method, decoding method, device, computer equipment, computer-readable storage medium and computer program product based on display stream compression that can improve coding and decoding efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides an encoding method based on display stream compression, the method comprising:

[0006] Obtain the video frame image to be encoded and image parameter information;

[0007] Based on the image parameter information, the video frame image to be encoded is divided into a plurality of slices; the number of columns of the slices corresponding to the video frame image to be encoded is determined, and based on the number of columns, a corresponding number of encoding cores for parallel operation is determined;

[0008] Allocating each column of the slices to the corresponding encoding core respectively, and one column of the slices corresponds to one encoding core;

[0009] The slices are encoded in parallel by the encoding cores.

[0010] In one embodiment, the step of respectively allocating each column of the slices to the corresponding encoding core comprises:

[0011] Determine a buffer area corresponding to each of the encoding cores;

[0012] The rows in each column of the slice are sequentially allocated to the buffer corresponding to the encoding core.

[0013] In one embodiment, after encoding the slices in parallel by the encoding cores, the method further comprises:

[0014] Obtaining encoding results of rows in each of the slices output by each of the encoding cores;

[0015] Based on the image parameter information, the encoding results of the rows in each of the slices are spliced ​​to obtain a video frame encoding result.

[0016] In one of the embodiments, the encoding rate control mode of the video frame image to be encoded is a fixed rate mode; and the step of splicing the encoding results of the rows in each of the slices based on the image parameter information to obtain the video frame encoding result includes:

[0017] Determine the size of the first row of the slice based on the slice width and the number of bits occupied by each pixel in the image parameter information;

[0018] Obtaining a first row offset based on the first row size of the slice and the number of columns;

[0019] Determine a first target offset based on a row index and the first row offset, and determine an encoding result of a current row of each slice based on the first target offset and each slice index;

[0020] The encoding result of the current row of the video frame is obtained by splicing the encoding results of the current row of each slice;

[0021] The video frame encoding result is obtained based on the encoding results of each current row of the video frame.

[0022] In one of the embodiments, the encoding rate control mode of the video frame image to be encoded is a dynamic rate mode; and the step of splicing the encoding results of the rows in each of the slices based on the image parameter information to obtain the video frame encoding result includes:

[0023] Get the row size of the default slice;

[0024] Obtain a second row offset based on the row size of the default slice and the number of columns;

[0025] Determine a second target offset based on the row index and the second row offset, and determine an encoding result of a current row of each slice based on the second target offset and each slice index;

[0026] Preprocessing the encoding results of the current row of each of the slices, and splicing the preprocessed encoding results of the current row of the slices to obtain the encoding result of the current row of the video frame;

[0027] The video frame encoding result is obtained based on the encoding results of each current row of the video frame.

[0028] In a second aspect, the present application further provides a decoding method based on display stream compression, the method comprising:

[0029] Obtain the video frame code stream and image parameter information to be decoded;

[0030] Obtaining slice information corresponding to the to-be-decoded video frame code stream based on the image parameter information;

[0031] Determine the number of columns of the slice of the to-be-decoded video frame code stream based on the slice information, and determine a corresponding number of decoding cores for parallel operation based on the number of columns;

[0032] Allocating the bitstreams corresponding to each column of the slices to the corresponding decoding cores, and the bitstreams corresponding to one column of the slices correspond to one decoding core;

[0033] The code stream corresponding to the slice is decoded in parallel by each of the decoding cores.

[0034] In one embodiment, the step of respectively allocating the bitstream corresponding to each column of the slices to the corresponding decoding core includes:

[0035] Determine the buffer area corresponding to each of the decoding cores;

[0036] The code streams corresponding to the rows in each column of the slice are sequentially allocated to the buffer corresponding to the decoding core.

[0037] In one of the embodiments, after decoding the bitstream corresponding to the slice in parallel by each of the decoding cores, the method includes:

[0038] Obtaining decoding results of code streams corresponding to rows in each of the slices output by each of the decoding cores;

[0039] Based on the image parameter information, decoding results of the code streams corresponding to the rows in each of the slices are spliced ​​to obtain a video frame decoding result.

[0040] In one of the embodiments, the decoding rate control mode of the video frame code stream to be decoded is a fixed rate mode; and the decoding results of the code streams corresponding to the rows in each of the slices are spliced ​​based on the image parameter information to obtain the video frame decoding result, including:

[0041] Determine a second row size of the slice based on the slice width and the number of bits occupied by each pixel in the image parameter information;

[0042] Obtain a third row offset based on the second row size of the slice and the number of columns;

[0043] Determine a third target offset based on the row index and the third row offset, and determine a decoding result of a code stream corresponding to a current row of each slice based on the third target offset and each slice index;

[0044] The decoding result of the current row of the video frame is obtained by splicing the decoding results of the code streams corresponding to the current rows of the slices;

[0045] The video frame decoding result is obtained based on the decoding results of each current row of the video frame.

[0046] In a third aspect, the present application further provides an encoding device based on display stream compression, the device comprising:

[0047] A first acquisition module, used to acquire image parameter information of a video frame image to be encoded;

[0048] A slicing module, used for dividing the to-be-encoded video frame image into a plurality of slices based on the image parameter information;

[0049] A coding core determination module, used to determine the number of columns of the slice corresponding to the video frame image to be encoded, and determine a corresponding number of coding cores for parallel operation based on the number of columns;

[0050] A first allocation module is used to allocate the bitstream corresponding to each column of the slices to the corresponding encoding core, and the bitstream corresponding to one column of the slices corresponds to one encoding core;

[0051] The parallel encoding module is used to encode the code stream corresponding to the slice in parallel through each of the encoding cores.

[0052] In a fourth aspect, the present application further provides a decoding device based on display stream compression, the device comprising:

[0053] The second acquisition module is used to obtain the video frame code stream and image parameter information to be decoded;

[0054] A slice information acquisition module, used to obtain slice information corresponding to the video frame code stream to be decoded based on the image parameter information;

[0055] A decoding core determination module, used to determine the number of columns of the slice of the video frame code stream to be decoded based on the slice information, and determine a corresponding number of decoding cores for parallel operation based on the number of columns;

[0056] A second allocation module is used to allocate the bitstream corresponding to each column of the slices to the corresponding decoding core, and the bitstream corresponding to one column of the slices corresponds to one decoding core;

[0057] The parallel decoding module is used to decode the code stream corresponding to the slice in parallel through each of the decoding cores.

[0058] In a fifth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in any one of the above-mentioned embodiments when executing the computer program.

[0059] In a sixth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method in any one of the above-mentioned embodiments.

[0060] In a seventh aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the method in any one of the above-mentioned embodiments when executed by a processor.

[0061] The above-mentioned encoding method, decoding method, device, computer equipment, computer-readable storage medium and computer program product based on display stream compression obtain the video frame image to be encoded and image parameter information; based on the image parameter information, divide the video frame image to be encoded into a plurality of slices; determine the number of columns of slices corresponding to the video frame image to be encoded, and determine the corresponding number of encoding cores running in parallel based on the number of columns; assign each column of slices to the corresponding encoding core respectively, and one column of slices corresponds to one encoding core; encode the slices in parallel through each encoding core, so that the encoding of the video frame through parallel encoding cores can improve the encoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0063] FIG1 is a schematic diagram of a DSC encoding system in one embodiment;

[0064] FIG2 is a schematic diagram of a DSC standard encoder pipeline in one embodiment;

[0065] FIG3 is a schematic diagram of a DSC standard decoder pipeline in one embodiment;

[0066] FIG4 is a schematic diagram of image slicing in one embodiment;

[0067] FIG5 is a schematic diagram of an encoder pipeline for multi-core parallel processing in one embodiment;

[0068] FIG6 is a schematic diagram of a decoder pipeline for multi-core parallel processing in one embodiment;

[0069] FIG7 is a schematic flow chart of an encoding method based on display stream compression in one embodiment;

[0070] FIG8 is a schematic diagram of an encoding core in one embodiment;

[0071] FIG9 is a schematic diagram of a frame splitting module Frame Splitter in one embodiment;

[0072] FIG10 is a schematic diagram of the corresponding relationship between a slice chunk and a coding core in one embodiment;

[0073] FIG11 is a schematic diagram of a first splicing module in one embodiment;

[0074] FIG12 is a schematic diagram of Bitstream data and a Bitstream splicing method in one embodiment;

[0075] FIG13 is a schematic flow chart of a decoding method based on display stream compression in one embodiment;

[0076] FIG14 is a schematic diagram of a decoding core in one embodiment;

[0077] FIG15 is a schematic diagram of a code stream splitting module in one embodiment;

[0078] FIG16 is a schematic diagram of the corresponding relationship between a slice chunk and a decoding core in one embodiment;

[0079] FIG17 is a schematic diagram of a second splicing module in one embodiment;

[0080] FIG18 is a schematic diagram of the correspondence between slice data output by a decoding core and a video frame in one embodiment;

[0081] FIG19 is a structural block diagram of an encoding device based on display stream compression in one embodiment;

[0082] FIG20 is a structural block diagram of a decoding device based on display stream compression in one embodiment;

[0083] FIG. 21 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0085] Display Stream Compression (DSC) uses a whole picture as a unit. Each frame of the image is divided into several rectangular blocks of the same size according to the PPS configuration. These blocks are called slices. Each slice can be encoded and decoded independently without referring to the data of other slices.

[0086] DSC display technology uses independent slices for encoding and decoding. The data between slices has no reference or dependency, and slices can be assigned to different cores for parallel processing. According to the row-by-row display rule of DSC, slices are assigned to multiple cores for processing by column, that is, the slice data or corresponding bit stream data of the same column are assigned to the same core for encoding and decoding, and then the code stream or frame data output by the encoding and decoding system is obtained according to the DSC slice structure or the corresponding code stream structure and addressing method.

[0087] Specifically, as shown in Figure 1, the DSC codec system includes a source device and a receiving device. The image source in the source device transmits uncompressed original image data to the encoder, and the encoder compresses pixels according to the DSC standard and set parameters to form a bit stream; the bit stream is arranged in a specified format and transmitted to the receiving device through a display link, and the decoder restores the image according to the parameters set during compression and presents it on the display device.

[0088] As shown in Figures 2 and 3, Figure 2 is the DSC standard encoder pipeline, and Figure 3 is the DSC standard decoder pipeline, where the video stream is composed of multiple consecutive frames of pictures, and the compression process is performed inside each frame of the picture, and is not related to the previous and next frame images. According to the information such as the image width pic_width, image height pic_height, slice width slice_width, and slice height slice_height in the picture parameter set (PPS) configuration, the entire picture is divided into an integer n continuous, non-overlapping, and equal-sized rectangular slices (Slices), and each slice is encoded, transmitted, and decoded in units of slices, so as to achieve the purpose of compression, transmission, and decompression of the entire picture. The process of dividing the picture into slices is shown in Figure 4 below. The slice layer specifies the slice encoding and decoding method, each slice can be encoded independently, and each row has one or more slices, and the number of bits of a row of slices in the bit stream can be processed by slice multiplexing.

[0089] The video frame structure is shown in Figure 4. Considering that there is no dependency between slices and they can be encoded independently, and the DSC display is displayed line by line, the slices in the same column are assigned to one Core for processing.

[0090] Since the DSC codec needs to refer to the pixels of the previous row during the prediction process, the pixels of the previous row are stored in LineBuffer in the hardware implementation. Considering the storage and update issues of LineBuffer, the number of Cores should be equal to the number of slices per row. Therefore, for each row, slice_num = 4, 4 Cores should be used; slice_num = 2, 2 Cores should be used.

[0091] Specifically, a video frame image to be encoded and image parameter information are obtained; based on the image parameter information, the video frame image to be encoded is divided into a plurality of slices; the number of columns of slices corresponding to the video frame image to be encoded is determined, and based on the number of columns, a corresponding number of encoding cores running in parallel are determined; each column of slices is respectively assigned to a corresponding encoding core, and one column of slices corresponds to one encoding core; the slices are encoded in parallel by each encoding core, so that encoding of video frames by parallel encoding cores can improve encoding efficiency.

[0092] The encoding method based on display stream compression provided in the embodiment of the present application can be applied to the encoder pipeline of multi-core parallel processing as shown in Figure 5. The decoding method based on display stream compression can be applied to the decoder pipeline of multi-core parallel processing as shown in Figure 6.

[0093] The encoder pipeline includes multiple encoding cores, the decoder pipeline may include multiple decoding cores, the multiple encoding cores run in parallel, and the multiple decoding cores run in parallel.

[0094] In an exemplary embodiment, as shown in FIG7 , a coding method based on display stream compression is provided, and the method is applied to the encoder pipeline of multi-core parallel processing in FIG5 as an example for description, including the following steps 702 to 710. Among them:

[0095] S702: Obtain the video frame image to be encoded and image parameter information.

[0096] The image parameter information includes information such as image width pic_width, image height pic_height, slice width slice_width, slice height slice_height, etc. The PPS parser in FIG5 parses the image parameter information to obtain specific image parameters.

[0097] The video frame image to be encoded is a frame of video that needs to be encoded. Specifically, the RGB toYcoCg-R conversion module can convert the RGB video frame into the YcoCg-R space based on the image parameter information to obtain the video frame image to be encoded, and store the video frame image to be encoded in a cache, such as the Buffer in Figure 5.

[0098] S704: Divide the to-be-encoded video frame image into a plurality of slices based on the image parameter information.

[0099] The video frame image to be encoded is divided into a number of slices, which can be specifically shown in FIG. 4 .

[0100] S706: Determine the number of columns of the slice corresponding to the video frame image to be encoded, and determine a corresponding number of encoding cores for parallel operation based on the number of columns.

[0101] Among them, considering that there is no dependency between slices and they can be encoded independently, and the DSC is displayed line by line, the slices in the same column are assigned to one encoding core for processing.

[0102] Since DSC codec needs to refer to the pixels of the previous row during prediction, the pixels of the previous row are stored in LineBuffer in hardware implementation. Considering the storage and update of LineBuffer, the number of encoding cores should be equal to the number of slices in each row, that is, the number of columns of slices corresponding to the video frame image to be encoded, so the number of encoding cores can be obtained based on the number of columns of slices. For each row, slice_num = 4, 4 encoding cores should be used; slice_num = 2, 2 encoding cores should be used.

[0103] S708: Allocate each column of slices to a corresponding encoding core respectively, and one column of slices corresponds to one encoding core.

[0104] Here, the number of slices in each row is 4, which means that the number of columns of slices corresponding to the video frame image to be encoded is 4. After a video frame image to be encoded is divided into slices, the columns of the slices are successively allocated to encoding cores Core0, Core1, Core2, and Core3, that is, each column of slices is sent to Buffer0, Buffer1, Buffer2, and Buffer3 as shown in FIG9, and processed by corresponding encoding cores Core0, Core1, Core2, and Core3. The four encoding cores are exactly the same.

[0105] S710: Encode the slices in parallel using each encoding core.

[0106] The encoding cores are run in parallel, so the corresponding slices can be encoded in parallel. As shown in FIG8 , FIG8 is a schematic diagram of an encoding core in an embodiment, wherein the Line Buffer is used to store pixels of the previous row.

[0107] The above-mentioned encoding method based on display stream compression obtains the video frame image to be encoded and image parameter information; based on the image parameter information, the video frame image to be encoded is divided into a plurality of slices; the number of columns of the slices corresponding to the video frame image to be encoded is determined, and the corresponding number of encoding cores running in parallel is determined based on the number of columns; each column of slices is respectively assigned to the corresponding encoding core, and one column of slices corresponds to one encoding core; the slices are encoded in parallel by each encoding core, so that the encoding of the video frame by parallel encoding cores can improve the encoding efficiency.

[0108] In one of the optional embodiments, each column of slices is respectively assigned to a corresponding encoding core, including: determining a buffer corresponding to each encoding core; and sequentially assigning rows in each column of slices to the buffer of the corresponding encoding core.

[0109] Specifically, in combination with Figure 9, Figure 9 is a schematic diagram of a frame division module Frame Splitter in an embodiment. In this embodiment, the frame division module Frame Splitter is used to divide a frame of video frame image to be encoded according to the slices shown in Figure 4 to obtain slices, and assign the slices in the same column to a coding core for encoding.

[0110] Slice-column0, Slice-column1, Slice-column2, and Slice-column3 in Figure 9 are the chunks corresponding to slice column0, column1, column2, and column3 in Figure 4. A chunk refers to a portion of a bitstream that includes a group of data bytes. For each slice, the number of chunks in a slice is the same as the number of rows. As shown in Figure 10, the data of each slice column is sent to the same encoding core for processing, chunk by chunk and row by row.

[0111] Buffer0, Buffer1, Buffer2, and Buffer3 are corresponding buffers. The buffer size is chunk_size* n. It is recommended that n be 3. In actual applications, the specific value can be obtained through profiling. The corresponding relationship between slicechunk and Encoding Core is shown in Figure 10.

[0112] In one of the optional embodiments, after encoding the slices in parallel by each encoding core, it includes: obtaining the encoding results of the rows in each slice output by each encoding core; based on the image parameter information, splicing the encoding results of the rows in each slice to obtain the video frame encoding result.

[0113] After each encoding core obtains the encoding result of the row in the corresponding slice, the encoding results of the code streams output by multiple encoding cores can be spliced ​​into the encoding result of a complete frame of code stream through the first splicing module Bitstream Combiner, as shown in Figure 11, Bitstream-0, Bitstream-1, Bitstream-2, Bitstream-3 are the code stream chunks corresponding to slice column0, column1, column2, column3 in Figure 4. Buffer0, Buffer1, Buffer2, Buffer3 are the corresponding buffers, and the buffer size is chunk_size * n. It is recommended that n be 3. In practical applications, the specific value can also be obtained by profiling. The Bitstream data output after each Core is processed and the Bitstream splicing method is shown in Figure 12.

[0114] In one of the optional embodiments, the encoding rate control mode of the video frame image to be encoded is a fixed rate mode; based on the image parameter information, the encoding results of the rows in each slice are spliced ​​to obtain the video frame encoding result, including: determining the first row size of the slice based on the slice width in the image parameter information and the number of bits occupied by each pixel; obtaining the first row offset based on the first row size of the slice and the number of columns; determining the first target offset based on the row index and the first row offset, and determining the encoding result of the current row of each slice based on the first target offset and each slice index; splicing the encoding results of the current row of each slice to obtain the encoding result of the current row of the video frame; obtaining the video frame encoding result based on the encoding results of each current row of the video frame.

[0115] In one of the optional embodiments, the encoding rate control mode of the video frame image to be encoded is a dynamic rate mode; based on the image parameter information, the encoding results of the rows in each slice are spliced ​​to obtain the video frame encoding result, including: obtaining the row size of the default slice; obtaining the second row offset based on the row size and the number of columns of the default slice; determining the second target offset based on the row index and the second row offset, and determining the encoding result of the current row of each slice based on the second target offset and each slice index; preprocessing the encoding result of the current row of each slice, and splicing the encoding results of the current row of the preprocessed slice to obtain the encoding result of the current row of the video frame; obtaining the video frame encoding result based on the encoding results of each current row of the video frame.

[0116] Among them, in the DSC standard, there are two bit rate control modes for encoding and decoding, namely CBR (Constant Bit Rate) mode (fixed bit rate mode) and VBR (Variable Bit Rate) mode (dynamic bit rate mode).

[0117] In CBR mode, the size of a chunk is a fixed value, chunksize=ceil(bpp * slice_width / 8) bytes, where bpp is the number of bits per pixel and slice_width is the slice width. The addressing method for a specified chunk in the bitstream of Figure 12 is: Pos = Ptr + chunkoffset, where Chunkoffset = chunksize*slice_num_per_line. Slice_num_per_line is the number of slices in each line, that is, the number of slice columns.

[0118] Therefore, in the CBR mode, the first splicing module Bitstream Combiner can quickly calculate the location where the chunk bitstream is stored according to the size of chunk_size, thereby realizing the splicing of the bitstream.

[0119] In VBR mode, the chunk size is an indefinite value, which is the actual number of encodings. The calculation method of chunk_size is Chunk_size=ceil((nominalChunkSize - clampedBits) / 8) bytes, where nominalChunkSize=bpp*slice_width(or bpp* (slice_width>>1) in Native 4:2:2 or4:2:0 mode), and clampedBits needs to be determined in the Buffer Level Tracker stage.

[0120] Therefore, in VBR mode, the Bitstream Combiner module cannot quickly calculate the location of chunk bitstream storage based on chunk_size to achieve bitstream splicing as in CBR mode. However, the maximum value of chunk_size does not exceed nominalChunkSize / 8 bytes, so a chunk buffer of nominalChunkSize / 8 bytes can be used to store the bitstream of each chunk, and finally remove the empty space without bitstream value to achieve bitstream splicing.

[0121] In an exemplary embodiment, as shown in FIG13 , a decoding method based on display stream compression is provided, and the method is applied to the decoder pipeline of multi-core parallel processing in FIG6 as an example for description, including the following steps 1302 to 1310. Among them:

[0122] S1302: Obtain the video frame code stream and image parameter information to be decoded.

[0123] The image parameter information includes information such as image width pic_width, image height pic_height, slice width slice_width, slice height slice_height, etc. The PPS parser in FIG6 parses the image parameter information to obtain specific image parameters.

[0124] The code stream of the video frame to be decoded needs to be decoded, wherein the code stream is stored in the code stream buffer module RateBuffer, and then the demultiplexer Demultiplexing decomposes the code stream of the Y, Co, and Cg components together to obtain the tributary Substream.

[0125] S1304: Obtain slice information corresponding to the video frame code stream to be decoded based on the image parameter information.

[0126] The slice information includes slice width and slice height, the number of columns and rows of the slice, etc. The number of columns of the slice can be obtained based on the image width pic_width and the slice width slice_width, and the number of columns of the slice can be used to determine the number of decoding cores.

[0127] S1306: Determine the number of columns of the slice of the video frame code stream to be decoded based on the slice information, and determine a corresponding number of decoding cores for parallel operation based on the number of columns.

[0128] Considering that there is no dependency between the slices and they are encoded independently, and the DSC is displayed line by line, the slices in the same column are assigned to a decoding core for processing. The structure of the decoding core can be shown in FIG14 .

[0129] S1308: Allocate the bitstream corresponding to each column of slices to the corresponding decoding core respectively, and the bitstream corresponding to one column of slices corresponds to one decoding core.

[0130] S1310: Decode the bitstream corresponding to the slice in parallel through each decoding core.

[0131] As shown in FIG6 , the bit stream corresponding to the chunk corresponding to each column slice is sequentially allocated to decoding cores Core0, Core1, Core2, and Core4 for parallel decoding processing. After decoding, row-by-row data of each column slice is obtained, and then the complete picture data is formed according to the rules of FIG4 .

[0132] The above-mentioned decoding method based on display stream compression obtains the video frame code stream to be decoded and the image parameter information; obtains the slice information corresponding to the video frame code stream to be decoded based on the image parameter information; determines the number of columns of the slices of the video frame code stream to be decoded based on the slice information, and determines the corresponding number of decoding cores for parallel operation based on the number of columns; respectively allocates the code stream corresponding to each column of slices to the corresponding decoding core, and the code stream corresponding to one column of slices corresponds to one decoding core; decodes the slices in parallel by each decoding core; decodes the code stream corresponding to the slices in parallel by each decoding core, so that decoding of video frames by parallel decoding cores can improve decoding efficiency.

[0133] In one of the optional embodiments, the code stream corresponding to each column of slices is respectively allocated to the corresponding decoding core, including: determining the buffer corresponding to each decoding core; and sequentially allocating the code stream of the rows in each column of slices to the buffer of the corresponding decoding core.

[0134] As shown in Figure 15, Figure 15 is a schematic diagram of a bitstream splitting module in an embodiment. The main function of the bitstream splitting module Bitstream Splitter is to divide a frame of bitstream according to the original image slicing method, and assign the bitstream corresponding to the same column of slices to a decoding core Decoding Core for decoding.

[0135] FIG15 shows the process of splitting the bitstream into chunk data. Bitstream-0, Bitstream-1, Bitstream-2, and Bitstream-3 are the bitstream chunks corresponding to slice column0, column1, column2, and column3 in FIG4. Buffer0, Buffer1, Buffer2, and Buffer3 are the corresponding buffers. The buffer size is chunk_size *n. It is recommended that n be 3. In practical applications, the specific value can also be obtained by profiling. This process is the inverse process of the method in FIG11, and the addressing method of the bitstream chunk is the same. As shown in FIG16, slice0 chunk0, slice1 chunk0, slice2 chunk0,…slice(N-1)chunk(M-1) in FIG16 are the bitstreams corresponding to the corresponding slice chunk positions. After the bitstream is split, it is allocated to each decoding core for decoding processing according to the chunk behavior unit.

[0136] In one of the optional embodiments, after the slices are decoded in parallel by each decoding core, it includes: obtaining the decoding results of the code stream corresponding to the rows in each slice output by each decoding core; based on the image parameter information, splicing the decoding results of the code stream corresponding to the rows in each slice to obtain the video frame decoding results.

[0137] FIG17 is a schematic diagram of a second splicing module in an embodiment. The main function of the second splicing module SliceCombiner is to splice the slice data output by multiple decoding cores into a complete frame of picture, wherein slice-column0, slice-column1, slice-column2, and slice-column3 are slice chunks corresponding to slicecolumn0, column1, column2, and column3 in FIG4. Buffer0, Buffer1, Buffer2, and Buffer3 are corresponding buffers, and the buffer size is chunk_size * n. It is recommended that n be 3. In practical applications, the specific value can also be obtained by profiling.

[0138] Each decoding core outputs a decoding result of the code stream corresponding to a chunk each time, and the decoding results output by each decoding core are spliced ​​into frames line by line in an arrangement as shown in FIG. 18 .

[0139] In one of the optional embodiments, the decoding rate control mode of the video frame code stream to be decoded is a fixed rate mode; based on the image parameter information, the decoding results of the rows in each slice are spliced ​​to obtain the video frame decoding result, including: determining the second row size of the slice based on the slice width and the number of bits occupied by each pixel in the image parameter information; obtaining the third row offset based on the second row size of the slice and the number of columns; determining the third target offset based on the row index and the third row offset, and determining the decoding result of the code stream corresponding to the current row of each slice based on the third target offset and each slice index; splicing the decoding results of the code stream corresponding to the current row of each slice to obtain the decoding result of the current row of the video frame; obtaining the video frame decoding result based on the decoding results of each current row of the video frame.

[0140] Among them, in the multi-core parallel decoder, the Bitstream Splitter module needs to split the input bitstream data in units of chunk_size according to the size of chunk_size. Regarding the value of chunk_size, the calculation method of CBR and VBR is as follows:

[0141] In CBR mode, the chunk size is a fixed value, which is Chunk_size = ceil(bpp * slice_width / 8) bytes. The addressing method for the specified chunk in the bitstream in Figure 18 is: Pos = Ptr + chunkoffset, where Chunkoffse = chunksize * slice_num_per_line. Therefore, for the bitstream encoded in CBR mode, the bitstream can be easily divided according to the chunk_size.

[0142] In VBR mode, the chunk size is an indefinite value, which is the actual number of encodings. The calculation method of Chunk_size is Chunk_size=ceil((nominalChunkSize-clampedBits) / 8)bytes, where nominalChunkSize=bit_per_pixel*slice_width(orbit_per_pixel*(slice_width>>1) in Native 4:2:2 or4:2:0 mode), and clampedBits needs to be determined in the Buffer Level Tracker stage. Therefore, for the bitstream encoded in VBR mode, the bitstream cannot be divided according to the chunk_size.

[0143] Therefore, the multi-core parallel decoder proposed in this application is only applicable to the CBR mode, and is not applicable to the VBR mode. For the VBR mode, decoding can be performed based on a single-core decoder.

[0144] The encoding method and decoding method in the present application perform encoding and decoding in parallel according to multi-core processing of slice columns, and design a multi-core parallel processing encoder and decoder system and pipeline, which improves the encoding and decoding efficiency compared with the traditional DSC encoding and decoding pipeline; a method for assembling the bitstream chunk data obtained by the multi-core encoder system into a complete Bitstream and the slice data obtained by the multi-core decoder system into a complete picture is proposed, and an implementation method for multi-core parallel encoding and decoding data output is proposed; the DSC multi-core parallel processing encoding system and pipeline are easy to implement and low in cost, can significantly improve the encoding and decoding efficiency of high-resolution video frames, enhance risk resistance, and improve user experience and user satisfaction.

[0145] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0146] Based on the same inventive concept, the embodiment of the present application also provides a coding device and a decoding device based on display stream compression for implementing the coding method and decoding method based on display stream compression mentioned above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the coding device and the decoding device based on display stream compression provided below can refer to the limitations of the coding method and the decoding method based on display stream compression above, and will not be repeated here.

[0147] In an exemplary embodiment, as shown in FIG. 19 , a coding apparatus based on display stream compression is provided, comprising: a first acquisition module 1901, a slicing module 1902, a coding core determination module 1903, a first allocation module 1904 and a parallel coding module 1905, wherein:

[0148] The first acquisition module 1901 is used to acquire the video frame image to be encoded and image parameter information;

[0149] The slicing module 1902 is used to divide the video frame image to be encoded into a plurality of slices based on the image parameter information;

[0150] The coding core determination module 1903 is used to determine the number of columns of the slice corresponding to the video frame image to be encoded, and determine the corresponding number of coding cores for parallel operation based on the number of columns;

[0151] A first allocation module 1904 is used to allocate each column of slices to a corresponding encoding core, and one column of slices corresponds to one encoding core;

[0152] The parallel encoding module 1905 is used to encode the slices in parallel through each encoding core.

[0153] In one of the optional embodiments, the first allocation module 1904 is specifically used to determine the buffer corresponding to each encoding core; and allocate the rows in each column slice to the buffer corresponding to the encoding core in sequence.

[0154] In one of the optional embodiments, the above device further includes:

[0155] The first splicing module is used to obtain the encoding results of the rows in each slice output by each encoding core; based on the image parameter information, the encoding results of the rows in each slice are spliced ​​to obtain the video frame encoding result.

[0156] In one of the optional embodiments, the encoding rate control mode of the video frame image to be encoded is a fixed rate mode; the above-mentioned first splicing module is also used to determine the first row size of the slice based on the slice width in the image parameter information and the number of bits occupied by each pixel; obtain the first row offset based on the first row size of the slice and the number of columns; determine the first target offset based on the row index and the first row offset, and determine the encoding result of the current row of each slice based on the first target offset and each slice index; obtain the encoding result of the current row of the video frame by splicing the encoding results of the current row of each slice; obtain the video frame encoding result based on the encoding results of each current row of the video frame.

[0157] In one of the optional embodiments, the encoding rate control mode of the video frame image to be encoded is a dynamic rate mode; the above-mentioned first splicing module is also used to obtain the row size of the default slice; obtain the second row offset based on the row size and the number of columns of the default slice; determine the second target offset based on the row index and the second row offset, and determine the encoding result of the current row of each slice based on the second target offset and each slice index; preprocess the encoding result of the current row of each slice, and splice the encoding results of the current row of the preprocessed slice to obtain the encoding result of the current row of the video frame; obtain the video frame encoding result based on the encoding results of each current row of the video frame.

[0158] In an exemplary embodiment, as shown in FIG. 20 , a decoding device based on display stream compression is provided, comprising: a second acquisition module 2001, a slice information acquisition module 2002, a decoding core determination module 2003, a second allocation module 2004 and a parallel decoding module 2005, wherein:

[0159] The second acquisition module 2001 is used to obtain the video frame code stream and image parameter information to be decoded;

[0160] The slice information acquisition module 2002 is used to obtain the slice information corresponding to the video frame code stream to be decoded based on the image parameter information;

[0161] A decoding core determination module 2003 is used to determine the number of columns of the slice of the video frame code stream to be decoded based on the slice information, and determine a corresponding number of decoding cores for parallel operation based on the number of columns;

[0162] The second allocation module 2004 is used to allocate the bitstream corresponding to each column of slices to the corresponding decoding core, and the bitstream corresponding to one column of slices corresponds to one decoding core;

[0163] The parallel decoding module 2005 is used to decode the code stream corresponding to the slice in parallel through each decoding core.

[0164] In one of the optional embodiments, the second allocation module 2004 is further used to determine the buffer corresponding to each decoding core; and allocate the code stream of the row in each column slice to the buffer of the corresponding decoding core in sequence.

[0165] In one of the optional embodiments, the above device further includes:

[0166] The second splicing module is used to obtain the decoding results of the rows in each slice output by each decoding core; based on the image parameter information, the decoding results of the rows in each slice are spliced ​​to obtain the video frame decoding result.

[0167] In one of the optional embodiments, the decoding rate control mode of the video frame code stream to be decoded is a fixed rate mode; the above-mentioned second splicing module is also used to determine the second row size of the slice based on the slice width in the image parameter information and the number of bits occupied by each pixel; obtain the third row offset based on the second row size of the slice and the number of columns; determine the third target offset based on the row index and the third row offset, and determine the decoding result of the code stream corresponding to the current row of each slice based on the third target offset and each slice index; splice the decoding results of the code stream corresponding to the current row of each slice to obtain the decoding result of the current row of the video frame; obtain the video frame decoding result based on the decoding results of each current row of the video frame.

[0168] Each module in the encoding device and decoding device based on display stream compression can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0169] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG21. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, an encoding method and a decoding method based on display stream compression are implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0170] Those skilled in the art will understand that the structure shown in FIG. 21 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0171] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0172] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0173] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0174] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0175] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0176] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A coding method based on display stream compression, characterized in that: The method comprises: Obtain the video frame image to be encoded and image parameter information; Based on the image parameter information, the video frame image to be encoded is divided into a plurality of slices; the number of columns of the slices corresponding to the video frame image to be encoded is determined, and based on the number of columns, a corresponding number of encoding cores for parallel operation is determined; Allocating each column of the slices to the corresponding encoding core respectively, and one column of the slices corresponds to one encoding core; The slices are encoded in parallel by the encoding cores.

2. The method according to claim 1, characterized in that: The step of respectively allocating each column of the slices to the corresponding encoding core comprises: Determine a buffer area corresponding to each of the encoding cores; The rows in each column of the slice are sequentially allocated to the buffer corresponding to the encoding core.

3. The method according to claim 1, characterized in that After encoding the slices in parallel by the encoding cores, the method further comprises: Obtaining encoding results of rows in each of the slices output by each of the encoding cores; Based on the image parameter information, the encoding results of the rows in each of the slices are spliced ​​to obtain a video frame encoding result.

4. The method according to claim 3, characterized in that The encoding rate control mode of the video frame image to be encoded is a fixed rate mode; and the encoding results of the rows in each of the slices are spliced ​​based on the image parameter information to obtain the video frame encoding result, including: Determine the size of the first row of the slice based on the slice width and the number of bits occupied by each pixel in the image parameter information; Obtaining a first row offset based on the first row size of the slice and the number of columns; Determine a first target offset based on a row index and the first row offset, and determine an encoding result of a current row of each slice based on the first target offset and each slice index; The encoding result of the current row of the video frame is obtained by splicing the encoding results of the current row of each slice; The video frame encoding result is obtained based on the encoding results of each current row of the video frame.

5. The method according to claim 3, characterized in that: The encoding rate control mode of the video frame image to be encoded is a dynamic rate mode; and the encoding results of the rows in each of the slices are spliced ​​based on the image parameter information to obtain the video frame encoding result, including: Get the row size of the default slice; Obtain a second row offset based on the row size of the default slice and the number of columns; Determine a second target offset based on the row index and the second row offset, and determine an encoding result of a current row of each slice based on the second target offset and each slice index; Preprocessing the encoding results of the current row of each of the slices, and splicing the preprocessed encoding results of the current row of the slices to obtain the encoding result of the current row of the video frame; The video frame encoding result is obtained based on the encoding results of each current row of the video frame.

6. A decoding method based on display stream compression, characterized in that: The method comprises: Obtain the video frame code stream and image parameter information to be decoded; Obtaining slice information corresponding to the to-be-decoded video frame code stream based on the image parameter information; Determine the number of columns of the slice of the to-be-decoded video frame code stream based on the slice information, and determine a corresponding number of decoding cores for parallel operation based on the number of columns; Allocating the bitstreams corresponding to each column of the slices to the corresponding decoding cores, and the bitstreams corresponding to one column of the slices correspond to one decoding core; The code stream corresponding to the slice is decoded in parallel by each of the decoding cores.

7. The method according to claim 6, characterized in that The step of respectively allocating the code stream corresponding to each column of the slices to the corresponding decoding core includes: Determine the buffer area corresponding to each of the decoding cores; The code streams corresponding to the rows in each column of the slice are sequentially allocated to the buffer corresponding to the decoding core.

8. The method according to claim 6, characterized in that After decoding the bitstream corresponding to the slice in parallel by each of the decoding cores, the method further comprises: Obtaining decoding results of code streams corresponding to rows in each of the slices output by each of the decoding cores; Based on the image parameter information, decoding results of the code streams corresponding to the rows in each of the slices are spliced ​​to obtain a video frame decoding result.

9. The method according to claim 8, characterized in that The decoding rate control mode of the video frame code stream to be decoded is a fixed rate mode; the decoding results of the code streams corresponding to the rows in each of the slices are spliced ​​based on the image parameter information to obtain the video frame decoding result, including: Determine a second row size of the slice based on the slice width and the number of bits occupied by each pixel in the image parameter information; Obtain a third row offset based on the second row size of the slice and the number of columns; Determine a third target offset based on the row index and the third row offset, and determine a decoding result of a code stream corresponding to a current row of each slice based on the third target offset and each slice index; The decoding result of the current row of the video frame is obtained by splicing the decoding results of the code streams corresponding to the current rows of the slices; The video frame decoding result is obtained based on the decoding results of each current row of the video frame.

10. An encoding device based on display stream compression, characterized in that: The device comprises: The first acquisition module is used to acquire the video frame image to be encoded and image parameter information; A slicing module, used for dividing the to-be-encoded video frame image into a plurality of slices based on the image parameter information; A coding core determination module, used to determine the number of columns of the slice corresponding to the video frame image to be encoded, and determine a corresponding number of coding cores for parallel operation based on the number of columns; A first allocation module, used for respectively allocating each column of the slices to the corresponding encoding core, and one column of the slices corresponds to one encoding core; The parallel encoding module is used to encode the slice in parallel through each of the encoding cores.

11. A decoding device based on display stream compression, characterized in that: The device comprises: The second acquisition module is used to obtain the video frame code stream and image parameter information to be decoded; A slice information acquisition module, used to obtain slice information corresponding to the video frame code stream to be decoded based on the image parameter information; A decoding core determination module, used to determine the number of columns of the slice of the video frame code stream to be decoded based on the slice information, and determine a corresponding number of decoding cores for parallel operation based on the number of columns; A second allocation module is used to allocate the bitstream corresponding to each column of the slices to the corresponding decoding core, and the bitstream corresponding to one column of the slices corresponds to one decoding core; The parallel decoding module is used to decode the code stream corresponding to the slice in parallel through each of the decoding cores.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images