Entropy decoding processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an entropy decoding processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: for a context model of each intra-frame coding strip, if a current syntax element is a first type syntax element, initializing the current context model, and determining a context variable in the current context model; the first type syntax element comprises a first syntax element of a first coding tree unit of a slice, slice or coding tree unit row; analyzing the current syntax element through the initialized current context model; and storing the context variables in each current context model until the current syntax element is the second type syntax element. By adopting the method, the data volume processed in the entropy decoding process can be reduced, the processing time is shortened, and the processing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an entropy decoding processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of the Internet and mobile communication technologies, the demand for video content has surged, placing higher demands on video coding technology. As the latest generation of video coding standards, H.266 / VVC (Versatile Video Coding, VVC) is designed to provide more efficient compression performance than its predecessor, H.265 / HEVC (High Efficiency Video Coding, HEVC), especially when processing high-resolution and high-dynamic-range video content, and can significantly reduce the bit rate while maintaining or improving visual quality.

[0003] Entropy encoding / decoding is a key component in the video compression / decompression process, responsible for lossless compression / decompression of information such as prediction residuals and motion vectors. Like H.265 / HEVC, VVC's entropy encoding / decoding is mainly implemented through context-adaptive binary arithmetic coding (CABAC). However, with the increase in video coding complexity, data processing during VVC entropy decoding takes longer and is less efficient. Summary of the invention

[0004] Based on this, it is necessary to provide an entropy decoding processing method, device, computer equipment, computer-readable storage medium and computer program product that can shorten the data processing time during entropy decoding and improve processing efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides an entropy decoding processing method, the method comprising:

[0006] For a context model of a coding slice within each frame, if a current syntax element is a first type syntax element, the current context model is initialized to determine a context variable in the current context model; the first type syntax element includes a first syntax element of a first coding tree unit of a slice, a slice, or a coding tree unit row; wherein the slice refers to splitting a frame of an image into a strip shape determined by a plurality of coding tree units; the slice refers to splitting a frame of an image into a rectangular shape determined by a plurality of coding tree units;

[0007] Parsing the current grammatical element by using the initialized current context model;

[0008] Until the current syntax element is a syntax element of the second type, the context variables in each of the current context models are stored.

[0009] In one of the embodiments, the first type of syntax element includes the first syntax element of a slice or the first syntax element of a slice; the current context model includes a first context initialization variable; if the current syntax element is a syntax element of the first type, initializing the current context model to obtain context variables in the current context model includes:

[0010] If the current syntax element is the first syntax element of a slice or the first syntax element of a slice, the first context initial variables are initialized to obtain context variables of the current context model.

[0011] In one of the embodiments, the first type of syntax element is also included in the first coding tree unit of the coding tree unit row, and the syntax element coding mark is a first mark, and the first mark is used to indicate the coding mark of opening the syntax element; the current context model also includes a second context initialization variable; if the current syntax element is the first type of syntax element, initializing the current context model to obtain the context variables in the current context model includes:

[0012] If the current syntax element is in the first coding tree unit of the coding tree unit row, and the syntax element coding flag is the first flag, determining whether the current coding tree unit corresponding to the current syntax element has an adjacent unit;

[0013] If so, the second context initial variable is updated through the adjacent unit, and the updated second context initial variable is used as the context variable of the current context model.

[0014] In one of the embodiments, the current coding tree unit includes a plurality of coding units; and the method further includes:

[0015] Parsing syntax elements of a current coding unit, obtaining a neighboring context variable of a first neighboring unit of the current coding unit, and a neighboring context variable of a second neighboring unit; wherein the first neighboring unit and the second neighboring unit are located in the same slice or slice; and the current coding unit is determined based on the coding unit;

[0016] The neighboring context variable of the first neighboring cell is stored in a register, and the neighboring context variable of the second neighboring cell is stored in a first on-chip storage cell.

[0017] In one embodiment, storing the neighboring context variable of the second neighboring unit in the first on-chip storage unit includes:

[0018] Obtaining the horizontal coordinate of the current coding unit and the width of the current coding unit;

[0019] Determine, according to the horizontal coordinate, a row number and an in-row offset of a neighboring context variable of the second neighboring unit in the first on-chip storage unit;

[0020] Determining, according to the width of the current coding unit, the number of times required to store adjacent context variables;

[0021] The adjacent context variable is written into a target row and a target bit of a corresponding first on-chip storage unit according to the row number, the in-row offset and the number of times.

[0022] In one embodiment, the structural parameters include a width parameter and a depth parameter; and the method for determining the structural parameters of the first on-chip storage unit includes:

[0023] Obtaining a maximum size of a coding tree unit, a minimum size of a coding unit, and a width of each adjacent context variable of a target video coding protocol;

[0024] Determine a width parameter of the first on-chip storage unit based on the maximum size of the coding tree unit, the minimum size of the coding unit and the width of each of the adjacent context variables; wherein the width parameter indicates the total number of bits of each row of the on-chip storage unit;

[0025] Based on a width parameter of the first on-chip storage unit and a maximum width supported for video decoding, a depth parameter of the first on-chip storage unit is determined; wherein the depth parameter indicates a total number of rows of on-chip storage units.

[0026] In one of the embodiments, the second-type syntax element includes the last syntax element of a coding tree unit, and the coding tree unit is the first coding tree unit of a coding tree unit row, and the coding flag of the syntax element is a first flag, and the first flag is used to indicate the coding flag of opening the syntax element; until the current syntax element is a syntax element of the second type, storing the context variables in each of the current context models includes:

[0027] Until the current syntax element is the last syntax element of the coding tree unit, the coding tree unit is the first coding tree unit of the coding tree unit row, and the coding flag of the syntax element is the first flag; the context variables in each of the current context models are stored in the second on-chip storage unit.

[0028] In one of the embodiments, storing the context variables in the current context model in a second on-chip storage unit includes:

[0029] Obtaining a width of the second on-chip storage unit and a width of the context variable;

[0030] Determining the number of context models stored in each row of the second on-chip storage unit based on the width of the second on-chip storage unit and the width of the context variable;

[0031] For each of the context models, determining a row number and an in-row offset of a context variable of the context model in the second on-chip storage unit;

[0032] The context variable is written into a corresponding target row and target bit of the second on-chip storage unit according to the row number and the intra-row offset.

[0033] In one embodiment, the structural parameters include a width parameter and a depth parameter; and the method for determining the structural parameters of the second on-chip storage unit includes:

[0034] Obtaining the number of target context models of the target video coding protocol, the width of context variables of the target context model; and the width of context variables of candidate context models of candidate video coding protocols;

[0035] Determine a width parameter of the second on-chip storage unit based on the width of the context variables of the target context model and the width of the context variables of the candidate context model; wherein the width parameter indicates the total number of bits of each row of the on-chip storage unit; the value of the width parameter includes the width of the context variables of the target context model and the positive integer multiples of the width of the context variables of the candidate context model;

[0036] Based on the number of target context models, the width of the context variables of each target context model, and the width parameter of the second on-chip storage unit, determine the depth parameter of the second on-chip storage unit; wherein the depth parameter indicates the total number of rows of on-chip storage units.

[0037] In a second aspect, the present application further provides an entropy decoding processing device, the device comprising:

[0038] an initialization module, for a context model of a coding slice within each frame, if a current syntax element is a first type syntax element, initializing the current context model, and determining context variables in the current context model; the first type syntax element includes a first syntax element of a first coding tree unit of a slice, a slice, or a coding tree unit row; wherein the slice refers to splitting a frame of image into a strip shape determined by a plurality of coding tree units; the slice refers to splitting a frame of image into a rectangular shape determined by a plurality of coding tree units;

[0039] A first parsing module, configured to parse the current grammatical element by using the initialized current context model;

[0040] The first storage module is configured to store the context variables in each of the current context models until the current syntax element is a syntax element of the second type.

[0041] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0043] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0044] The above entropy decoding processing method, device, computer equipment, computer readable storage medium and computer program product include hundreds of context models in the traditional video coding protocol, among which some context models are specific to the inter prediction mode; and some context models are specific to the intra prediction mode. That is, the intra prediction mode will use a specific context model, but will not use the context model of the inter prediction mode. In the entropy decoding process, first, for the context model of each intra-frame coded slice, if the current syntax element is a first-type syntax element, and the first-type syntax element includes the first syntax element of the first coding tree unit of the slice, slice or coding tree unit row, the current context model is initialized to determine the context variables in the current context model; only for the context model initialization process of the intra-frame coded slice I Slice, the initialization of the context model of the inter-frame coded slice can be directly skipped, which can reduce the amount of calculation and reduce the time of context model initialization in the entropy decoding process. Secondly, the current syntax element is parsed through the initialized current context model; until the current syntax element is a syntax element of the second type, the context variables in each current context model are stored, thereby completing the entropy decoding process. Compared with the conventional technology, each context model is initialized regardless of the type of the current slice Slice. In the entropy decoding process, the present application only performs the context model initialization process for the intra-frame coded slice I Slice, which can reduce the amount of calculation in the entropy decoding process, thereby reducing the time consumption in the entropy decoding process, thereby improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram of a process of initialization during entropy decoding in one embodiment;

[0047] Figure 2 A schematic diagram of a process of initialization during entropy decoding in another embodiment;

[0048] Figure 3 A schematic diagram of a process of initializing a current context model in one embodiment;

[0049] Figure 4 is a schematic diagram of coding tree units adjacent to a current coding tree unit in one embodiment;

[0050] Figure 5 A flowchart of context model initialization in another embodiment;

[0051] Figure 6 A schematic diagram of a process for storing adjacent context variables during entropy decoding in one embodiment;

[0052] Figure 7 A schematic diagram of reference of adjacent coding units CU in one embodiment;

[0053] Figure 8 A schematic diagram of a process of storing adjacent context variables into a first on-chip storage unit in one embodiment;

[0054] Fig. 9 A schematic diagram of a process for determining structural parameters of a first on-chip storage unit in one embodiment;

[0055] Fig.10 A schematic diagram of a process of storing context variables into a second on-chip storage unit during entropy decoding in one embodiment;

[0056] Fig.11 A schematic diagram of a process for determining structural parameters of a second on-chip storage unit in one embodiment;

[0057] Fig.12 is a structural block diagram of an entropy decoding processing device in an embodiment;

[0058] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] In one embodiment, Figure 1 As shown, an entropy decoding processing method is provided. This embodiment takes the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server. In this embodiment, the method includes the following steps S102 to S106:

[0061] Step S102, for the context model of each intra-frame coding slice, if the current syntax element is a first type syntax element, the current context model is initialized to determine the context variables in the current context model; the first type syntax element includes the first syntax element of the first coding tree unit of the slice, the slice or the coding tree unit row.

[0062] Among them, the first type of syntax element includes the first syntax element of the first coding tree unit of a slice, a slice or a coding tree unit row. A slice refers to splitting a frame of an image into a strip determined by a number of coding tree units; a slice refers to splitting a frame of an image into a strip determined by a number of coding tree units; a frame of an image can be divided into one or more slices, each of which can be decoded independently and does not depend on other slices of the same picture. A tile refers to splitting a frame of an image into a rectangular shape determined by a number of coding tree units. A tile is directly composed of a coding tree unit CTU. Like a slice, a tile can also be decoded independently and does not depend on other tiles. A coding tree unit CTU row refers to a number of coding tree unit CTUs arranged in a two-dimensional array. Each row contains a certain number of coding tree unit CTUs, which are adjacent in the horizontal direction.

[0063] Entropy decoding occurs at the forefront of the video decoding process. The entropy decoder is the core of entropy decoding. The input of the entropy decoder is the bitstream, and the bitstream is the output of the entropy encoder. Generally, different video standards have different entropy decoders. The entropy decoder needs to parse each syntax element (syntax) according to the provisions of the video standard. The syntax element syntax is specified by the video standard. Different syntax elements have different meanings, and the entropy decoder has different parsing methods, but they all follow Figure 2 As shown in the process. Figure 2 As shown, when the syntax element syntax is the first syntax element of a slice, the first syntax element of a tile, or the first syntax element of a tile CTU row, an initialization operation is required. When the syntax element syntax is the last syntax element of a CTU, a storage operation is required after parsing. The output of the entropy decoder is the value of the current syntax element syntax. Some of these syntax elements syntax will participate in parsing other syntax elements syntax, and some will participate in other processes of video decoding (such as motion estimation, inverse transformation, etc.).

[0064] Optionally, the server determines the type of the current syntax element through the entropy decoder for the context model of each intra-frame coded slice, and if the current syntax element is any of the first syntax element of Slice, the first syntax element of Tile, and the first syntax element of TileCTU line, the current context model corresponding to the first type of syntax element is initialized to determine the context variables in the current context model. The context variables may be directly assigned or determined by the assignment of other context variables.

[0065] Step S104: parsing the current grammatical element by using the initialized current context model.

[0066] Optionally, the server parses the current grammatical element by using the initialized current context model, for example, each current grammatical element may be parsed by binarization.

[0067] Optionally, if the current syntax element is not a syntax element of the first type, the current syntax element is directly parsed in a binarization manner using a current context model corresponding to the current syntax element.

[0068] Optionally, in a frame of image, when the current syntax element is the first syntax element of a slice, the server initializes the current context model corresponding to the current syntax element, and parses the current syntax element through the initialized context model. After parsing the first syntax element of the slice, the second syntax element of the slice is used as the current syntax element in the parsing order, and the current syntax element is directly parsed through the current context model corresponding to the current syntax element. The third syntax element of the slice is used as the current syntax element, the fourth syntax element of the slice is used as the current syntax element, and so on, and the current syntax element is parsed.

[0069] Step S106 , until the current syntax element is a syntax element of the second type, then the context variables in each current context model are stored.

[0070] Optionally, until the current syntax element is a syntax element of the second type, the server stores the context variables in each current context model. For example, when the syntax element of the second type is the last syntax element of the coding tree unit, or is another specified element, the server stores the context variables in each current context model that was previously initialized and determined. If the syntax element of the second type is the last syntax element of the coding tree unit, the context variables in the n current context models that were previously initialized and determined are stored.

[0071] In an exemplary embodiment, Figure 2As shown. A frame of image is divided into several independent strips, and the strips are determined by several coding tree units. The server determines whether the current syntax element is a first-type syntax element through the decoder. If so, the current context model corresponding to the current syntax element is initialized, and the initialized current context model performs binary parsing on the current syntax element. If not, the current context model directly performs binary parsing on the current syntax element. Until the current syntax element is a second-type syntax element, the server stores the context variables in each current context model through the decoder.

[0072] In the above entropy decoding processing method, the conventional video coding protocol includes hundreds of context models, among which some context models are specific to the inter prediction mode; and some context models are specific to the intra prediction mode. That is, the intra prediction mode will use a specific context model, but will not use the context model of the inter prediction mode. In the entropy decoding process, firstly, for the context model of each intra-frame coded slice, if the current syntax element is a first-type syntax element, the first-type syntax element includes the first syntax element of the first coding tree unit of the slice, slice or coding tree unit row, then the current context model is initialized to determine the context variables in the current context model; only for the context model initialization process of the intra-frame coded slice I Slice, the initialization of the context model of the inter-frame coded slice can be directly skipped, which can reduce the amount of calculation and the time of context model initialization in the entropy decoding process. Secondly, the current syntax element is parsed by the initialized current context model; until the current syntax element is a second-type syntax element, the context variables in each current context model are stored, thereby completing the entropy decoding process. Compared with the traditional technology, no matter what type of the current slice Slice is, each context model will be initialized. In the entropy decoding process, the present application only performs the context model initialization process for the intra-frame coded slice I Slice, which can reduce the amount of calculation in the entropy decoding process, thereby reducing the time consumption in the entropy decoding process and improving efficiency.

[0073] In an exemplary embodiment, the first type of syntax element includes the first syntax element of a slice or the first syntax element of a slice; if the current syntax element is the first type of syntax element, the current context model is initialized to obtain context variables in the current context model, including: if the current syntax element is the first syntax element of a slice or the first syntax element of a slice, the first context initial variable is initialized to obtain the context variables of the current context model.

[0074] The first context initial variable refers to a variable included in the current context model corresponding to the first syntax element of the slice or the part.

[0075] Optionally, if the current syntax element syntax is the first syntax element of a slice Slice or the first syntax element of a tile Tile, the first context initial variable is initialized to obtain context variables (pStateIdx0 and pStateIdx1) of the current context model.

[0076] The process of initializing using the first context initial variable in the context model is shown in the following table. The initialization of each context model requires multiple shift, multiplication and addition operations.

[0077] Table 1 Context model initialization process

[0078]

[0079] In this embodiment, through the context model initialization process only for the intra-coded slice I Slice, the first type of syntax element includes the first syntax element of the slice or the first syntax element of the slice to meet the initialization condition, so that in the entropy decoding process, the amount of calculation of the entropy decoding process can be reduced, thereby reducing the time consumption in the entropy decoding process, thereby improving efficiency.

[0080] In an exemplary embodiment, Figure 3 As shown, the first type of syntax element is also included in the first coding tree unit of the coding tree unit row, and the syntax element coding mark is the first mark, and the first mark is used to indicate the coding mark of the open syntax element; the current context model also includes a second context initial variable; if the current syntax element is a first type of syntax element, the current context model is initialized to obtain the context variables in the current context model, including the following steps S302 to S304. Wherein:

[0081] Step S302: If the current syntax element is in the first coding tree unit of the coding tree unit row, and the syntax element coding flag is the first flag, it is determined whether the current coding tree unit corresponding to the current syntax element has an adjacent unit.

[0082] Optionally, if the current syntax element syntax is in the first coding tree unit of the coding tree unit row, and the syntax element entropy_coding_sync_enabled_flag is 1, the server determines whether the current coding tree unit corresponding to the current syntax element has an adjacent unit. At this time, the first flag is 1, and the first flag is used to indicate the coding flag of the syntax element. Figure 4 As shown, each grid is a coding tree unit. The coding tree unit in the lower left corner is the current coding tree unit, and it is necessary to further determine whether the current coding tree unit has an adjacent unit T.

[0083] Step S304: if it exists, update the second context initial variable through the adjacent unit, and use the updated second context initial variable as the context variable of the current context model.

[0084] The second context initial variable refers to the variable included in the current context model corresponding to the last grammatical element.

[0085] Optionally, continue as Figure 4 As shown, if there is an adjacent unit T, it is necessary to update the previous context model according to the adjacent unit T, that is, use the context initial variable values ​​in the decoded context model as the context variable values ​​(pStateIdx0 and pStateIdx1) of the current context model. The purpose of this is to make full use of the decoded information to predict the subsequent information.

[0086] Optionally, when the video coding protocol is H.266 / VVC, since VVC uses two hypothetical context models to predict the symbol probability of each context, the update of each context model requires two types of variables:

[0087] 1) Model update rate shiftIdx, the update rates shift0 and shift1 of the two context models are calculated by shiftIdx:

[0088] shift0 = ( shiftIdx >> 2 ) + 2

[0089] shift1 = ( shiftIdx & 3 ) + 3 + shift0

[0090] 2) Predict the probability states pStateIdx0 and pStateIdx1. After the entropy decoder parses each bin of the syntax in a normal way, the probability state will also be updated according to the parsed result binVal (0 or 1):

[0091] pStateIdx0 = pStateIdx0 - ( pStateIdx0 >> shift0 ) + ( 1023 * binVal >> shift0 )

[0092] pStateIdx1 = pStateIdx1 - ( pStateIdx1 >> shift1 ) + ( 16383 * binVal >> shift1 )

[0093] Since pStateIdx0 is a 10-bit unsigned integer and pStateIdx1 is a 14-bit unsigned integer, only 24 bits are needed to store a context model, and 378 context models require a total of 9072 bits to store.

[0094] In this application, the initial value initValue required for initializing shiftIdx and pStateIdx0 / pStateIdx1 is stored in a 378x4 constant two-dimensional array, each value occupies 6 bits, recorded as h266_init_table

[378] [4], and the array is stored in ROM. The array data is shown in Table 2

[0095] Table 2 VVC storage array

[0096]

[0097] Among them, i=0~377 of h266_init_table[i][j] represents the offset of a specific context; j=0~2 corresponds to initType=0~2, and initType is determined by the type of the current Slice; j=3 represents the update rate shiftIdx corresponding to the context model.

[0098] In an alternative embodiment, Figure 5 As shown, the server first determines whether the current syntax element is the first syntax element of the slice Slice or the first syntax element of the tile through the decoder. If so, the first context initial variable is initialized to obtain the context variable of the current context model. If not, it is further determined that the current syntax element is in the first coding tree unit of the coding tree unit row, and when the syntax element entropy_coding_sync_enabled_flag is 1, the server determines whether there is an adjacent unit for the current coding tree unit corresponding to the current syntax element. If there is an adjacent unit T, the previous context model needs to be updated according to the adjacent unit T, that is, the context initial variable value in the decoded context model is used as the context variable value of the current context model, so as to complete the decoder initialization.

[0099] For example, VVC has a total of 378 context models, of which 37 context models are specific to the inter-frame prediction mode, that is, the intra-frame prediction mode will not use these context models. Since there is only intra-frame prediction mode in I Slice, the context model initialization process for I Slice directly skips these 37 context models. However, the traditional initialization method will initialize all context models regardless of the type of the current Slice. Compared with the traditional initialization method, the initialization process optimization method involved in the present invention can reduce the amount of calculation by approximately 9.8% in the context model initialization process of I Slice, and reduces the time for context model initialization. It should be noted that this optimization method is also applicable to the context model initialization process of other video standards.

[0100] In this embodiment, the context model of each intra-frame coding slice is used to determine whether the current syntax element satisfies the first coding tree unit in the coding tree unit row, and the coding mark of the syntax element is the first mark. If it is satisfied, only the intra-frame prediction slice is initialized, which reduces the initialization calculation amount and the time for context model initialization.

[0101] In an exemplary embodiment, Figure 6 As shown, the current coding tree unit includes several coding units; the method further includes steps S602 to S604. Among them:

[0102] Step S602: parse the syntax elements of the current coding unit, and obtain the neighboring context variables of the first neighboring unit and the neighboring context variables of the second neighboring unit of the current coding unit.

[0103] The first adjacent unit refers to the adjacent unit above the current coding unit; the second adjacent unit refers to the adjacent unit to the left of the current coding unit; the first adjacent unit and the second adjacent unit are located in the same stripe or slice. One of the coding units is selected as the current coding unit.

[0104] Optionally, during the entropy decoding process, when parsing the syntax element syntax of the current coding unit CU, the information of the upper adjacent coding unit CU, i.e., the first adjacent unit and the left adjacent coding unit CU, i.e., the second adjacent unit, may also be used, which is called the neighbor context variable. Figure 7 As shown, if the current CU is CU2, when it parses certain syntax elements syntax, it needs to use the information of the upper adjacent unit CU0 and the left adjacent unit CU1.

[0105] Step S604: store the neighboring context variable of the first neighboring unit in a register, and store the neighboring context variable of the second neighboring unit in a first on-chip storage unit.

[0106] Since the sizes of the context variables of the first neighboring cell and the second neighboring cell are different, the neighboring context variables of the first neighboring cell and the neighboring context variables of the second neighboring cell are stored in different locations.

[0107] Among them, the first adjacent unit, that is, the upper adjacent unit, since CU0 and CU2 are not in the same CTU, and from the decoding order, CU0 and CU2 are far apart, if the information of the last row of CU in CTU0 and CTU1 is stored locally, a large storage space is required, so the information of the upper adjacent unit needs to be stored in the on-chip storage unit SRAM. When decoding CU2, it is necessary to read the information of CU0 from the on-chip storage unit SRAM according to the location information of CU2, that is, the upper adjacent context variable (above_neighbor_ctx). It should be noted that the adjacent context variable cannot cross Slices or Tiles, so in Slices or Tiles, except for the last row of CTU, the information of the bottom row of CU (gray area in the figure) of all other CTUs needs to be stored in the on-chip storage unit SRAM.

[0108] The second neighboring unit, that is, the left neighboring unit, needs to store up to 32 (the maximum size of CTU is 128, and the minimum size of CU is 4) neighboring context variables, and the required storage capacity is relatively small, so the left neighboring context variable (left_neighbor_ctx) does not need to be stored in the on-chip storage unit SRAM, and can be directly stored in the register.

[0109] CU2 is also the upper neighbor unit of CU3 and the left neighbor unit of CU4, so when CU2 is parsed, it is necessary to update the local upper neighbor context variable above_neighbor_ctx of the upper neighbor unit and the left neighbor context variable left_neighbor_ctx of the left neighbor unit in time.

[0110] Optionally, the server stores the neighboring context variable of the first neighboring unit, ie, the upper neighboring unit, in a register, and stores the neighboring context variable of the second neighboring unit, ie, the left neighboring unit, in a first on-chip storage unit.

[0111] There is no specific regulation for the adjacent context variables here. Generally, it is considered to be the information necessary for parsing the syntax element syntax (such as cqtDepth, CuPredMode, etc.). This information also needs to be reasonably optimized to reduce the amount of data. Each adjacent context in this application is 16 bits in total.

[0112] To facilitate understanding of adjacent context variables, the following takes parsing pred_mode_flag as an example. It is necessary to first calculate the ctxInc of the current syntax element syntax, and then calculate the value of pred_mode_flag from the corresponding context model. The calculation method of ctxInc is as follows:

[0113] ctxInc = ( condL && availableL ) | | ( condA && availableA )

[0114] Among them, condL indicates whether the CuPredMode of the left adjacent unit is MODE_INTRA, conA indicates whether the CuPredMode of the upper adjacent unit is MODE_INTRA, and availableL and availableA respectively indicate whether the left adjacent unit and the upper adjacent unit exist. It can be seen that CuPredMode is a necessary adjacent context variable for parsing pred_mode_flag.

[0115] In this embodiment, by parsing the syntax elements of the current coding unit, different adjacent context variables are obtained, and different adjacent context variables are stored in corresponding positions according to the sizes of different adjacent context variables, so that the corresponding context variables can be quickly read during entropy decoding.

[0116] In an exemplary embodiment, Figure 8 As shown, storing the adjacent context variable of the second adjacent unit in the first on-chip storage unit includes steps S802 to S808.

[0117] Step S802: Obtain the horizontal coordinate of the current coding unit and the width of the current coding unit.

[0118] Optionally, the server obtains the horizontal coordinate cuX of the current coding unit CU and the width cuWidth (an integer multiple of 4) of the current coding unit.

[0119] Step S804: Determine the row number and the in-row offset of the adjacent context variable of the second adjacent cell in the first on-chip storage unit according to the horizontal coordinate.

[0120] The first on-chip storage unit is a storage unit that stores an adjacent context variable of a left adjacent unit.

[0121] Optionally, the server calculates the target row number and the in-row offset of the on-chip storage unit SRAM to be written according to the horizontal coordinate cuX of the current CU.

[0122] Step S806: Determine the number of times required to store adjacent context variables according to the width of the current coding unit.

[0123] Optionally, the server determines the number of times data needs to be repeatedly stored according to the width cuWidth (an integer multiple of 4) of the current CU.

[0124] Step S808 , writing the adjacent context variable into the target row and target bit of the corresponding first on-chip storage unit according to the row number, the offset within the row and the number of times.

[0125] Optionally, the server writes the adjacent context variable contextData into a target row and a target bit of a corresponding first on-chip storage unit SRAM according to the row number, the offset within the row and the number of times.

[0126] Accordingly, the steps to read adjacent context data are as follows:

[0127] 1) According to the horizontal coordinate cuX of the current CU, the target row number and the offset within the row of the on-chip SRAM to be written are calculated.

[0128] 2) Read designated bits of data from a specific row and bit of the first on-chip SRAM.

[0129] 3) Return the read adjacent context data contextData.

[0130] In this embodiment, by storing the adjacent context variables of the left adjacent unit in the first on-chip storage unit for different types of data, the data reading and writing efficiency can be improved.

[0131] In the above embodiment, Fig. 9 As shown, the structural parameters include width parameters and depth parameters; the method for determining the structural parameters of the first on-chip storage unit includes steps S902 to S906. Among them:

[0132] Step S902: Obtain the maximum size of a coding tree unit of a target video coding protocol, the minimum size of a coding unit, and the width of each adjacent context variable.

[0133] The target video coding protocol includes VVC, and may also include HEVC or AVC.

[0134] Optionally, taking the target video coding protocol as VVC as an example, the server obtains the maximum size of the coding tree unit of VVC, the minimum size of the coding unit, and the width of each adjacent context variable. Among them, each reading and writing of the adjacent context variable is in units of CTU, the maximum size of CTU in VVC is 128, the minimum size of CU is 4, and the width of each adjacent context variable is 16 bits in total.

[0135] Step S904, determining a width parameter of the first on-chip storage unit based on the maximum size of the coding tree unit, the minimum size of the coding unit and the width of each adjacent context variable; wherein the width parameter indicates the total number of bits of each row of the on-chip storage unit.

[0136] Optionally, the server may determine a width parameter of the first on-chip storage unit by using a maximum size of a coding tree unit of a target video coding protocol, a minimum size of a coding unit, and a width of each adjacent context variable. The server divides the maximum size of the coding tree unit by the minimum size of the coding unit, and then multiplies the result by the width of each adjacent context variable, as shown in formula (1).

[0137] Formula (1)

[0138] Among them, W SRAM Represents the width parameter of the first on-chip storage unit; W CTU Indicates the maximum size of CTU, W CU Indicates the minimum size of CU, W ctx Indicates the width of each adjacent context data.

[0139] Optionally, the target video coding protocol is VVC, the maximum size of CTU in VVC is 128, and the minimum size of CU is 4. Therefore, at most 32 (128 / 4) adjacent context variables need to be read and written each time, and the width of each adjacent context variable is 16 bits, so the total is 512 bits (16 bits * 32). In this way, enough data can be stored or read in each clock cycle, thereby improving the data reading speed.

[0140] Optionally, if the target video coding protocol is HEVC, the maximum size of CTU in HEVC is 64, and the minimum size of CU is 8. If the target video coding protocol is AVC, the MB in AVC is a fixed size of 16x16. It can be seen that if the target video coding protocol is VVC, the adjacent context data of each CTU of HEVC and AVC is less than that of VVC, that is, HEVC and AVC can fully reuse the on-chip SRAM to ensure that the adjacent context variables of a CTU are pre-fetched in one clock cycle.

[0141] Step S906 , determining a depth parameter of the first on-chip storage unit based on a width parameter of the first on-chip storage unit and a maximum width supported for video decoding; wherein the depth parameter indicates the total number of rows of the on-chip storage unit.

[0142] Optionally, the server determines the depth parameter of the first on-chip storage unit by a ratio of a maximum width supporting video decoding to a width parameter of the first on-chip storage unit.

[0143] Formula (2)

[0144] Among them, W SRAM Represents the width parameter of the first on-chip storage unit; W F Indicates the maximum width supported for video decoding; D SRAM Indicates the depth parameter of the first on-chip storage unit.

[0145] Optionally, taking 8K resolution video as an example, the size of CTU is 128, the minimum size of CU is 4, and there are at most 64 CTUs in one row, that is, the on-chip SRAM only needs to store adjacent context variables of 64 CTUs at most, and each CTU requires at most one on-chip SRAM row, so a total of at most 64 on-chip SRAM rows are required. That is, the cache utilization of the first on-chip storage unit SRAM reaches 100%, thereby improving the cache utilization.

[0146] In an optional embodiment, the first on-chip storage unit SRAM has a width of 512 bits and a depth of 64 bits.

[0147] In this embodiment, through the maximum size of the coding tree unit of the target video coding protocol, the minimum size of the coding unit, the width of each adjacent context variable, and the maximum width supported for video decoding, it is possible to achieve a cache utilization rate of 100% for the first on-chip storage unit SRAM, thereby improving the cache utilization rate of the first on-chip storage unit SRAM. The first on-chip storage unit is easy to expand, and by increasing the width and depth of the on-chip SRAM, it can support larger resolutions or larger CTU sizes in the future.

[0148] In an exemplary embodiment, the second type syntax element includes the last syntax element of the coding tree unit, and the coding tree unit is the first coding tree unit of the coding tree unit row, and the coding mark of the syntax element is the first mark, and the first mark is used to indicate the coding mark of the open syntax element; until the current syntax element is the second type syntax element, the context variables in each current context model are stored, including: until the current syntax element is the last syntax element of the coding tree unit, and the coding tree unit is the first coding tree unit of the coding tree unit row, and the coding mark of the syntax element is the first mark; the context variables in each current context model are stored in the second on-chip storage unit.

[0149] The second on-chip storage unit is a storage unit for storing context variables in each context model.

[0150] Optionally, until the current syntax element is the last syntax element of the coding tree unit, the coding tree unit is the first coding tree unit of the coding tree unit row, and the coding mark of the syntax element is the first mark, the server stores each context variable in the current context model in the second on-chip storage unit.

[0151] In this embodiment, the context variables are all stored in the second on-chip storage unit according to the characteristics of the data to be stored, which can improve the data reading speed and the utilization rate of the on-chip storage unit.

[0152] In the above embodiment, Fig.10 As shown, storing the context variables in the current context model in the second on-chip storage unit includes steps S1002 to S1008.

[0153] Step S1002, obtaining the width of the second on-chip storage unit and the width of the context variable.

[0154] Optionally, assume that the width of the second on-chip storage unit SRAM is 168 bits and the depth is 54. The server obtains the width of the second on-chip storage unit 168 bits and the width of the context variable. Among them, in VVC, the context variables include pStateIdx0 and pStateIdx1. Since pStateIdx0 is a 10-bit unsigned integer and pStateIdx1 is a 14-bit unsigned integer, only 24 bits are needed to store a context model, that is, the width of the context variable is 24 bits. Since there are 378 context models in VVC, a total of 9072 bits are required for storage.

[0155] Step S1004 , determining the number of context models stored in each row of the second on-chip storage unit based on the width of the second on-chip storage unit and the width of the context variable.

[0156] Optionally, the server calculates how many context models can be stored in each row based on the width of the second on-chip storage unit (168 bits) and the width of the context variable (24 bits) by using the width of the second on-chip storage unit / width of the context variable = 168 / 24 = 7, that is, the second on-chip storage unit can store 7 context models in one row.

[0157] Step S1006: for each context model, determine the row number and the offset within the row of the context variable of the context model in the second on-chip storage unit.

[0158] Optionally, for each context model, the server calculates the current line currentLine and the intra-line offset lineOffset through division and modulo operations, that is, the row number and intra-line offset of the current context model data in the second on-chip storage unit SRAM.

[0159] Step S1008 , writing the context variable into the target row and target bit of the corresponding second on-chip storage unit according to the row number and the offset within the row.

[0160] Optionally, the server performs a corresponding write operation according to the number of bits required to be stored, and writes the context variable into a target row and a target bit of a corresponding second on-chip storage unit according to the row number and the offset within the row.

[0161] In this embodiment, by storing the context variables of the context model in the second on-chip storage unit for different types of data, data reading and writing efficiency can be improved.

[0162] In the above embodiment, Fig.11 As shown, the structural parameters include width parameters and depth parameters; the method for determining the structural parameters of the second on-chip storage unit includes steps S1102 to S1106. Among them:

[0163] Step S1102, obtaining the number of target context models of the target video coding protocol, the width of the context variables of the target context model; and the width of the context variables of the candidate context models of the candidate video coding protocol.

[0164] The target video coding protocol can be VVC, HEVC or AVC. VVC has a total of 378 context models, the context variable is 24 bits, and a total of 9072 bits are required for storage; HEVC has 460 context models, the context variable is only 7 bits, and a total of 3220 bits are required for storage; AVC has 1024 context models, and the context variable is the same as HEVC, only 7 bits, and a total of 7168 bits are required for storage. Candidate video coding protocols include HEVC or AVC.

[0165] Optionally, taking the target video coding protocol as VVC and the candidate video coding protocols including at least one of HEVC and AVC as an example, the number of target context models obtained by the server in VVC is 378, and the width of the context variable of the target context model is 24 bits; the candidate video coding protocol HEVC includes 460 context models, and the width of the context variable is only 7 bits; the candidate video coding protocol AVC has 1024 context models, and the width of the context variable is also 7 bits.

[0166] Step S1104 , determining a width parameter of a second on-chip storage unit based on the width of the context variables of the target context model and the width of the context variables of the candidate context model.

[0167] Among them, the width parameter indicates the total number of bits in each row of the on-chip storage unit; the value of the width parameter includes the width of the context variables of the target context model and the positive integer multiples of the width of the context variables of the candidate context model.

[0168] Optionally, the server determines that the width parameter of the second on-chip storage unit is 168 bits based on the width of the context variable of the target context model in VVC, which is 24 bits, the width of the context variable of the candidate context model in HEVC, which is 7 bits, and the width of the context variable of the candidate context model in AVC, which is 7 bits, such as taking the least common multiple of 24 and 7, which is 168 bits. By analogy, the width parameter of the second on-chip storage unit can also be 336 bits, 504 bits, and so on.

[0169] Step S1106 , determining a depth parameter of the second on-chip storage unit based on the number of target context models, the width of the context variable of each target context model, and the width parameter of the second on-chip storage unit.

[0170] The depth parameter indicates the total number of rows of memory cells on the chip.

[0171] Optionally, the number of target context models is 378, and the width of the context variable of each target context model is 24 bits, requiring a total of 9072 bits for storage. Assuming that the width parameter of the second on-chip storage unit is 168 bits, the server determines that the depth parameter of the second on-chip storage unit is 9072 bits / 168 bits=54. Similarly, assuming that the width parameter of the second on-chip storage unit is 336 bits, the server determines that the depth parameter of the second on-chip storage unit is 9072 bits / 336 bits=27.

[0172] In this embodiment, by designing the depth and width of the second on-chip storage unit, each context model of VVC is 24 bits, and the width of the second on-chip storage unit SRAM is an integer multiple of it, which can ensure that the data obtained in each clock cycle is complete, and there will be no cross-row situation, thereby improving the cache read and write efficiency. VVC has a total of 378 context models, requiring a total of 9072 bits for storage. The second on-chip storage unit SRAM can store 7 context models per row, requiring a total of 54 rows. That is, the cache utilization rate of the second on-chip storage unit SRAM reaches 100%, thereby improving the cache utilization rate. Each context model of HEVC and AVC is 7 bits, and the width of the second on-chip storage unit SRAM is an integer multiple of it, which can ensure that the data obtained in each clock cycle is complete, and there will be no cross-row situation. Moreover, HEVC has only 460 context models and AVC has 1024 context models, and the second on-chip storage unit SRAM can fully store these data; it can be used for the target video coding protocol while being compatible with the candidate video coding protocol, thereby reducing hardware costs and improving the reusability of the second on-chip storage unit.

[0173] In an exemplary embodiment, the target video parsing protocol is VVC. Entropy decoding occurs at the front end of the video decoding process. The entropy decoder is the core of entropy decoding. The input of the entropy decoder is the bitstream, and the bitstream is the output of the entropy encoder. Generally, different video standards have different entropy decoders. The entropy decoder needs to parse each syntax element (syntax) according to the provisions of the video standard. The syntax element syntax is specified by the video standard. Different syntax elements have different meanings, and the entropy decoder has different parsing methods, but they all follow Figure 2 As shown in the process. Figure 2As shown, for the context model of each intra-frame coded strip, when the syntax element syntax is the first syntax element of Slice, the first syntax element of Tile, or the first syntax element of Tile CTU row, the context model needs to be initialized (Initialization) operation. When the syntax element syntax is the last syntax element of CTU, a storage (Storage) operation is required after parsing. The output of the entropy decoder is the value of the current syntax element syntax. Some of these syntax elements syntax will participate in parsing other syntax elements syntax, and some will participate in other processes of video decoding (such as motion estimation, inverse transformation, etc.). The server determines the type of the current syntax element through the entropy decoder. If the current syntax element is any of the first syntax element of Slice, the first syntax element of Tile, and the first syntax element of Tile CTU row, the current context model corresponding to the first type of syntax element is initialized to determine the context variables in the current context model. Among them, the context variable can be directly assigned or determined by the assignment of other context variables.

[0174] like Figure 2 As shown. A frame of image is divided into several independent strips, and the strips are determined by several coding tree units. The server determines whether the current syntax element is a first-type syntax element through the decoder for the context model of the coded strips in each frame, if the first-type syntax element includes the first syntax element of the strip or the first syntax element of the slice. If so, the current context model corresponding to the current syntax element is initialized, and the initialized current context model performs binary parsing on the current syntax element. If not, the current context model directly performs binary parsing on the current syntax element. Until the current syntax element is a second-type syntax element, the server stores the context variables in each current context model through the decoder.

[0175] The first type of syntax element is also included in the first coding tree unit of the coding tree unit row, and the syntax element coding mark is the first mark, and the first mark is used to indicate the coding mark of the open syntax element; the current context model also includes a second context initial variable; if the current syntax element syntax is in the first coding tree unit of the coding tree unit row, and the syntax element entropy_coding_sync_enabled_flag is 1, the server determines whether there is an adjacent unit for the current coding tree unit corresponding to the current syntax element. At this time, the first mark is 1, and the first mark is used to indicate the coding mark of the open syntax element. Figure 4As shown in the figure, each grid is a coding tree unit. The coding tree unit in the lower left corner is the current coding tree unit. It is necessary to further determine whether the current coding tree unit has an adjacent unit T. If there is an adjacent unit T, the previous context model needs to be updated according to the adjacent unit T, that is, the context initial variable value in the decoded context model is used as the context variable value (pStateIdx0 and pStateIdx1) of the current context model. The purpose of this is to make full use of the decoded information to predict the subsequent information.

[0176] The process of initializing using the first context initial variable in the context model is shown in the following table. The initialization of each context model requires multiple shift, multiplication and addition operations.

[0177] Table 1 Context model initialization process

[0178]

[0179] VVC has a total of 378 context models, of which 37 are specific to the inter-prediction mode, i.e., the intra-prediction mode will not use these context models. Since there is only the intra-prediction mode in I Slice, the context model initialization process for I Slice in the present invention directly skips these 37 context models.

[0180] When the video coding protocol is H.266 / VVC, since VVC uses two hypothetical context models to predict the symbol probability of each context, the update of each context model requires two types of variables:

[0181] Model update rate shiftIdx, the update rates shift0 and shift1 of the two context models are calculated by shiftIdx:

[0182] shift0 = ( shiftIdx >> 2 ) + 2

[0183] shift1 = ( shiftIdx & 3 ) + 3 + shift0

[0184] Predict the probability states pStateIdx0 and pStateIdx1. After the entropy decoder parses each bin of the syntax in a normal way, the probability state will also be updated according to the parsed result binVal (0 or 1):

[0185] pStateIdx0 = pStateIdx0 - ( pStateIdx0 >> shift0 ) + ( 1023 * binVal >> shift0 )

[0186] pStateIdx1 = pStateIdx1 - ( pStateIdx1 >> shift1 ) + ( 16383 * binVal >> shift1 )

[0187] Since pStateIdx0 is a 10-bit unsigned integer and pStateIdx1 is a 14-bit unsigned integer, only 24 bits are needed to store a context model, and 378 context models require a total of 9072 bits to store.

[0188] In this application, the initial value initValue required for initializing shiftIdx and pStateIdx0 / pStateIdx1 is stored in a 378x4 constant two-dimensional array, each value occupies 6 bits, recorded as h266_init_table

[378] [4]. The array is stored in ROM, and the array data is shown in Table 2.

[0189] Table 2 VVC storage array

[0190]

[0191] Among them, i=0~377 of h266_init_table[i][j] represents the offset of a specific context; j=0~2 corresponds to initType=0~2, and initType is determined by the type of the current Slice; j=3 represents the update rate shiftIdx corresponding to the context model.

[0192] The current syntax element is parsed through the initialized current context model; the second type of syntax element includes the last syntax element of the coding tree unit; until the current syntax element is the last syntax element of the coding tree unit, the width of the second on-chip storage unit and the width of the context variable are obtained; it is assumed that the width of the second on-chip storage unit is 168 bits and the depth is 54. Based on the width of the second on-chip storage unit of 168 bits and the width of the context variable of 24 bits, the server determines the number of context models stored in each row of the second on-chip storage unit, which is 7; for each context model, the row number and the in-row offset of the context variable of the context model in the second on-chip storage unit are determined; and the context variable is written into the target row and target bit of the corresponding second on-chip storage unit according to the row number and the in-row offset.

[0193] The steps to store the context variable to the second on-chip storage unit are as follows: Calculate how many context models can be stored in each row based on the on-chip SRAM width and the width of the context model data. Calculate currentLine and lineOffset, that is, the row number and offset within the row of the current context model data in the on-chip SRAM, through division and modulo operations. Perform the corresponding write operation according to the number of bits to be stored, and lines[currentLine] represents the row with the row number currentLine in the on-chip SRAM. The writeBits function here is used to write the specified bit of data to a specific row and bit of the on-chip SRAM. The pseudo code is shown in Table 3:

[0194] Table 3 Pseudo code for storing context variables to the second on-chip storage unit

[0195]

[0196] The steps to read the context model data from the second on-chip storage unit are as follows: Calculate how many context models can be stored in each row based on the on-chip SRAM width and the width of the context model data. Calculate currentLine and lineOffset, that is, the row number and offset within the row of the current context model data in the on-chip SRAM, through division and modulo operations. Perform the corresponding read operation according to the number of bits to be read, and lines[currentLine] represents the row with the row number currentLine in the on-chip SRAM. The function returns the read model data modelData. The readBits function here is used to read specified bits of data from a specific row and bit of the on-chip SRAM. The pseudo code is shown in Table 4:

[0197] Table 4 Pseudo code for reading context model data from the second on-chip storage unit

[0198]

[0199] The second on-chip SRAM for storing context model variables has a width of 168 bits and a depth of 54. This design has the following advantages:

[0200] 1. Improve cache read and write efficiency: Each context model of VVC is 24 bits, and the width of the on-chip SRAM is an integer multiple of it, which can ensure that the data obtained in each clock cycle is complete and will not cross rows;

[0201] 2. Improve cache utilization: VVC has a total of 378 context models, which require a total of 9072 bits for storage. Each row of the on-chip SRAM can store 7 context models, requiring a total of 54 rows. That is, the cache utilization of the on-chip SRAM reaches 100%.

[0202] 3. Reduce hardware cost (reusability): Each context model of HEVC and AVC is 7 bits, and the width of the on-chip SRAM is an integer multiple of it, which can ensure that the data obtained in each clock cycle is complete and there will be no cross-row situation. In addition, HEVC has only 460 context models and AVC has 1024 context models, and the on-chip SRAM can fully store this data.

[0203] Among them, the structural parameters include width parameters and depth parameters; the method for determining the structural parameters of the second on-chip storage unit includes: obtaining the number of target context models of VVC, 378, the width of the context variables of the target context model, 24bit; and the width of the context variables of the candidate context models of the candidate video coding protocol; based on the width of the context variables of the target context model, and the candidate video coding protocol HEVC, including 460 context models, the width of the context variables is only 7bit, and the candidate video coding protocol AVC, there are 1024 context models, and the width of the context variables is also 7bit. The server is based on the width of the context variables of the target context model in VVC, 24bit, the width of the context variables of the candidate context model in HEVC, 7bit, and the width of the context variables of the candidate context model in AVC, 7bit. If the least common multiple of 24 and 7 is 168bit, then the width parameter of the second on-chip storage unit is determined to be 168bit. Then the server determines the depth parameter of the second on-chip storage unit to be 9072bit / 168bit=54. The width parameter indicates the total number of bits in each row of the on-chip storage unit; the depth parameter indicates the total number of rows of the on-chip storage unit.

[0204] During entropy decoding, when parsing the syntax element syntax of the current coding unit CU, the information of the upper adjacent coding unit CU, i.e. the first adjacent unit and the left adjacent coding unit CU, i.e. the second adjacent unit, may also be used, which is called the neighbor context variable. Figure 7As shown, if the current CU is CU2, when it parses certain syntax elements syntax, it needs to use the information of the upper adjacent unit CU0 and the left adjacent unit CU1. The server stores the adjacent context variables of the first adjacent unit, that is, the upper adjacent unit, in a register, and stores the adjacent context variables of the second adjacent unit, that is, the left adjacent unit, in the first on-chip storage unit. The adjacent context variables of the first adjacent unit are stored in the register, and the server obtains the horizontal coordinate cuX of the current coding unit CU and the width cuWidth (an integer multiple of 4) of the current coding unit; according to the horizontal coordinate cuX of the current CU, the target row number and the in-row offset of the on-chip storage unit SRAM to be written are calculated; the width cuWidth (an integer multiple of 4) of the current CU is used to determine the number of times the data needs to be repeatedly stored. The server writes the adjacent context variable contextData to the target row and target bit of the corresponding first on-chip storage unit SRAM according to the row number, the in-row offset and the number of times.

[0205] The adjacent context data storage operation occurs after each CU is parsed. The later parsed CU may overwrite the earlier parsed CU information at the same horizontal position, so this operation also belongs to the adjacent context data update. Since the width of the on-chip SRAM is 512 bits, the bit width of each adjacent context data is 16 bits, and the minimum CU width is 4, the on-chip SRAM can store 32 CU information with a width of 4 in one row. The pseudo code for storage is shown in Table 5:

[0206] Table 5 Pseudo code for storing (updating) adjacent context data

[0207]

[0208] Accordingly, the steps for reading the adjacent context data are as follows: According to the horizontal coordinate cuX of the current CU, calculate the target row number and the offset within the row of the on-chip SRAM to be written. Read the specified bit of data from the specific row and bit of the first on-chip SRAM. Return the read adjacent context data contextData. The pseudo code for reading is shown in Table 6:

[0209] Table 6 Pseudo code for reading adjacent context data

[0210]

[0211] The first on-chip SRAM that stores adjacent context variables has a width of 512 bits and a depth of 64 bits. This design has the following advantages:

[0212] 1. Improve cache read and write efficiency: Each read and write of adjacent context is in CTUs. The maximum size of CTU in VVC is 128, and the minimum size of CU is 4. Therefore, at most 32 adjacent context data need to be read and written each time, a total of 512 bits. In this way, enough data can be stored or read in each clock cycle.

[0213] 2. Improve cache utilization: Taking 8K resolution video as an example, the size of CTU is 128, and there are 64 CTUs in a row at most. That is, the on-chip SRAM only needs to store the adjacent context data of 64 CTUs at most, and each CTU requires at most one on-chip SRAM row, so a total of 64 on-chip SRAM rows are required at most. That is, the cache utilization of the on-chip SRAM reaches 100%.

[0214] 3. Reduce hardware cost (reusability): The maximum size of CTU in HEVC is 64, and the minimum size of CU is 8. In AVC, it is a fixed size of 16x16 MB. It can be seen that the adjacent context data of each CTU in HEVC and AVC is less than that of VVC, that is, HEVC and AVC can fully reuse the on-chip SRAM to ensure that the adjacent context data of a CTU is pre-fetched in one clock cycle.

[0215] 4. Easy to expand: If you need to support a larger resolution or a larger CTU size in the future, you only need to design the width WSRAM and depth DSRAM of the on-chip SRAM according to the following formula:

[0216]

[0217]

[0218] in, Indicates the maximum size of a CTU. Indicates the minimum size of CU, Indicates the width of each adjacent context data, Indicates the maximum width supported for video decoding. It can be seen that as long as the existing SRAM is reasonably expanded in width or depth, it can support larger resolutions or larger CTU sizes in the future.

[0219] The first on-chip storage unit structural parameter includes a width parameter and a depth parameter; the method for determining the structural parameter of the first on-chip storage unit includes: the server obtains the maximum size of the coding tree unit of the VVC, the minimum size of the coding unit and the width of each adjacent context variable. The adjacent context variables are read and written in units of CTU each time, the maximum size of the CTU in the VVC is 128, the minimum size of the CU is 4, and the width of each adjacent context variable is 16 bits in total. Therefore, at most 32 (128 / 4) adjacent context variables need to be read and written each time, and the width of each adjacent context variable is 16 bits, so the total is 512 bits (16 bits * 32). In this way, enough data can be stored or read in each clock cycle, thereby improving the data reading speed. Taking 8K resolution video as an example, the size of CTU is 128, the minimum size of CU is 4, and there are at most 64 CTUs in one row, that is, the on-chip SRAM only needs to store the adjacent context variables of 64 CTUs at most, and each CTU requires at most one on-chip SRAM row, so a total of 64 on-chip SRAM rows are required. That is, the cache utilization rate of the first on-chip storage unit SRAM reaches 100%, thereby improving the cache utilization rate.

[0220] By optimizing the context model initialization process, improving the structure of on-chip SRAM and efficient reading and writing methods, the context model initialization time is reduced, the on-chip SRAM utilization rate is improved, and it is compatible with early video standards (such as HEVC and AVC) to reduce hardware costs. The on-chip storage unit in the present invention is also scalable to support future video standards.

[0221] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0222] Based on the same inventive concept, the embodiment of the present application also provides an entropy decoding processing device for implementing the entropy decoding processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more entropy decoding processing device embodiments provided below can refer to the limitations on the entropy decoding processing method above, and will not be repeated here.

[0223] In an exemplary embodiment, Fig.12 As shown, an entropy decoding processing device is provided, including: an initialization module 1201, a first parsing module 1202 and a first storage module 1203, wherein:

[0224] The initialization module 1201 is used to initialize the current context model for the coding slice within each frame, and determine the context variables in the current context model if the current syntax element is a first type syntax element; the first type syntax element includes the first syntax element of the first coding tree unit of the slice, the slice or the coding tree unit row; wherein, the slice refers to splitting a frame of image into strips determined by a number of coding tree units; the slice refers to splitting a frame of image into rectangles determined by a number of coding tree units.

[0225] The first parsing module 1202 is used to parse the current grammatical element through the initialized current context model.

[0226] The first storage module 1203 is configured to store each context variable in the current context model until the current syntax element is a syntax element of the second type.

[0227] In an exemplary embodiment, the first type of syntax element includes the first syntax element of a strip or the first syntax element of a slice; the current context model includes a first context initial variable; the initialization module 1201 is further used to: if the current syntax element is the first syntax element of a strip or the first syntax element of a slice, initialize the first context initial variable to obtain the context variable of the current context model.

[0228] In an exemplary embodiment, the first type of syntax element is also included in the first coding tree unit of the coding tree unit row, and the coding mark of the syntax element is the first mark, and the first mark is used to indicate the coding mark of the open syntax element; the current context model also includes a second context initial variable; the initialization module 1201 is also used to: if the current syntax element is in the first coding tree unit of the coding tree unit row, and the syntax element coding mark is the first mark, then determine whether there is an adjacent unit for the current coding tree unit corresponding to the current syntax element; if so, update the second context initial variable through the adjacent unit, and use the updated second context initial variable as the context variable of the current context model.

[0229] In an exemplary embodiment, the current coding tree unit includes a plurality of coding units; and the apparatus further includes:

[0230] The second parsing module is used to parse the syntax elements of the current coding unit, obtain the adjacent context variables of the first adjacent unit of the current coding unit, and the adjacent context variables of the second adjacent unit; wherein the first adjacent unit and the second adjacent unit are located in the same stripe or slice; the current coding unit is determined based on the coding unit.

[0231] The second storage module is used to store the adjacent context variable of the first adjacent unit in the register, and store the adjacent context variable of the second adjacent unit in the first on-chip storage unit.

[0232] In an exemplary embodiment, the second storage module is used to: obtain the horizontal coordinates of the current coding unit and the width of the current coding unit; determine the row number and the in-row offset of the adjacent context variable of the second adjacent unit in the first on-chip storage unit according to the horizontal coordinates; determine the number of times required to store the adjacent context variable according to the width of the current coding unit; and write the adjacent context variable into the target row and target bit of the corresponding first on-chip storage unit according to the row number, the in-row offset and the number of times.

[0233] In an exemplary embodiment, the device also includes: a first structure parameter determination module, used to obtain the maximum size of the coding tree unit of the target video coding protocol, the minimum size of the coding unit and the width of each adjacent context variable; based on the maximum size of the coding tree unit, the minimum size of the coding unit and the width of each adjacent context variable, determine the width parameter of the first on-chip storage unit; wherein the width parameter indicates the total number of bits of each row of the on-chip storage unit; based on the width parameter of the first on-chip storage unit and the maximum width supported for video decoding, determine the depth parameter of the first on-chip storage unit; wherein the depth parameter indicates the total number of rows of the on-chip storage unit.

[0234] In an exemplary embodiment, the second type of syntax element includes the last syntax element of the coding tree unit, and the coding tree unit is the first coding tree unit of the coding tree unit row, and the coding mark of the syntax element is a first mark, and the first mark is used to indicate the coding mark of the open syntax element; the first storage module is also used until the current syntax element is the last syntax element of the coding tree unit, and the coding tree unit is the first coding tree unit of the coding tree unit row, and the coding mark of the syntax element is the first mark; store the context variables in each current context model in the second on-chip storage unit.

[0235] In an exemplary embodiment, a first storage module is used to obtain the width of a second on-chip storage unit and the width of a context variable; based on the width of the second on-chip storage unit and the width of the context variable, determine the number of context models stored in each row of the second on-chip storage unit; for each context model, determine the row number and the row offset of the context variable of the context model in the second on-chip storage unit; and write the context variable into the target row and target bit of the corresponding second on-chip storage unit according to the row number and the row offset.

[0236] In an exemplary embodiment, the device also includes: a second structural parameter determination module, which is used to obtain the number of target context models of the target video coding protocol, the width of the context variables of the target context model; and the width of the context variables of the candidate context model of the candidate video coding protocol; based on the width of the context variables of the target context model and the width of the context variables of the candidate context model, determine the width parameter of the second on-chip storage unit; wherein the width parameter indicates the total number of bits in each row of the on-chip storage unit; the value of the width parameter includes the width of the context variables of the target context model and the positive integer multiples of the width of the context variables of the candidate context model; based on the number of target context models, the width of the context variables of each target context model, and the width parameter of the second on-chip storage unit, determine the depth parameter of the second on-chip storage unit; wherein the depth parameter indicates the total number of rows of the on-chip storage unit.

[0237] Each module in the above entropy decoding processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0238] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.13As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data in the entropy decoding process. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an entropy decoding processing method is implemented.

[0239] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0240] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0241] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0242] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0243] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0244] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not every possible combination of the technical features in the above embodiments is described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0245] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An entropy decoding processing method, characterized in that: The method comprises: For a context model of a coding slice within each frame, if a current syntax element is a first type syntax element, the current context model is initialized to determine a context variable in the current context model; the first type syntax element includes a first syntax element of a first coding tree unit of a slice, a slice, or a coding tree unit row; wherein the slice refers to splitting a frame of an image into a strip shape determined by a plurality of coding tree units; the slice refers to splitting a frame of an image into a rectangular shape determined by a plurality of coding tree units; Parsing the current grammatical element by using the initialized current context model; Until the current syntax element is a syntax element of the second type, the context variables in each current context model are stored.

2. The method according to claim 1, characterized in that The first type of syntax element includes the first syntax element of a slice or the first syntax element of a slice; the current context model includes a first context initial variable; If the current syntax element is a syntax element of the first type, initializing the current context model to obtain context variables in the current context model includes: If the current syntax element is the first syntax element of a slice or the first syntax element of a slice, the first context initial variables are initialized to obtain context variables of the current context model.

3. The method according to claim 1, characterized in that: The first type of syntax element is also included in the first coding tree unit of the coding tree unit row, and the coding flag of the syntax element is a first flag, and the first flag is used to indicate turning on the coding flag of the syntax element; The current context model also includes a second context initial variable; If the current syntax element is a syntax element of the first type, initializing the current context model to obtain context variables in the current context model includes: If the current syntax element is in the first coding tree unit of the coding tree unit row, and the syntax element coding flag is the first flag, determining whether the current coding tree unit corresponding to the current syntax element has an adjacent unit; If so, the second context initial variable is updated through the adjacent unit, and the updated second context initial variable is used as the context variable of the current context model.

4. The method according to claim 3, characterized in that The current coding tree unit includes a plurality of coding units; the method further includes: Parsing syntax elements of a current coding unit, obtaining a neighboring context variable of a first neighboring unit of the current coding unit, and a neighboring context variable of a second neighboring unit; wherein the first neighboring unit and the second neighboring unit are located in the same slice or slice; and the current coding unit is determined based on the coding unit; The neighboring context variable of the first neighboring cell is stored in a register, and the neighboring context variable of the second neighboring cell is stored in a first on-chip storage cell.

5. The method according to claim 4, characterized in that The storing the adjacent context variable of the second adjacent unit in the first on-chip storage unit includes: Obtaining the horizontal coordinate of the current coding unit and the width of the current coding unit; Determine, according to the horizontal coordinate, a row number and an in-row offset of a neighboring context variable of the second neighboring unit in the first on-chip storage unit; Determining, according to the width of the current coding unit, the number of times required to store adjacent context variables; The adjacent context variable is written into a target row and a target bit of a corresponding first on-chip storage unit according to the row number, the in-row offset and the number of times.

6. The method according to any one of claims 4 or 5, characterized in that: The structural parameters include width parameter and depth parameter; The method for determining the structural parameters of the first on-chip storage unit comprises: Obtaining a maximum size of a coding tree unit, a minimum size of a coding unit, and a width of each adjacent context variable of a target video coding protocol; Determine a width parameter of the first on-chip storage unit based on the maximum size of the coding tree unit, the minimum size of the coding unit and the width of each of the adjacent context variables; wherein the width parameter indicates the total number of bits of each row of the on-chip storage unit; Based on a width parameter of the first on-chip storage unit and a maximum width supported for video decoding, a depth parameter of the first on-chip storage unit is determined; wherein the depth parameter indicates a total number of rows of the on-chip storage unit.

7. The method according to claim 1, characterized in that The second-type syntax element includes the last syntax element of a coding tree unit, the coding tree unit is the first coding tree unit of a coding tree unit row, and the coding flag of the syntax element is a first flag, and the first flag is used to indicate the coding flag of turning on the syntax element; The step of storing the context variables in each of the current context models until the current grammar element is a grammar element of the second type includes: Until the current syntax element is the last syntax element of the coding tree unit, the coding tree unit is the first coding tree unit of the coding tree unit row, and the coding flag of the syntax element is the first flag; the context variables in each of the current context models are stored in the second on-chip storage unit.

8. The method according to claim 7, characterized in that The storing the context variables in the current context model in a second on-chip storage unit comprises: Obtaining a width of the second on-chip storage unit and a width of the context variable; Determining the number of context models stored in each row of the second on-chip storage unit based on the width of the second on-chip storage unit and the width of the context variable; For each of the context models, determining a row number and an in-row offset of a context variable of the context model in the second on-chip storage unit; The context variable is written into a corresponding target row and target bit of the second on-chip storage unit according to the row number and the intra-row offset.

9. The method according to any one of claims 7 or 8, characterized in that: The structural parameters include width parameter and depth parameter; The method for determining the structural parameters of the second on-chip storage unit comprises: Obtaining the number of target context models of the target video coding protocol, the width of context variables of the target context model; and the width of context variables of candidate context models of candidate video coding protocols; Determine a width parameter of the second on-chip storage unit based on the width of the context variables of the target context model and the width of the context variables of the candidate context model; wherein the width parameter indicates the total number of bits of each row of the on-chip storage unit; the value of the width parameter includes the width of the context variables of the target context model and the positive integer multiples of the width of the context variables of the candidate context model; Based on the number of target context models, the width of the context variables of each target context model, and the width parameter of the second on-chip storage unit, determine the depth parameter of the second on-chip storage unit; wherein the depth parameter indicates the total number of rows of on-chip storage units.

10. An entropy decoding processing device, characterized in that: The device comprises: an initialization module, for a context model of a coding slice within each frame, if a current syntax element is a first type syntax element, initializing the current context model, and determining context variables in the current context model; the first type syntax element includes a first syntax element of a first coding tree unit of a slice, a slice, or a coding tree unit row; wherein the slice refers to splitting a frame of image into a strip shape determined by a plurality of coding tree units; the slice refers to splitting a frame of image into a rectangular shape determined by a plurality of coding tree units; A first parsing module, configured to parse the current grammatical element by using the initialized current context model; The first storage module is configured to store the context variables in each of the current context models until the current syntax element is a syntax element of the second type.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images