Device

Abstract

To improve coding efficiency.SOLUTION: A device includes a recording medium storing instructions for causing a processor to perform an operation of generating image data corresponding to a moving image bitstream, the image data includes a plurality of pictures of a video bitstream including a first picture including a first slice and a second slice, a slice header including a first slice header associated with the first slice of the first picture and a second slice header associated with the second slice of the first picture, the first slice and the second slice include blocks of entropy encoded samples, and the second slice header shares slice characteristics with the first slice header, and the image data of the first slice and the second slice include transform coefficients.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] Embodiments of the present invention generally relate to video encoding.

Background Art

[0002] Latest video encoding methods and standards such as H.264 / MPEG-4 AVC (H.264 / AVC) provide higher encoding efficiency compared to conventional methods and standards at the expense of increased complexity. The increasing requirements regarding image quality and resolution in video encoding methods and standards are also factors contributing to the increased complexity. In a decoder supporting parallel decoding processing, the decoding speed is improved and the required memory amount is reduced. In addition, the progress of multi-core processors makes encoders and decoders supporting parallel decoding processing desirable.

[0003] H.264 / MPEG-4 AVC (Non-Patent Document 1) is a specification for a video codec (the entire specification is incorporated herein by reference), and in this specification, macroblock prediction for reducing temporal and spatial redundancy in a video sequence and subsequent residual coding are used for efficient compression.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the H.264 / AVC decoder, entropy decoding needs to be performed prior to all processing in the decoder. Therefore, entropy decoding is a potential bottleneck in the decoding process.

[0006] The present invention has been made in view of the above problems, and its main object is to solve the above problems.

Means for Solving the Problems

[0007] One aspect of the present invention is an apparatus comprising a non-transitory computer-readable recording medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform an operation of generating image data corresponding to a moving image bitstream, the image data including a plurality of pictures in the moving image bitstream including a first picture including a first slice and a second slice, a plurality of slice headers including a first slice header associated with the first slice of the first picture and a second slice header associated with the second slice of the first picture, the first slice and the second slice including a plurality of blocks of a plurality of entropy-encoded samples, the second slice header being different from the first slice header, sharing some slice characteristics with the first slice header, having a size smaller than the size of the first slice header, a value of a flag of the first slice header indicating that the first slice header is a header of an entropy slice that is the start of a reconstructed slice, a value of a flag of the second slice header indicating that the second slice header is a header of an entropy slice that is not the start of a reconstructed slice, the value of the flag of the first slice header being set to 0, and the image data of the first slice and the second slice including transform coefficients.

[0008] The above-described or other objects, features, and advantages of the present invention will be more readily understood by considering the following detailed description of the present invention together with the accompanying drawings.

Effect of the Invention

[0009] In H.264 / AVC, entropy decoding has become a potential bottleneck in the decoding process. The configuration in one embodiment of the present invention enables parallel entropy decoding processes necessary for reconstructing an image. Therefore, according to the method of the present invention, the above problem can be solved.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

[0011] Embodiments of the present invention will be best understood by reference to the drawings. In the drawings, like parts will be given like reference numerals. Also, the above drawings are explicitly incorporated as part of the detailed description.

[0012] As generally described and illustrated in the drawings of this specification, it will be readily understood that the elements of the present invention can be arranged and designed in various configurations. Accordingly, the more detailed description of the embodiments of the method, apparatus, and system according to the present invention described below does not limit the scope of the present invention, but merely represents the preferred embodiments at the present stage.

[0013] Each element in the embodiments of the present invention may be realized by hardware, firmware, and / or software. The embodiments exemplarily disclosed in this specification merely describe one of these forms, and it is understood that those skilled in the art can realize each element in any of these forms within the scope of the present invention.

[0014] Any coder / decoder (codec) using entropy encoding / decoding is included in the embodiments of the present invention. Exemplary embodiments of the present invention are described in the context of an H.264 / AVC encoder and an H.264 / AVC decoder. This is for the purpose of explaining the present invention and does not limit the present invention.

[0015] Latest video coding methods and standards such as H.264 / MPEG-4 AVC (H.264 / AVC) provide higher coding efficiency compared to conventional methods and standards at the expense of increased complexity. The increasing requirements for image quality and resolution in video coding methods and standards are also factors contributing to the increased complexity. In decoders that support parallel decoding processing, the decoding speed is improved and the required memory amount is reduced. In addition, the progress of multi-core processors makes encoders and decoders that support parallel decoding processing desirable.

[0016] H.264 / AVC and many other video coding standards and methods are based on a block-based hybrid video coding approach. In them, the source-coding algorithm is a hybrid of (a) inter-picture (also called inter-frame) prediction, (b) intra-picture (also called intra-frame) prediction, and (c) transform coding of the prediction-residual. Inter-frame prediction utilizes temporal redundancy, and intra-frame and transform coding of the prediction-residual utilize spatial redundancy.

[0017] FIG. 1 is a block diagram of an exemplary H.264 / AVC video encoder 2. An input picture 4, which can also be regarded as an input frame, exists as an object to be encoded. A predicted signal 6 and a residual signal 8 are generated. Here, the predicted signal 6 is based on either inter-frame prediction 10 or intra-frame prediction 12. The inter-frame prediction 10 is determined by a motion compensation (unit) 14 that uses (1) an accumulated reference picture 16 (also referred to as a reference frame), and (2) motion information 19 determined by a motion detection 18 process between the input picture (input frame) 4 and the reference frame (reference picture) 16. The intra-frame prediction 12 is determined by an intra-frame prediction (unit) 20 using a decoded signal 22. The residual signal 8 is determined by subtracting the prediction (predicted picture) 6 from the input picture 4. The residual signal 8 is transformed, scaled, and quantized 24, thereby generating quantized transform coefficients 26. The decoded signal 22 is generated by adding the predicted signal 6 to a signal 28 generated by inverse-transforming, scaling, and inverse-quantizing 30 the quantized transform coefficients 26. The motion information 19 and the quantized transform coefficients 26 are entropy encoded 32 and written into a compressed video bitstream 34. An output picture area 38 (e.g., a part of a reference frame) is generated in the encoder 2 by applying a filter 36 to a reconstructed pre-filter signal (decoded signal) 22.

[0018] FIG. 2 is a block diagram of an exemplary 264 / AVC video decoder 50. An input signal 52, which can also be regarded as a bitstream, exists as the object of decoding. The received symbols are entropy decoded 54, thereby generating (1) motion information 56 and (2) quantized and scaled transform coefficients 58. The motion information 56 is combined by motion compensation 60 with a part of a reference frame 84 in a frame memory 64 to generate an inter-frame prediction 68. The quantized and scaled transform coefficients 58 are inverse quantized, (inversely) scaled, and inverse transformed 62, thereby generating a decoded residual signal 70. The residual signal 70 is added to a prediction signal 78. Here, the prediction signal 78 is either an inter-frame prediction signal 68 or an intra-frame prediction signal 76. The intra-frame prediction signal 76 is predicted by intra-frame prediction 74 from information 72 already decoded in the current frame. The added signal 72 is filtered by a deblocking filter 80, and the filtered signal 82 is written into the frame memory 64.

[0019] In H.264 / AVC, an input picture is divided into fixed-size macroblocks, each of which covers a rectangular image area of 16×16 samples for the luminance component and 8×8 samples for each of the two chrominance components. The decoding process in the H.264 / AVC standard is specified to process macroblocks as units. The entropy decoder 54 syntax-analyzes the syntax elements of the compressed video bitstream 52 and demultiplexes them. H.264 / AVC is specified to use two different methods as entropy decoding. One is a low-complexity technique based on using a set of variable-length codes that adaptively switch contexts, called CAVLC, and the other is an algorithm that requires more computational effort, called CABAC, which performs context-based adaptive binary arithmetic coding. In both entropy decoding methods, the decoding process of the current symbol depends on the symbols correctly decoded previously and the context model updated adaptively. In addition, different data information such as, for example, prediction data information, residual data information, and different color planes are multiplexed together. Demultiplexing does not end until each element is entropy-decoded.

[0020] After entropy decoding, a macroblock is reconstructed by obtaining (1) a residual signal that has undergone inverse quantization and inverse transformation, and (2) a prediction signal that is either an intra-frame prediction signal or an inter-frame prediction signal. Block distortion is reduced by applying a deblocking filter to each decoded macroblock. No processing starts until the input signal is entropy-decoded. Therefore, entropy decoding is a potential bottleneck in the decoding process.

[0021] Similarly, in a codec that permits different prediction mechanisms such as inter-layer prediction in H.264 / AVC and inter-layer prediction in other scalable codecs, entropy decoding needs to be performed prior to all processing in the decoder. Therefore, entropy decoding is a potential bottleneck in the decoding process.

[0022] In H.264 / AVC, an input picture including a plurality of macroblocks is divided into one or more slices. Assuming that the reference pictures used in the encoder and decoder are the same, the sample values in the area of the picture indicated by one slice can be correctly decoded without using the data of other slices. Therefore, entropy decoding for one slice and macroblock reconstruction do not depend on other slices. In particular, at the start of each slice, the entropy coding state is reset. The data of other slices are marked as unavailable when defining the neighborhood availability for both entropy decoding and reconstruction. In H.264 / AVC, slices are entropy decoded and reconstructed in parallel. Intra (intra-picture) prediction and motion vector prediction across slice boundaries are prohibited. The deblocking filter can use information across slice boundaries.

[0023] FIG. 3 shows an exemplary video image (video picture) 90. The video image 90 includes 11 macroblocks in the horizontal direction and 9 macroblocks in the vertical direction (the nine exemplary macroblocks are numbered 91 to 99). FIG. 3 shows three exemplary slices: The first slice, denoted as “Slice #0” 100, the second slice, denoted as “Slice #1” 101, and the third slice, denoted as “Slice #2” 102, are shown. The H.264 / AVC decoder can decode and reconstruct three slices 100, 101, 102 in parallel. At the beginning of the decoding / reconstruction process of each slice, the context model is initialized or reset, and macroblocks in other slices are marked as unavailable for both entropy decoding and macroblock reconstruction. Thus, for the macroblocks in “Slice #1”, for example, the macroblock numbered 93, the macroblocks in “Slice #0” (for example, the macroblocks numbered 91 and 92) are not used for context model selection and reconstruction. On the other hand, for the macroblocks in “Slice #1”, for example, the macroblock numbered 95, the other macroblocks in “Slice #1” (for example, the macroblocks numbered 93 and 94) are used for context model selection and reconstruction. Therefore, entropy decoding and macroblock reconstruction need to be performed serially within a slice. Unless the slice is defined using flexible macroblock ordering (FMO), the macroblocks within a slice are processed in raster scan order.

[0024] Flexible macroblock ordering defines slice groups that change the manner of slicing a picture into slices. The macroblocks within a slice group are defined by a macroblock-to-slice-group map. Here, the macroblock-to-slice-group map is indicated by the contents of the picture parameter set and additional information in the slice header. The macroblock-to-slice-group map is composed of slice-group identification numbers for each macroblock in the picture. The slice-group identification number specifies to which slice each macroblock belongs. Each slice group can be divided into one or more slices. Here, a slice consists of a series of macroblocks within the same slice group that are processed in raster scan order in a set of macroblocks of a slice group. Entropy decoding and macroblock reconstruction need to be performed sequentially within a slice.

[0025] Figure 4 shows an exemplary arrangement of macroblocks into three slice groups: a first slice group denoted as "slice group #0" 103, a second slice group denoted as "slice group #1" 104, and a third slice group denoted as "slice group #2" 105. These slice groups 103, 104, 105 are each associated with two foreground regions and one background region in picture 90.

[0026] Some embodiments of the present invention involve splitting a picture into one or more reconstruction slices. Here, assuming that the reference pictures used in the encoder and decoder are the same, a reconstruction slice is self - contained in that the sample values in the area represented by the reconstruction slice on the picture are correctly reconstructed without using data from other reconstruction slices. All reconstructed macroblocks in a reconstruction slice are available in the definition of the neighborhood for reconstruction.

[0027] Some embodiments of the present invention involve splitting a reconstruction slice into more than one entropy slice. Here, an entropy slice is self - contained in that the symbol values in the area represented by the entropy slice on the picture are correctly entropy - decoded without using data from other entropy slices. In some embodiments of the present invention, at the start of decoding each entropy slice, the entropy - coding state is reset. In some embodiments of the present invention, the data of other entropy slices are marked as unavailable when defining the availability of the neighborhood for entropy decoding. In some embodiments of the present invention, macroblocks in other entropy slices are not used in the selection of the context model for the current block. In some embodiments of the present invention, the context model is updated only within an entropy slice. In these embodiments of the present invention, each entropy decoder used for one entropy slice maintains its own set for the context model.

[0028] Some embodiments of the present invention involve CABAC encoding / decoding. The CABAC encoding process includes the following steps.

[0029] · Binarization: Non-binary-valued symbols (e.g., transform coefficients, motion vectors, or other coded data), also called bin strings, are converted into a binary code.

[0030] Following binarization, for each bin that can also be regarded as a bit of the binarized symbol, context model selection is performed.

[0031] · Context model selection: A context model is a probability model for one or more bins of a binarized symbol. The context model includes, for each bin, the probabilities of whether the bin is a "1" or a "0". The selection of the model depends on the statistics of the most recently coded data symbols and is usually performed for the available model selection options based on the left and upper neighboring symbols if they are available.

[0032] · Binary arithmetic coding: An arithmetic coder encodes each bin based on a recursive interval subdivision according to the selected probability model.

[0033] · Probability update: The selected context model is updated based on the actual coded value.

[0034] In some embodiments of the present invention including CABAC encoding / decoding, at the start of decoding of an entropy slice, all context models are initialized or reset to a predetermined model.

[0035] Some embodiments of the present invention are understood in relation to FIG. 5. FIG. 5 shows an exemplary video frame 110. The video frame 110 includes 11 macroblocks in the horizontal direction and 9 macroblocks in the vertical direction (the 9 exemplary macroblocks are numbered 115 to 123). FIG. 5 shows three exemplary reconstruction slices: a first reconstruction slice shown as “R_slice#0” 111, a second reconstruction slice shown as “R_slice#1” 112, and a third reconstruction slice shown as “R_slice#2” 113. FIG. 5 further shows the division of the second reconstruction slice “R_slice#1” 112 into three entropy slices: a first entropy slice 112-1 represented by cross-hatching and shown as “E_slice#0”, a second entropy slice 112-2 represented by vertical hatching and shown as “E_slice#1”, and a third entropy slice 112-3 represented by diagonal hatching and shown as “E_slice#2”. Each entropy slice 112-1, 112-2, 112-3 is entropy decoded in parallel. Here, the first entropy slice shown as “E_slice#0” and the second entropy slice shown as “E_slice#1” are also referred to as the first part and the second part of the bitstream.

[0036] In some embodiments of the present invention, only the data from the macroblocks within an entropy slice is available for the selection of the context model during the entropy decoding of the entropy slice. All other macroblocks are marked as unavailable. In this exemplary split, the macroblocks numbered 117 and 118 are unavailable for the selection of the context model when decoding the symbols corresponding to the region of the macroblock numbered 119. This is because the macroblocks numbered 117 and 118 are located outside the entropy slice that includes the macroblock 119. However, these macroblocks 117, 118 are available when the macroblock 119 is being reconstructed.

[0037] In some embodiments of the present invention, the encoder determines whether to split a reconstruction slice into entropy slices and includes that determination as a signal in the bitstream. In some embodiments of the present invention, that signal includes an entropy - slice frag (the entropy - slice frag in the first entropy slice is also called the first flag). In some embodiments of the present invention, the entropy - slice frag is represented as "entropy_slice_frag".

[0038] Some embodiments of the decoders of the present invention are described in relation to FIG. 6. In these embodiments, an entropy slice flag is examined (S130), and if the entropy slice flag indicates that there is no entropy slice associated with the picture or the reconstructed slice (NO in step S130), the header is parsed as a regular slice header (S134). The state of the entropy decoder is reset (S136), and neighbor information for entropy decoding and reconstruction is defined (S138). Then, the slice data is entropy decoded (S140), and the slice is reconstructed (S142). If the entropy slice flag indicates that there is an entropy slice associated with the picture (YES in step S130), the header is parsed as an entropy-slice header (S148). The state of the entropy decoder is reset (S150), neighbor information for entropy decoding is defined (S152), and the data of the entropy slice is entropy decoded (S154). Then, neighbor information for reconstruction is defined (S156), and the slice is reconstructed (S142). After the reconstruction of the slice in step S142, the next slice or picture is analyzed.

[0039] Some other embodiments of the decoder of the present invention are described in relation to FIG. 7. In these embodiments, the decoder is capable of performing parallel decoding and defines its own degree of parallelism. For example, consider a decoder that can decode N entropy slices in parallel. The decoder identifies N entropy slices (S170). In some embodiments of the present invention, when fewer than N entropy slices are available in the current picture or reconstructed slice, the decoder decodes the entropy slices from the next picture or reconstructed slice if they are available. In other embodiments, the decoder waits until the current picture or reconstructed slice is fully processed before decoding a part of the next picture or reconstructed slice. In step S170, after identifying up to N entropy slices, each of the identified entropy slices is independently entropy decoded. The first entropy slice is decoded (S172 - S176). Decoding of the first entropy slice includes resetting the state of the decoder (S172). In some embodiments including CABAC entropy decoding, the state of CABAC is reset. Neighborhood information for entropy decoding of the first entropy slice is defined (S174), and the data of the first entropy slice is decoded (S176). These steps are performed for each of the up to N entropy slices (S178 - S182 for the Nth entropy slice). In some embodiments of the present invention, the decoder reconstructs the entropy slices when all the entropy slices have been entropy decoded (S184). In other embodiments of the present invention, the decoder starts the reconstruction of step S184 after one or more entropy slices have been decoded.

[0040] In some embodiments of the present invention, when there are more than N entropy slices, the decode thread starts decoding the next entropy slice as soon as the entropy decoding of an entropy slice is completed. Therefore, when a thread finishes decoding an entropy slice with low complexity, the thread starts decoding a further entropy slice without waiting for other threads to complete decoding.

[0041] In some embodiments of the present invention, including existing standards or methods, entropy slices share many of the slice attributes of regular slices according to such standards or methods. Therefore, entropy slices require a small header. In some embodiments of the present invention, the entropy slice header permits the decoder to identify the start of an entropy slice and start entropy decoding. In some embodiments, at the start of a picture or a reconstruction slice, the entropy slice header is a regular header or a reconstruction slice header.

[0042] In some embodiments of the present invention, including the H.264 / AVC codec, entropy slices are signaled by adding a new bit "entropy_slice_flag" to an existing slice header. Table 1 shows a list of the syntax of the entropy slice header according to an embodiment of the present invention. In Table 1, C indicates a Category, and the descriptors u(1), ue(v) indicate fixed length or variable length decoding methods.

[0043] "first_mb_in_slice" identifies the address of the first macroblock in an entropy slice associated with an entropy slice header. In some embodiments, an entropy slice contains a sequence of macroblocks.

[0044] "cabac_init_idc" identifies an index for determining an initialization table used in the initialization process of the context mode.

[0045]

Table 1

[0046] In some embodiments of the present invention, the entropy decoding of an entropy slice includes initializing a plurality of context models and updating the plurality of context models while performing the entropy decoding of the entropy slice.

[0047] In some embodiments of the present invention, a network abstraction layer (NAL) unit type different from that of a standard slice is assigned to an entropy slice. In these embodiments, a decoder can identify a standard slice and an entropy slice based on the NAL unit type. In these embodiments, a bit field "entropy_slice_flag" is not required.

[0048] In some embodiments of the present invention, the entropy slice is configured by changing data multiplexing. In some embodiments of the present invention, a group of symbols included in the entropy slice is multiplexed at the macroblock level. In another embodiment of the present invention, a group of symbols included in the entropy slice is multiplexed at the picture level. In yet another embodiment of the present invention, a group of symbols included in the entropy slice is multiplexed by data type. In still another embodiment of the present invention, a group of symbols included in the entropy slice is multiplexed by a combination of the above-mentioned ones.

[0049] Some embodiments of the method according to the present invention include: (1) encoding a video frame in a video sequence, (2) thereby generating a reconstructed slice, and (3) dividing the reconstructed slice into a plurality of entropy slices. Here, (1) encoding a video frame in a video sequence includes dividing a frame in the video sequence into at least one reconstructed slice.

[0050] Some embodiments of the present invention, which include constructing entropy slices based on multiplexing at the picture level, are understood in relation to FIGS. 8 and 9. In some embodiments of the present invention shown in FIG. 8, prediction data 190 and residual data 192 are individually entropy encoded by a prediction encoder 194 and a residual encoder 196, and multiplexed at the picture level by a picture-level multiplexer 198. In some embodiments of the present invention, the prediction data 190 for a picture is associated with a first entropy slice, and the residual data 192 for the picture is associated with a second entropy slice. The encoded prediction data and the encoded entropy data are decoded in parallel. In some embodiments of the present invention, each partition including prediction data or residual data is divided into a plurality of entropy slices that are decoded in parallel.

[0051] In some embodiments of the present invention shown in FIG. 9, the residuals of each color plane, for example, the luma (Y) residual 200, and the two chroma (U and V) residuals 202, 204 are individually entropy encoded by the Y encoder 206, the U encoder 208, and the V encoder 210, and multiplexed at the picture level by the picture level multiplexer 212. In some embodiments of the present invention, the luma residual for picture 200 is associated with the first entropy slice, the first chroma (U) residual for picture 202 is associated with the second entropy slice, and the second chroma (V) residual for picture 204 is associated with the third entropy slice. The encoded residual data for the three color planes are decoded in parallel. In some embodiments of the present invention, each partition containing color plane residual data is divided into a plurality of entropy slices that are decoded in parallel. In some embodiments of the present invention, the luma residual 200 has a relatively large number of entropy slices compared to the chroma residuals 202, 204.

[0052] In some embodiments of the present invention, the bitstream of the compressed video is transcoded to include entropy slices, whereby parallel entropy decoding as included in the embodiments of the present invention described above is performed. Some embodiments of the present invention are described in relation to FIG. 10. An input bitstream without entropy slices is processed picture by picture according to FIG. 10. In these embodiments of the present invention, the pictures from the input bitstream are entropy decoded (S220). Encoded data such as mode data, motion information, residual information, and other data are obtained. Entropy slices are each composed from the data (S222). An entropy slice header corresponding to the entropy slice is written to a new bitstream (S224). The state of the encoder is reset and neighborhood information is defined (S226). The entropy slice is entropy encoded (S228) and written to the new bitstream. If there is picture data not consumed in the composed entropy slice (NO in step S230), other entropy slices are composed in step S222, and the processing of S224 to S230 is continued until all picture data are consumed in the composed entropy slice (YES in step S230), and then the next picture is processed.

[0053] 〔Supplementary Notes〕 Some embodiments of the present invention include a method, apparatus, and system for parallel entropy encoding and decoding of a video bitstream by dividing slice data into a plurality of entropy slices that are independently entropy encoded and decoded.

[0054] According to one aspect of the present application, a method for decoding a moving image bitstream is provided. The method includes a first entropy decoding step of generating a first portion of decoded data by entropy decoding a first portion of the moving image bitstream, which is a first portion associated with a moving image frame; a second entropy decoding step of generating a second portion of decoded data by entropy decoding a second portion of the moving image bitstream, which is a second portion associated with the moving image frame, independently of the entropy decoding of the first portion; and a reconstruction step of reconstructing a first portion of the moving image frame associated with the moving image bitstream using the first portion in the decoded data and the second portion in the decoded data.

[0055] According to another aspect of the present application, a method for decoding a moving image frame in a moving image sequence is provided. The method includes a receiving step of receiving a bitstream; a reconstruction slice identification step of identifying a reconstruction slice in the bitstream; an entropy slice identification step of identifying a plurality of entropy slices associated with the reconstruction slice in the bitstream; an entropy decoding step of generating a plurality of entropy decoded entropy slices by entropy decoding each of the plurality of entropy slices associated with the reconstruction slice; and a reconstruction step of reconstructing a part of the moving image frame associated with the reconstruction slice using the plurality of entropy decoded entropy slices.

[0056] According to another aspect of the present application, a method for encoding a moving image frame in a moving image sequence is provided. The method includes a reconstruction slice generation step of generating a first reconstruction slice by dividing a first frame in the moving image sequence into at least one reconstruction slice; and a division step of dividing the first reconstruction slice into a plurality of entropy slices.

[0057] According to another aspect of the present application, a method for generating a moving picture bitstream for parallel decoding is disclosed. The method includes a receiving step of receiving a first moving picture bitstream, an identifying step of identifying a reconstructed slice in the moving picture bitstream, an entropy decoding step of generating entropy decoded data associated with the reconstructed slice by entropy decoding a plurality of symbols from the reconstructed slice, a dividing step of dividing the entropy decoded data associated with the reconstructed slice into a plurality of entropy slices associated with the reconstructed slice, an entropy encoding step of generating a plurality of entropy encoded entropy slices by individually entropy encoding the entropy decoded data for each of the plurality of entropy slices, and a bitstream generating step of generating a second moving picture bitstream including the plurality of entropy encoded entropy slices.

[0058] In some embodiments of the present invention, the first portion and the second portion of an input compressed-video bitstream are entropy decoded separately. A block consisting of samples of a moving picture frame associated with the second portion of the input compressed-video bitstream is reconstructed using decoded data from the first portion and the second portion. Thus, the definition of neighborhood during reconstruction is not the same as the definition of neighborhood during entropy decoding.

[0059] In some embodiments of the present invention, the encoder divides the input data into a plurality of entropy slices. The encoder individually entropy-encodes the plurality of entropy slices. The encoder generates (forms) a bitstream including a plurality of entropy slice headers. Here, each of the plurality of entropy slice headers indicates the position of the data regarding the entropy slice in the bitstream. In some embodiments of the present invention, the decoder parses the bitstream received, which is the bitstream regarding the entropy slice headers, and the decoder entropy-decodes the plurality of entropy slices according to the level of parallelism defined by the decoder.

[0060] In some embodiments of the present invention, in order to generate (form) an entropy slice, the data is multiplexed at the picture level. In some embodiments, one or more entropy slices correspond to predicted data, and one or more entropy slices correspond to residual data. In other embodiments of the present invention, one or more entropy slices correspond to each of a plurality of color-planes.

[0061] In some embodiments of the present invention, the bitstream is transcoded so as to include entropy slices. In these embodiments, the received bitstream is entropy-decoded, a plurality of entropy slices are configured, each of the plurality of entropy slices is individually encoded, and written into the transcoded bitstream together with the corresponding entropy slice headers.

[0062] In H.264 / AVC, entropy decoding has become a potential bottleneck in the decoding process. The configuration in one embodiment of the present invention enables parallel execution of the entropy decoding process required to reconstruct an image. Therefore, according to the method, apparatus, and system of the present invention, the above problem can be solved.

[0063] Note that the decoding method according to the present invention is a decoding method for decoding a moving image bitstream. The method includes an entropy decoding step of generating decoded data by entropy decoding the moving image bitstream, a syntax analysis step of performing syntax analysis on the decoded data to identify first decoded data associated with a first part of the moving image bitstream and second decoded data associated with a second part, a first reconstruction step of reconstructing a first region related to an image in the moving image using the first decoded data, and a second reconstruction step of reconstructing a second region related to an image in the moving image using the second decoded data. The second reconstruction step may further use the first region reconstructed in the first reconstruction step to reconstruct the second region.

[0064] Also, the encoding method according to the present invention is an encoding method for encoding a moving image into a bitstream. The method may include a first configuration step of configuring first encoded data representing a first region related to an image in the moving image based on the first region, a second configuration step of configuring second encoded data representing a second region related to an image of the moving image based on the second region and the first region, a syntax addition step of adding a syntax expression that enables identification of the first encoded data and the second encoded data, and an entropy encoding step of generating the moving image bitstream by entropy encoding the first encoded data and the second encoded data that have undergone the syntax addition step.

[0065] The decoding method described above can also be expressed as follows. To solve the above problems, the decoding method according to the present invention is a decoding method for decoding moving image frames in a moving image sequence, comprising: a) receiving a bitstream; b) identifying a reconstructed slice in the bitstream; c) analyzing a flag in a slice header in the bitstream; d) identifying a first sliced slice and a second sliced slice indicated by a division of one region in the reconstructed slice; e) entropy-decoding the first sliced slice to generate first decoded data; f) entropy-decoding the second sliced slice to generate second decoded data; and g) reconstructing the reconstructed slice. The step of reconstructing the reconstructed slice includes: i) reconstructing a first part in a moving image frame associated with the first sliced slice using the first decoded data; and ii) reconstructing a second part in the moving image frame associated with the second sliced slice using the second decoded data and the first part. When the flag is 1, the slice header is a header of a sliced slice, and when the flag is 0, the slice header is a header of a standard slice. The header of the sliced slice shares the slice characteristics of the header of the standard slice, and the size of the header of the sliced slice is smaller than the size of the header of the standard slice.

[0066] The decoding method according to the present invention is a method for decoding an image of a moving image sequence using a decoding apparatus, including: a step of receiving a bitstream; a step of identifying a reconstruction slice indicating a region of the image in the bitstream; a step of analyzing a flag of a header of two divided slices in the bitstream; a step of identifying one of the two divided slices as a first divided slice of the reconstruction slice and identifying the other of the two divided slices as a second divided slice of the reconstruction slice based on the flag in the header; a step of entropy-decoding the first divided slice to generate first decoded data; and a step of entropy-decoding the second divided slice to generate second decoded data. The entropy decoding of the first divided slice does not depend on any other divided slice in the reconstruction slice. Further, the method may include a step of reconstructing the region of the image indicated by the reconstruction slice using the first decoded data and the second decoded data.

[0067] Furthermore, the encoding method according to the present invention is an encoding method for images of a moving image sequence using an encoding apparatus, including a step of entropy encoding a first divided slice to generate first entropy-encoded data, and a step of entropy encoding a second divided slice to generate second entropy-encoded data, wherein a reconstruction slice indicating a region of the image includes the first divided slice and the second divided slice. Further, the method includes a step of encoding a first divided slice header for the first divided slice, and a step of encoding a second divided slice header for the second divided slice. Both the first divided slice header and the second divided slice header include a flag indicating whether the corresponding divided slice is the first divided slice or the second divided slice. The second divided slice header shares slice characteristics of the first divided slice header. Further, the method may include a step of transmitting a bitstream including the first divided slice, the second divided slice, the first divided slice header, and the reconstruction slice including the second divided slice header.

[0068] The apparatus according to the present invention is an apparatus that is accessible by a decoding apparatus and includes a recording medium for non-temporarily recording data in a computer-readable manner. The computer-readable recording medium includes bitstream data including data indicating a reconstruction slice indicating a region in an image. The reconstruction slice includes a first divided slice and a second divided slice. The first divided slice includes a first divided slice header, and the second divided slice includes a second divided slice header. Each of the first divided slice header and the second divided slice header includes a flag indicating whether the corresponding divided slice is the first divided slice or the second divided slice. This may also be the case.

[0069] The apparatus according to the present invention is an apparatus including a non-transitory computer-readable recording medium. In the non-transitory computer-readable recording medium, when executed by one or more processors, instructions are stored for causing the one or more processors to perform a process of generating a plurality of reconstruction slices in encoded image data representing a moving image frame. Each reconstruction slice has a plurality of entropy slices, and each entropy slice has a plurality of macroblocks. The macroblocks included in one reconstruction slice are not included in two different entropy slices. The first entropy slice in the first reconstruction slice is entropy-coded independently of the remaining entropy slices in the first reconstruction slice. The first entropy slice represents a first portion of the moving image frame, and the second entropy slice in the first reconstruction slice represents a second portion of the moving image frame. The first portion in the moving image frame is used for encoding. Each entropy slice has a header, and each header has a flag. When the flag in the header is 0, the entropy slice is the first entropy slice of the reconstruction slice. When the flag in the header is 1, the entropy slice is not the first entropy slice of the reconstruction slice, and the header of the second entropy slice may be smaller than the header of the first entropy slice.

[0070] The terms and expressions used in the above specification are used only for the purpose of explanation and are not used for limitation. Also, those terms and expressions are not used to exclude the above-described characteristics or some of their equivalences. The scope of the present invention is defined by the claims and is limited only by the claims.

[0071] 〔Supplementary Note〕 The apparatus according to the present invention is an apparatus including a non-transitory computer-readable recording medium. In the non-transitory computer-readable recording medium, when executed by one or more processors, instructions are stored for causing the one or more processors to perform a process of generating a plurality of reconstruction slices in encoded image data representing moving image frames. Among the plurality of reconstruction slices, a certain reconstruction slice includes a plurality of entropy slices. The plurality of entropy slices include a plurality of macroblocks. The macroblocks included in one reconstruction slice are not included in two different entropy slices. The first entropy slice in the first reconstruction slice is entropy-coded independently of the remaining entropy slices in the first reconstruction slice. The first entropy slice represents a first portion of the moving image frame. The second entropy slice in the first reconstruction slice represents a second portion of the moving image frame. The first portion in the moving image frame is used for encoding. The first entropy slice and the second entropy slice each include a header. The header of the first entropy slice and the header of the second entropy slice each include a flag. When the flag in the header is a first predetermined value, each of the entropy slices is the first entropy slice of the first reconstruction slice. When the flag in the header is a second predetermined value, each of the entropy slices is not the first entropy slice of the first reconstruction slice. The header of the second entropy slice is smaller than the header of the first entropy slice. The plurality of entropy slices include residual data.

Industrial Applicability

[0072] The present invention can generally be suitably applied to encoding of moving images and decoding of moving images.

Claims

1. An apparatus comprising: a non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations to generate image data corresponding to a video bitstream; The image data includes: a plurality of pictures in the video bitstream, including a first picture including a first slice and a second slice; a plurality of slice headers including a first slice header associated with the first slice of the first picture and a second slice header associated with the second slice of the first picture; the first slice and the second slice include a plurality of blocks of entropy coded samples; The second slice header is different from the first slice header and shares some slice characteristics with the first slice header, and the size of the second slice header is smaller than the size of the first slice header; a value of the flag of the first slice header indicates that the first slice header is a header of an entropy slice that is the start of a reconstruction slice; a value of the flag of the second slice header indicates that the second slice header is a header of an entropy slice that does not start a reconstruction slice; The value of the flag in the first slice header is set to 0; The image data for the first slice and the second slice includes transform coefficients.

2. The apparatus of claim 1 , wherein a value of the flag in the second slice header is set to 1.

Images