Image processing apparatus and image processing method

Abstract

The present disclosure pertains to an image processing device and an image processing method that make it possible to reduce the implementation load required for image processing. A decoding unit decodes a bitstream including a parameter set in which a maximum number of parameter sets referenced by an adaptive loop filter is defined as a fixed value, and generates a decoded image. The filter unit refers to the parameter set decoded by the decoding unit and applies an adaptive loop filter to the decoded image generated by the decoding unit. The disclosed technology can be applied, for example, to an image processing system that performs image processing using an adaptive loop filter.

Description

[0001] This application is a divisional application of the patent application with international application number PCT / JP2020 / 023504 filed on June 16, 2020, application number 202080043358.6 entering the Chinese national phase, and titled “Image Processing Device and Image Processing Method”. Technical Field

[0002] The present disclosure relates to an image processing apparatus and an image processing method, and more particularly, to an image processing apparatus and an image processing method capable of reducing an installation load required for image processing. Background Art

[0003] Generally, as disclosed in Non-Patent Literature 1, parameter sets (up to 25 categories per set) of an adaptive loop filter (ALF) can be switched in units of coding tree units (CTUs).

[0004] For example, there are up to 22 types of parameter sets that can be switched, 16 types of fixed filters whose coefficients are determined by the standard, and 6 types of user-specified filters transmitted by the user using an adaptive parameter set (APS). In addition, APS is also used to transmit parameters.

[0005] Citation List

[0006] Non-patent literature

[0007] Non-patent document 1: Benjamin Bross, Jianle Chen, Shan Liu, Versatile Video Coding (Draft 5), JVET-N1001-v7 (version 7-date May 29, 2019) Summary of the invention

[0008] Problems to be solved by the present invention

[0009] However, the current APS specification has too many degrees of freedom (including ambiguity), so unless there is a restriction on how to use it, the decoder side needs to always decode the maximum number of APS IDs (=32). For this reason, there has been a concern in the past that the installation load would be high.

[0010] The present disclosure has been proposed in view of such circumstances, and can reduce the installation load required for image processing.

[0011] Solution to the problem

[0012] An image processing device of a first aspect of the present disclosure includes: a decoding unit configured to decode a bit stream including a parameter set to generate a decoded image, in which a maximum number of parameter sets used as references for an adaptive loop filter is defined as a fixed value; and a filter unit configured to apply an adaptive loop filter to the decoded image generated by the decoding unit by referring to the parameter set decoded by the decoding unit.

[0013] An image processing method according to a first aspect of the present disclosure includes: a decoding process of decoding a bit stream including a parameter set to generate a decoded image, in which a maximum number of parameter sets used as references for an adaptive loop filter is defined as a fixed value; and applying an adaptive loop filter to a decoded image generated by a decoding unit by referring to the parameter set decoded in the decoding process.

[0014] In a first aspect of the present disclosure, a bitstream including parameter sets is decoded to generate a decoded image, and the decoded parameter sets are referenced to apply an adaptive loop filter to the generated decoded image, wherein the maximum number of parameter sets referenced by the adaptive loop filter in the parameter set is defined as a fixed value.

[0015] An image processing device according to a second aspect of the present disclosure includes: a setting unit configured to set a parameter set in which a maximum number of parameter sets used as references for an adaptive loop filter is defined as a fixed value; and an encoding unit configured to encode an image to generate a bit stream including the parameter set set by the setting unit.

[0016] The image processing method of the second aspect of the present disclosure includes: a setting process of setting a parameter set, in which the maximum number of parameter sets used as references for an adaptive loop filter is defined as a fixed value; and an encoding process of encoding an image to generate a bit stream of the parameter set included in the setting process.

[0017] In a second aspect of the present disclosure, a parameter set in which a maximum number of parameter sets referred to by an adaptive loop filter is defined as a fixed value is set, and an image is encoded to generate a bitstream including the set parameter set. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a block diagram showing a configuration example of an embodiment of an image processing system to which the present technology is applied.

[0019] Figure 2 is a diagram showing a usage example of ALF_APS.

[0020] Figure 3is a diagram showing an example of an APS parameter type code.

[0021] Figure 4 is a diagram showing an example of the maximum number of APSs defined as bit-aligned according to levels.

[0022] Figure 5 is a diagram showing an example of the maximum number of APSs defined for each aps_params_type according to levels.

[0023] Figure 6 is a diagram for describing a detailed example of the maximum number of APSs defined for each aps_params_type according to levels.

[0024] Figure 7 is a block diagram showing a configuration example of an embodiment of a computer-based system to which the present technology is applied.

[0025] Figure 8 is a block diagram showing a configuration example of an embodiment of an image encoding device.

[0026] Fig. 9 is a flowchart for describing the encoding process.

[0027] Fig.10 is a block diagram showing a configuration example of an embodiment of an image decoding device.

[0028] Fig.11 is a flowchart for describing the decoding process.

[0029] Fig.12 is a block diagram showing a configuration example of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION

[0030] <Documents supporting technical content and technical terminology, etc.>

[0031] The scope disclosed herein is not limited to the contents of the embodiments, and the contents of the following references REF 1 to REF 6 known at the time of filing are also incorporated herein by reference. In other words, the contents described in references REF 1 to REF 6 are also the basis for judging the support requirements. In addition, for example, even when technical terms such as parsing, grammar, and semantics are not directly defined in the detailed description of the present invention, they are also within the scope of the present disclosure and should satisfy the support requirements of the claims.

[0032] REF 1 Recommendation ITU-T H.264 (04 / 2017) “Advanced video coding for generic audio visual services”, April 2017

[0033] REF 2 Recommendation ITU-T H.265 (02 / 2018) “High efficiency video coding”, February 2018

[0034] REF 3 Benjamin Bross, Jianle Chen, Shan Liu, Versatile Video Coding (Draft5), JVET-N1001-v7 (version 7 - date May 29, 2019)

[0035] REF 4: Jianle Chen, Yan Ye, Seung Hwan Kim, Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5), JVETN1002-v1

[0036] REF 5: Ye-Kui Wang, Hendry, Jianle Chen, Peng Yin, Taoran Lu, Fangjun Pu, Sean McCarthy, AHG17: Signalling of reshaper parameters in APS, JVET-N0117-v1 (version 1 - date March 12, 2019)

[0037] REF 6: Nael Ouedraogo, Eric Nassor, Jonathan Taquet, Gerald Kergourlay, Frederic Maze, [AHG17 / AHG12] On APS id for bitstreammerging for VVC, JVET-N0191-v1 (version 1 - date March 12, 2019)

[0038] <Term>

[0039] In this application, the following terms are defined as follows.

[0040] <block>

[0041] Unless otherwise specified, a "block" (not a block indicating a processing unit) used as a partial area or a processing unit of an image (picture) indicates an arbitrary partial area in a picture, and the size, shape, characteristics, etc. of the block are not limited. For example, a "block" includes any partial area (processing unit), such as a transform block (TB), a transform unit (TU), a prediction block (PB), a prediction unit (PU), a minimum coding unit (SCU), a coding unit (CU), a maximum coding unit (LCU), a coding tree block (CTB), a coding tree unit (CTU), a transform block, a subblock, a macroblock, a block, or a slice.

[0042] <Definition of block size>

[0043] In addition, when specifying the size of such a block, the block size can be specified not only directly but also indirectly. For example, the block size can be specified using identification information for identifying the size. In addition, for example, the block size can be specified by a ratio or difference with the size of a reference block (e.g., LCU, SCU, etc.). For example, when transmitting information for specifying the block size as a syntax element, etc., information for indirectly specifying the size as described above can be used as the information. By doing so, the amount of information can be reduced and the coding efficiency can be improved. In addition, the specification of the block size also includes the specification of the range of the block size (e.g., the specification of the range of allowable block sizes, etc.).

[0044] <Information / Processing Unit>

[0045] The data units for which various information is set and the data units for which various types of processing are targeted are each arbitrary and are not limited to the above examples. For example, this information or processing can be set for each transform unit (TU), transform block (TB), prediction unit (PU), prediction block (PB), coding unit (CU), maximum coding unit (LCU), sub-block, block, block, slice, picture, sequence or component, or this information or processing can be for the data in these data units. Of course, the data unit can be set for each information or processing, and it is not necessary to unify the data units for all information or processing. Note that the storage location of this information is arbitrary, and this information can be stored in the header, parameter set, etc. of the above-mentioned data unit. In addition, this information can be stored in multiple locations.

[0046] <Control Information>

[0047] Control information related to the present technology may also be transmitted from the encoding side to the decoding side. For example, control information (e.g., enabled_flag) for controlling whether the application of the above-mentioned present technology is allowed (or prohibited) may be transmitted. In addition, for example, control information indicating a target for applying the present technology (or a target for not applying the present technology) may be transmitted. For example, control information specifying the block size (upper and lower limits, or both), frame, component, layer, etc., for applying (or allowing or prohibiting) the present technology may be transmitted.

[0048] <Logo>

[0049] Note that in this specification, a "flag" is information for identifying multiple states, and includes not only information for identifying two states of true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that the "flag" can take can be, for example, 2 values ​​of 1 / 0, or 3 or more values. That is, the number of bits constituting the "flag" is arbitrary, and can be 1 bit or more bits. In addition, since it is assumed that the identification information (including the flag) includes not only the identification information in the bit stream, but also the difference information of the identification information that becomes the specific reference information in the bit stream, therefore, in this specification, the "flag" or "identification information" includes not only the information, but also the difference information that becomes the reference information.

[0050] <Associated metadata>

[0051] In addition, as long as the information is associated with the encoded data, various types of information (metadata, etc.) about the encoded data (bitstream) can be transmitted or recorded in any form. Here, for example, the term "associated" means making other data available (linkable) when processing one data. That is, data associated with each other can be combined into one data, or can be separate data. For example, information associated with encoded data (image) can be transmitted on a transmission path different from that of the encoded data (image). In addition, for example, information associated with encoded data (image) can also be recorded on a recording medium different from that of the encoded data (image) (or another recording area of ​​the same recording medium). Note that this "association" can be a part of the data, not the entire data. For example, an image and information corresponding to the image can be associated with each other in arbitrary units such as multiple frames, one frame, or a portion within a frame.

[0052] In addition, in this specification, terms such as "synthesize", "multiplex", "add", "integrate", "include", "store", "push", "put in", and "insert" refer to combining multiple things into one, for example, combining coded data and metadata into one data, and refer to a method of "associating" described above. In addition, in this specification, encoding includes not only all processes for converting an image into a bit stream, but also a part of the processes. For example, encoding includes not only processes including prediction processing, orthogonal transform, quantization, arithmetic coding, etc., but also processes collectively referred to as quantization and arithmetic coding, including prediction processing, quantization and arithmetic coding, etc. Similarly, decoding includes not only all processes for converting a bit stream into an image, but also a part of the processes. For example, decoding includes not only processes including inverse arithmetic decoding, inverse quantization, inverse orthogonal transform, prediction processing, etc., but also processes including inverse arithmetic decoding and inverse quantization, inverse arithmetic decoding, inverse quantization, prediction processing, etc.

[0053] Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the accompanying drawings.

[0054] <Configuration Example of Image Processing System>

[0055] Figure 1 is a block diagram showing a configuration example of an embodiment of an image processing system to which the present technology is applied.

[0056] like Figure 1 As shown in , the image processing system 11 includes an image encoding device 12 and an image decoding device 13. For example, in the image processing system 11, an image input to the image encoding device 12 is encoded, a bit stream obtained by encoding is transmitted to the image decoding device 13, and the image decoding device 13 outputs a decoded image decoded from the bit stream.

[0057] The image encoding device 12 has a setting unit 21 , an encoding unit 22 , a filter unit 23 , and a database 24 , and the image decoding device 13 has a decoding unit 25 , a filter unit 26 , and a database 27 .

[0058] The setting unit 21 sets a parameter set in which the maximum number of the number of parameter sets referred to by the adaptive loop filter is defined as a fixed value.

[0059] The encoding unit 22 encodes the image to generate a bit stream including the parameter set set by the setting unit 21. In this case, the encoding unit 22 encodes the image using the filtered image to which the adaptive loop filter is applied by the filter unit 23.

[0060] The filter unit 23 applies an adaptive loop filter to the locally encoded image when encoding is performed in the encoding unit 22 .

[0061] The database 24 stores various types of parameters such as filter coefficients required when the encoding unit 22 encodes an image.

[0062] The decoding unit 25 decodes a bit stream including a parameter set in which the maximum number of the number of parameter sets referred to by the adaptive loop filter is defined as a fixed value to generate a decoded image.

[0063] The filter unit 26 refers to the parameter set decoded by the decoding unit 25 and applies the adaptive loop filter to the decoded image generated by the decoding unit 25 .

[0064] The database 27 stores various types of parameters such as filter coefficients required when the decoding unit 25 decodes an image.

[0065] Here, in the adaptive loop filter applied in the filter unit 23 and the filter unit 26, up to 6 sets of filter coefficients can be stored in the APS, and one slice has the current filter coefficients and the filter coefficients in the time direction. This has 5 layers with a temporal layer (temporal_id) from 0 to 4, and the design is made assuming that a total of 6 sets of filter coefficients including 1 set of filter coefficients for each layer and the current 1 set of filter coefficients are stored in the APS.

[0066] On the other hand, when one slice is divided into a plurality of tile groups (tile_of_group), data of (6 groups×a plurality of tile groups) is transmitted, and the number of APS IDs increases in proportion to the number of tile groups. Therefore, not only the amount of processing for parsing the APS in the image decoding device 13 increases, but also the buffer size of the database 27 for storing parameters parsed from the APS needs to be increased.

[0067] Will refer to Figure 2 Describes an example of using ALF_APS.

[0068] like Figure 2 As shown in A in FIG. 1 , the adaptive loop filter can refer to up to 6 APSs in units of slices. On the other hand, when a slice is divided into a plurality of blocks, up to 6 APSs can be referred to in units of blocks.

[0069] Therefore, if Figure 2 As shown in B, when the slice is divided into 6 blocks, 6 APSs x 6 blocks = 36 APSs, and the number of APSs exceeds the maximum number of APS IDs (= 32).

[0070] In addition, the number of APS IDs applicable for use will differ depending on the use case (e.g., 8K vs. HD, 2D vs. spherical images, etc.). Therefore, when there is no use restriction, the image decoding device 13 needs to prepare a buffer size of the database 27 that can always extend the maximum number of APS IDs, and the installation load required for image processing increases.

[0071] Therefore, in the image processing system 11, in order to reduce the installation load required for image processing, the maximum number of parameter sets (eg, APSs) referenced by the adaptive loop filter (hereinafter, referred to as the maximum number of APSs) is set to a fixed value.

[0072] First, in the first specification example, the maximum number of APSs is defined for each aps_params_type that specifies the type of APS parameters.

[0073] For example, as aps_params_type, 0 and 1 have been used to specify the type of APS parameters, and 2-7 are reserved, such as Figure 3 Then, in case of using aps_params_type of ALF parameters (aps_params_type=0), the maximum number of APSs is defined as 6.

[0074] That is, regardless of the number of tile_of_groups, the maximum number of APS IDs that can be referenced by the adaptive loop filter is set to a fixed value. That is, the maximum number of APSs is defined as a fixed value that does not depend on the number of tiles. However, each of the APS IDs referenced by each tile_of_group may be different. Therefore, the maximum number of APS IDs of the adaptive loop filter is defined. In other words, in the first specification example, for each aps_params_type, the maximum number of APS IDs to be used is defined.

[0075] Next, in the second specification example, the maximum number of APSs is defined as bit-aligned depending on the level. By defining the maximum number of APSs for each level in this way, the APS specification can be made suitable for the use case.

[0076] Figure 4 An example of a second specification example is shown, which defines the maximum number of APSs for bit alignment depending on the level.

[0077] exist Figure 4In the example shown in , the maximum number of APSs is defined in bytes, so that the maximum number of APSs for levels from 1 to 4.1 is 8 (3 bits), the maximum number of APSs for levels from 5 to 5.2 is 16 (4 bits), and the maximum number of APSs for levels from 6 to 6.2 is 32 (5 bits).

[0078] In addition, a maximum number of APSs is defined for celestial images (for each projection format).

[0079] In addition, as a third specification example combining the first specification example and the second specification example, the maximum number of APSs may be defined by a multiple of the unit usage number set for each aps_params_type depending on the level. That is, the maximum number of APSs is defined for each aps_params_type, and the maximum number of APSs is a fixed value that is different for each level. Therefore, the maximum number of APS IDs that can be used for each aps_params_type may change depending on the level. That is, the maximum number of APS IDs is defined by "tool (function)" × "parallelization requirement (level)".

[0080] Figure 5 An example of the third definition example combining the first definition example and the second definition example is shown.

[0081] exist Figure 5 In the example shown in , the maximum number of APSs of levels 1 to 4.1 is defined as 1 times the number of units used set for each aps_params_type. In addition, the maximum number of APSs of levels 5 to 5.2 is defined as twice the number of units used set for each aps_params_type, and the maximum number of APSs of levels from 6 to 6.2 is defined as four times the number of units used set for each aps_params_type.

[0082] Specifically, Figure 6 As shown in , for the maximum number of APSs for HD images (level=4.x), the maximum number of APSs for 4K images (level=5.x) is defined as twice the unit usage number set for each aps_params_type.

[0083] As described above, in the image processing system 11, when encoding and decoding an image, a parameter set in which the maximum number of APSs to be referred to as an adaptive loop filter is defined as a fixed value can be used. In this way, by setting the maximum number of APSs to a fixed value, for example, in the image decoding device 13, it is possible to avoid making the buffer size of the database 27 for storing parameters parsed from the APS larger than necessary, and reduce the installation load required for image processing compared to the prior art.

[0084] Here, in this specification, an image refers to an original image before encoding, and a decoded image refers to an image output after performing a decoding process. A locally decoded image is an image output after being locally decoded when encoding an image. Encoded data refers to data after an image (texture) is encoded, and a bitstream (or encoded bitstream and encoded stream) refers to data including data in which parameters required for encoding or decoding are encoded and encoded data.

[0085] In addition, parameter (coding parameter) is a general term for data required for encoding or decoding, and is generally a bitstream syntax, parameter set, etc. In addition, parameter (coding parameter) also includes variables used in derivation processing, etc. In the present disclosure, identification data for identifying multiple modes can be set as bitstream syntax. In this case, the decoder can perform processing more efficiently by parsing and referring to the identification data.

[0086] <Configuration example of computer-based system>

[0087] Figure 7 is a block diagram showing a configuration example of an embodiment of a computer base system to which the present technology is applied.

[0088] Figure 7 is a block diagram showing a configuration example of a network system that connects one or more computers or servers via a network. Note that Figure 7 The hardware and software environments shown in the embodiments are shown as examples that can provide a platform for installing software and / or methods according to the present disclosure.

[0089] like Figure 7 As shown in FIG. 3 , the network system 31 includes a computer 32, a network 33, a remote computer 34, a web server 35, a cloud storage server 36, and a computer server 37. Figure 1 One or more of the functional blocks shown in perform multiple instances.

[0090] also, Figure 7The detailed configuration of the computer 32 is shown. Note that the functional blocks shown in the computer 32 are shown for establishing exemplary functions and are not limited to such configurations. In addition, the detailed configurations of the remote computer 34, the web server 35, the cloud storage server 36, and the computer server 37 are not shown, but include configurations similar to the functional blocks shown in the computer 32.

[0091] Computer 32 may be a personal computer, desktop computer, laptop computer, tablet computer, netbook computer, personal digital assistant, smart phone, or other programmable electronic device capable of communicating with other devices on a network.

[0092] Computer 32 includes bus 41, processor 42, memory 43, non-volatile storage device 44, network interface 46, peripheral device interface 47, and display interface 48. In some embodiments, each of these functions is implemented in a separate electronic subsystem (an integrated circuit chip or a combination of a chip and associated devices), or in other embodiments, some of the functions are combined and can be installed on a single chip (system on chip (SOC)).

[0093] The bus 41 may adopt various proprietary or industry-standard high-speed parallel or serial peripheral interconnection buses.

[0094] Processor 42 may employ processors designed and / or manufactured as one or more single or multiple chip microprocessors.

[0095] The memory 43 and the non-volatile storage device 44 are storage media that can be read by the computer 32. For example, the memory 43 can adopt any suitable volatile storage device, such as a dynamic random access memory (DRAM) or a static RAM (SRAM). The non-volatile storage device 44 can adopt at least one of the following: a flexible disk, a hard disk, a solid state drive (SSD), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM), a flash memory, a compact disk (CD or CD-ROM), a digital versatile disk (DVD), a card-type memory, or a stick-type memory.

[0096] In addition, program 45 is stored in non-volatile storage 44. For example, program 45 is a set of machine-readable instructions and / or data used to create, manage, and control specific software functions. Note that in configurations where memory 43 is much faster than non-volatile storage 44, program 45 can be transferred from non-volatile storage 44 to memory 43 before being executed by processor 42.

[0097] Computer 32 can communicate and interact with other computers via network interface 46 and network 33. Network 33 can adopt a configuration including wired, wireless or fiber optic connections, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination of LAN and WAN. In general, network 33 includes any combination of connections and protocols that support communication between two or more computers and related devices.

[0098] The peripheral device interface 47 can input data to other devices that can be locally connected to the computer 32 and output data from other devices that can be locally connected to the computer 32. For example, the peripheral device interface 47 provides a connection with an external device 51. The external device 51 includes a keyboard, a mouse, a keypad, a touch screen, and / or other suitable input devices. The external device 51 can also include, for example, a thumb drive, a portable optical or magnetic disk, and a portable computer-readable storage medium such as a memory card.

[0099] In an embodiment of the present disclosure, for example, software and data for implementing the program 45 may be stored on such a portable computer-readable storage medium. In such an embodiment, the software may be loaded directly into the non-volatile storage device 44 or into the memory 43 via the peripheral device interface 47. The peripheral device interface 47 may use industry standards such as RS-232 or a universal serial bus (USB) for connecting to an external device 51.

[0100] The display interface 48 may connect the computer 32 to the display 52, and may be used to present a command line or a graphical user interface to a user of the computer 32 by using the display 52. ​​For example, the display interface 48 may adopt industry standards such as Video Graphics Array (VGA), Digital Visual Interface (DVI), Display Port, and High-Definition Multimedia Interface (HDMI) (registered trademark).

[0101] <Configuration Example of Image Coding Device>

[0102] Figure 8 A configuration of an embodiment of an image encoding device as an image processing device to which the present disclosure is applied is shown.

[0103] Figure 8 The image encoding device 60 shown in encodes image data using a prediction process. Here, as an encoding method, for example, a High Efficiency Video Coding (HEVC) method or the like is used.

[0104] Figure 8The image encoding device 60 has an A / D conversion unit 61, a screen rearrangement buffer 62, a calculation unit 63, an orthogonal transform unit 64, a quantization unit 65, a reversible encoding unit 66, and an accumulation buffer 67. In addition, the image encoding device 60 includes an inverse quantization unit 68, an inverse orthogonal transform unit 69, a calculation unit 70, a deblocking filter 71, an adaptive offset filter 72, an adaptive loop filter 73, a frame memory 74, a selection unit 75, an intra-frame prediction unit 76, a motion prediction / compensation unit 77, a prediction image selection unit 78, and a rate control unit 79.

[0105] The A / D conversion unit 61 A / D-converts input image data (picture), and supplies the A / D-converted image data to the screen rearrangement buffer 62. Note that a digital data image may be input without providing the A / D conversion unit 61.

[0106] The screen rearrangement buffer 62 stores the image data supplied from the A / D conversion unit 61, and rearranges the frames of the stored display order images in the frame order for encoding according to the picture group (GOP) structure. The screen rearrangement buffer 62 outputs the images whose frame order is rearranged to the calculation unit 63, the intra prediction unit 76, and the motion prediction / compensation unit 77.

[0107] The calculation unit 63 subtracts the predicted image supplied from the intra prediction unit 76 or the motion prediction / compensation unit 77 via the predicted image selection unit 78 from the image output by the screen rearrangement buffer 62 , and outputs the difference information to the orthogonal transformation unit 64 .

[0108] For example, in the case of an image to be intra-encoded, the calculation unit 63 subtracts the predicted image supplied from the intra-prediction unit 76 from the image output by the screen rearrangement buffer 62. Furthermore, for example, in the case of an image to be inter-encoded, the calculation unit 63 subtracts the predicted image supplied from the motion prediction / compensation unit 77 from the image output by the screen rearrangement buffer 62.

[0109] The orthogonal transform unit 64 performs orthogonal transform such as discrete cosine transform and Karhunen-Loève transform on the difference information supplied from the calculation unit 63 , and supplies the conversion coefficient to the quantization unit 65 .

[0110] The quantization unit 65 quantizes the conversion coefficient output by the orthogonal transformation unit 64. The quantization unit 65 supplies the quantized conversion coefficient to the reversible encoding unit 66.

[0111] The reversible coding unit 66 applies reversible coding such as variable length coding and arithmetic coding to the quantized transform coefficient.

[0112] The reversible encoding unit 66 acquires parameters such as information indicating the intra prediction mode from the intra prediction unit 76 , and acquires parameters such as information indicating the inter prediction mode or motion vector information from the motion prediction / compensation unit 77 .

[0113] The reversible encoding unit 66 encodes the quantized conversion coefficient, and encodes each acquired parameter (syntax element) as a part of header information of the encoded data (multiplexing). The reversible encoding unit 66 supplies the encoded data obtained by the encoding to the accumulation buffer 67 for storage.

[0114] For example, in the reversible coding unit 66, a reversible coding process such as variable length coding or arithmetic coding is performed. Examples of variable length coding include context adaptive variable length coding (CAVLC) and the like. Examples of arithmetic coding include context adaptive binary arithmetic coding (CABAC) and the like.

[0115] The accumulation buffer 67 temporarily stores the coded stream (coded data) supplied from the reversible coding unit 66, and outputs the stored coded stream as a coded image at a predetermined timing to, for example, a recording device or a transmission line (not shown) in a subsequent stage. That is, the accumulation buffer 67 is also a transmission unit that transmits the coded stream.

[0116] Furthermore, the conversion coefficient quantized in the quantization unit 65 is also supplied to the inverse quantization unit 68. The inverse quantization unit 68 inversely quantizes the quantized conversion coefficient by a method corresponding to the quantization of the quantization unit 65. The inverse quantization unit 68 supplies the obtained conversion coefficient to the inverse orthogonal transformation unit 69.

[0117] The inverse orthogonal transform unit 69 performs inverse orthogonal transform on the supplied transform coefficient by a method corresponding to the orthogonal transform process of the orthogonal transform unit 64. The inverse orthogonal transform output (restored difference information) is supplied to the calculation unit 70.

[0118] The calculation unit 70 adds the predicted image supplied from the intra prediction unit 76 or the motion prediction / compensation unit 77 via the predicted image selection unit 78 and the inverse orthogonal transformation result supplied from the inverse orthogonal transformation unit 69, that is, the restored difference information, and obtains a locally decoded image.

[0119] For example, when the difference information corresponds to an image to be intra-coded, the calculation unit 70 adds the predicted image supplied from the intra-prediction unit 76 to the difference information. Also, for example, when the difference information corresponds to an image to be inter-coded, the calculation unit 70 adds the predicted image supplied from the motion prediction / compensation unit 77 to the difference information.

[0120] The decoded image as a result of the addition is supplied to the deblocking filter 71 and the frame memory 74 .

[0121] The deblocking filter 71 suppresses block distortion of the decoded image by appropriately performing deblocking filtering processing on the image from the calculation unit 70, and supplies the filtering processing result to the adaptive offset filter 72. The deblocking filter 71 has parameters β and Tc obtained based on the quantization parameter QP. The parameters β and Tc are thresholds (parameters) for determining the deblocking filter.

[0122] Note that the parameters β and Tc of the deblocking filter 71 are extended from β and Tc defined by the HEVC method. Each offset in the parameters β and Tc is encoded by the reversible encoding unit 66 as a parameter of the deblocking filter and transmitted to the . Fig.10 An image decoding device 80 is provided.

[0123] The adaptive offset filter 72 performs an offset filter (sample adaptive offset (SAO)) process for mainly suppressing ringing on the image filtered by the deblocking filter 71 .

[0124] There are 9 types of offset filters: 2 types of band offsets, 6 types of edge offsets, and no offset. The adaptive offset filter 72 performs filtering processing on the image filtered by the deblocking filter 71 by using a quadtree structure in which the type of offset filter is determined for each divided area and an offset value for each divided area. The adaptive offset filter 72 supplies the filtered image to the adaptive loop filter 73.

[0125] Note that in the image encoding device 60, the quadtree structure and the offset value of each divided region are calculated and used by the adaptive offset filter 72. The calculated quadtree structure and the offset value of each divided region are encoded by the reversible encoding unit 66 as adaptive offset parameters and transmitted to the image encoding device 60 to be described later. Fig.10 An image decoding device 80 is provided.

[0126] The adaptive loop filter 73 performs adaptive loop filtering (ALF) processing on the image filtered by the adaptive offset filter 72 in the processing unit by using the filter coefficient. In the adaptive loop filter 73, for example, a two-dimensional Wiener filter is used as a filter. Of course, a filter other than the Wiener filter can be used. The adaptive loop filter 73 supplies the filtering processing result to the frame memory 74.

[0127] Note that although Figure 8Although not shown in the example of FIG. 6 , in the image encoding device 60, a filter coefficient is calculated for each processing unit by the adaptive loop filter 73 so as to minimize the residual in the original image from the screen rearrangement buffer 62. The calculated filter coefficient is encoded as an adaptive loop filter parameter by the reversible encoding unit 66 and transmitted to the image encoding device 60 described later. Fig.10 An image decoding device 80 is provided.

[0128] The frame memory 74 outputs the accumulated reference image to the intra prediction unit 76 or the motion prediction / compensation unit 77 via the selection unit 75 at predetermined timing.

[0129] For example, in the case of an image to be intra-coded, the frame memory 74 supplies the reference image to the intra-prediction unit 76 via the selection unit 75. Also, for example, in the case of an image to be intra-coded, the frame memory 74 supplies the reference image to the motion prediction / compensation unit 77 via the selection unit 75.

[0130] When the reference image supplied from the frame memory 74 is an image to be intra-coded, the selection unit 75 supplies the reference image to the intra-prediction unit 76. In addition, when the reference image supplied from the frame memory 74 is an image to be inter-coded, the selection unit 75 supplies the reference image to the motion prediction / compensation unit 77.

[0131] The intra prediction unit 76 performs intra prediction (intra-screen prediction) of generating a predicted image using pixel values ​​within the screen. The intra prediction unit 76 performs intra prediction in a plurality of modes (intra-prediction modes).

[0132] The intra prediction unit 76 generates prediction images in all intra prediction modes, evaluates each prediction image, and selects the optimal mode. When the optimal intra prediction mode is selected, the intra prediction unit 76 supplies the prediction image generated in the optimal mode to the calculation unit 63 and the calculation unit 70 via the prediction image selection unit 78.

[0133] Furthermore, as described above, the intra prediction unit 76 appropriately supplies parameters such as intra prediction mode information indicating the adopted intra prediction mode to the reversible encoding unit 66 .

[0134] The motion prediction / compensation unit 77 performs motion prediction on the image to be inter-coded by using the input image supplied from the screen rearrangement buffer 62 and the reference image supplied from the frame memory 74 via the selection unit 75. Furthermore, the motion prediction / compensation unit 77 performs motion compensation processing according to the motion vector detected by the motion prediction, and generates a predicted image (inter-prediction image information).

[0135] The motion prediction / compensation unit 77 performs inter prediction processing in all candidate inter prediction modes and generates a predicted image. The motion prediction / compensation unit 77 supplies the generated predicted image to the calculation unit 63 or the calculation unit 70 via the predicted image selection unit 78. In addition, the motion prediction / compensation unit 77 supplies parameters such as inter prediction mode information indicating the adopted inter prediction mode or motion vector information indicating the calculated motion vector to the reversible encoding unit 66.

[0136] The prediction image selection unit 78 supplies the output of the intra-frame prediction unit 76 to the calculation unit 63 or the calculation unit 70 in the case of an image to be intra-frame encoded, and supplies the output of the motion prediction / compensation unit 77 to the calculation unit 63 or the calculation unit 70 in the case of an image to be inter-frame encoded.

[0137] The rate control unit 79 controls the rate of the quantization operation of the quantization unit 65 based on the compressed images accumulated in the accumulation buffer 67 so that overflow or underflow does not occur.

[0138] In this way, the image encoding device 60 is configured, and the reversible encoding unit 66 corresponds to Figure 1 The encoding unit 22 in the embodiment, and the adaptive loop filter 73 corresponds to Figure 1 The filter unit 23 in FIG. 2 has the same function as the setting unit 21. Therefore, as described above, the image encoding device 60 can reduce the installation load required for image processing.

[0139] <Operation of Image Coding Apparatus>

[0140] Will refer to Fig. 9 The flow of the encoding process performed by the image encoding device 60 described above is described.

[0141] In step S31 , the A / D conversion unit 61 performs A / D conversion on the input image.

[0142] In step S32 , the screen rearrangement buffer 62 stores the A / D converted images of the A / D conversion unit 61 , and rearranges the images from the display order of each picture to the encoding order.

[0143] When the image to be processed supplied from the screen rearrangement buffer 62 is an image of a block to be subjected to intra processing, a reference image is read from the frame memory 74 and supplied to the intra prediction unit 76 via the selection unit 75 .

[0144] Based on these images, in step S33, the intra prediction unit 76 performs intra prediction on the pixels of the block to be processed in all candidate intra prediction modes. Note that as decoded pixels to be referred to, pixels that are not filtered by the deblocking filter 71 are used.

[0145] Through this processing, intra prediction is performed in all candidate intra prediction modes, and cost function values ​​are calculated for all candidate intra prediction modes. Then, the best intra prediction mode is selected based on the calculated cost function value, and the predicted image generated by the intra prediction of the best intra prediction mode and its cost function value are supplied to the predicted image selection unit 78.

[0146] When the image to be processed supplied from the screen rearrangement buffer 62 is an image to be subjected to intra-frame processing, a reference decoded image is read from the frame memory 74, and the reference decoded image is supplied to the motion prediction / compensation unit 77 via the selection unit 75. Based on these images, the motion prediction / compensation unit 77 performs motion prediction / compensation processing in step S34.

[0147] Through this process, motion prediction processing is performed in all candidate inter prediction modes, cost function values ​​are calculated for all candidate inter prediction modes, and the optimal inter prediction mode is determined based on the calculated cost function values. Then, the predicted image generated by the optimal inter prediction mode and its cost function value are supplied to the predicted image selection unit 78.

[0148] In step S35, the predicted image selection unit 78 determines one of the optimal intra prediction mode and the optimal inter prediction mode as the optimal prediction mode based on each cost function value output from the intra prediction unit 76 and the motion prediction / compensation unit 77. Then, the predicted image selection unit 78 selects the predicted image of the determined optimal prediction mode, and supplies the selected predicted image to the calculation units 63 and 70. The predicted image is used for calculations of steps S36 and S41 described later.

[0149] Note that the selection information of the predicted image is supplied to the intra prediction unit 76 or the motion prediction / compensation unit 77. When selecting the predicted image of the optimal intra prediction mode, the intra prediction unit 76 supplies information indicating the optimal intra prediction mode (i.e., parameters related to intra prediction) to the reversible encoding unit 66.

[0150] When the predicted image of the optimal inter prediction mode is selected, the motion prediction / compensation unit 77 outputs information indicating the optimal inter prediction mode and information according to the optimal inter prediction mode (i.e., parameters related to motion prediction) to the reversible encoding unit 66. Examples of the information according to the optimal inter prediction mode include motion vector information or reference frame information.

[0151] In step S36, the calculation unit 63 calculates the difference between the image rearranged in step S32 and the predicted image selected in step S35. The predicted image is supplied to the calculation unit 63 from the motion prediction / compensation unit 77 in the case of inter prediction and from the intra prediction unit 76 in the case of intra prediction via the predicted image selection unit 78, respectively.

[0152] The amount of difference data is smaller than that of the original image data. Therefore, the amount of data can be compressed compared to the case where the image is encoded as it is.

[0153] In step S37, the orthogonal transform unit 64 performs an orthogonal transform on the difference information supplied from the calculation unit 63. Specifically, an orthogonal transform such as a discrete cosine transform and a Karhunen-Loève transform is performed, and a conversion coefficient is output.

[0154] In step S38, the conversion coefficient is quantized by the quantization unit 65. In this quantization, the rate is controlled as described in the process of step S49 described later.

[0155] The difference information quantized as described above is locally decoded as follows. That is, in step S39, the inverse quantization unit 68 inversely quantizes the transform coefficient quantized by the quantization unit 65 by a characteristic corresponding to the characteristic of the quantization unit 65. In step S40, the inverse orthogonal transform unit 69 inversely orthogonally transforms the transform coefficient inversely quantized by the inverse quantization unit 68 by a characteristic corresponding to the characteristic of the orthogonal transform unit 64.

[0156] In step S41 , the calculation unit 70 adds the predicted image input via the predicted image selection unit 78 and the locally decoded difference information, and generates a locally decoded (ie, locally decoded) image (an image corresponding to the input to the calculation unit 63 ).

[0157] In step S42, the deblocking filter 71 performs a deblocking filter process on the image output from the calculation unit 70. In this case, as a threshold value for determination of the deblocking filter, parameters β and Tc extended from β and Tc defined by the HEVC method are used. The filtered image from the deblocking filter 71 is output to the adaptive offset filter 72.

[0158] Note that the parameter β and the offset of Tc used in the deblocking filter 71 input by the user by operating the operation unit or the like are supplied to the reversible encoding unit 66 as parameters of the deblocking filter.

[0159] In step S43, the adaptive offset filter 72 performs an adaptive offset filter process. Through this process, the adaptive offset filter 72 performs a filter process on the image filtered by the deblocking filter 71 by using a quadtree structure in which the type of offset filter is determined for each divided area and an offset value for each divided area. The filtered image is fed to the adaptive loop filter 73.

[0160] Note that the determined quadtree structure and the offset value of each divided area are supplied to the reversible encoding unit 66 as an adaptive offset parameter.

[0161] In step S44, the adaptive loop filter 73 performs adaptive loop filter processing on the image filtered by the adaptive offset filter 72. For example, for each processing unit, filter processing is performed on the image after filtering by the adaptive offset filter 72 by using a filter coefficient, and the filter processing result is supplied to the frame memory 74.

[0162] In step S45 , the frame memory 74 stores the filtered image. Note that an image that is not filtered by the deblocking filter 71 , the adaptive offset filter 72 , and the adaptive loop filter 73 is also supplied from the calculation unit 70 and stored in the frame memory 74 .

[0163] On the other hand, the conversion coefficient quantized in the above-mentioned step S38 is also supplied to the reversible coding unit 66. In step S46, the reversible coding unit 66 encodes the quantized conversion coefficient output from the quantization unit 65 and each supplied parameter. That is, the difference image is variable length encoded, such as arithmetic encoding, and compressed. Here, examples of each of the encoding parameters include parameters of a deblocking filter, parameters of an adaptive offset filter, parameters of an adaptive loop filter, quantization parameters, motion vector information, reference frame information, or prediction mode information, etc.

[0164] In step S47, the accumulation buffer 67 accumulates the encoded difference image (ie, the encoded stream) as a compressed image. The compressed image accumulated in the accumulation buffer 67 is appropriately read out and transmitted to the decoding side via the transmission path.

[0165] In step S48 , the rate control unit 79 controls the rate of the quantization operation of the quantization unit 65 based on the compressed images accumulated in the accumulation buffer 67 so that overflow or underflow does not occur.

[0166] When the process of step S48 is completed, the encoding process is completed.

[0167] In the above encoding process, when the adaptive loop filter process is performed in step S44, a parameter set is set in which the maximum number of parameter sets referenced by the adaptive loop filter is defined as a fixed value, and in step S46, a bit stream including the set parameter set is generated.

[0168] <Configuration Example of Image Decoding Device>

[0169] Fig.10 A configuration of an embodiment of an image encoding device as an image processing device to which the present disclosure is applied is shown. Fig.10 The image decoding device 80 shown in FIG. Figure 8 A decoding device of the image encoding device 60.

[0170] It is assumed that a coded stream (coded data) coded by the image coding device 60 is transmitted to the image decoding device 80 corresponding to the image coding device 60 via a predetermined transmission line, and is decoded.

[0171] like Fig.10 As shown in the figure, the image decoding device 80 includes an accumulation buffer 81, a reversible decoding unit 82, an inverse quantization unit 83, an inverse orthogonal transform unit 84, a calculation unit 85, a deblocking filter 86, an adaptive offset filter 87, an adaptive loop filter 88, a picture reordering buffer 89, a D / A conversion unit 90, a frame memory 91, a selection unit 92, an intra-frame prediction unit 93, a motion prediction / compensation unit 94 and a selection unit 95.

[0172] The accumulation buffer 81 is also a receiver for receiving the transmitted coded data. The accumulation buffer 81 receives the transmitted coded data and accumulates the received coded data. The coded data is encoded by the image encoding device 60. The reversible decoding unit 82 Figure 8 The encoded data read from the accumulation buffer 81 is decoded at a predetermined timing by a method corresponding to the encoding method of the reversible encoding unit 66 in .

[0173] The reversible decoding unit 82 acquires parameters such as information indicating the decoded intra prediction mode from the intra prediction unit 93, and supplies parameters such as information indicating the inter prediction mode or motion vector information to the motion prediction / compensation unit 94. Furthermore, the reversible decoding unit 82 supplies the decoded parameters of the deblocking filter to the deblocking filter 86, and supplies the decoded adaptive offset parameters to the adaptive offset filter 87.

[0174] The inverse quantization unit 83 is connected with Figure 8The coefficient data (quantized coefficient) obtained by decoding by the reversible decoding unit 82 is quantized by a method corresponding to the quantization method of the quantization unit 65 in FIG. That is, the inverse quantization unit 83 uses the quantization parameter supplied from the image encoding device 60 by a method similar to Figure 8 The inverse quantization of the quantized coefficients is performed by the method of the inverse quantization unit 68 in FIG.

[0175] The inverse quantization unit 83 supplies the inverse quantization coefficient data, that is, the orthogonal transformation coefficient, to the inverse orthogonal transformation unit 84. The inverse orthogonal transformation unit 84 Figure 8 The orthogonal transform coefficients are inversely orthogonally transformed by a method corresponding to the orthogonal transform coefficients of the orthogonal transform unit 64 in the image encoding device 60, and decoded residual data corresponding to the residual data before the orthogonal transform of the image encoding device 60 is obtained.

[0176] The decoded residual data obtained by the inverse orthogonal transform is supplied to the calculation unit 85. Furthermore, the calculation unit 85 is supplied with the predicted image from the intra prediction unit 93 or the motion prediction / compensation unit 94 via the selection unit 95.

[0177] The calculation unit 85 adds the decoded residual data and the predicted image, and obtains decoded image data corresponding to the image data before the predicted image is subtracted by the calculation unit 63 of the image encoding device 60. The calculation unit 85 supplies the decoded image data to the deblocking filter 86.

[0178] The deblocking filter 86 suppresses block distortion of the decoded image by appropriately performing deblocking filtering processing on the image from the calculation unit 85, and supplies the filtering processing result to the adaptive offset filter 87. Figure 8 The deblocking filter 86 is configured by using the deblocking filter 71 shown in FIG. That is, the deblocking filter 86 has parameters β and Tc obtained based on the quantization parameter. The parameters β and Tc are thresholds (parameters) for determining the deblocking filter.

[0179] Note that the parameters β and Tc of the deblocking filter 86 are extended from β and Tc defined by the HEVC method. Each offset of the parameters β and Tc of the deblocking filter encoded by the image encoding device 60 is received by the image decoding device 80 as a parameter of the deblocking filter, decoded by the reversible decoding unit 82, and used by the deblocking filter 86.

[0180] The adaptive offset filter 87 performs offset filtering (SAO) processing for mainly suppressing ringing on the image filtered by the deblocking filter 86 .

[0181] The adaptive offset filter 87 performs filtering processing on the image filtered by the deblocking filter 86 by using a quadtree structure in which the type of offset filter is determined for each divided region and an offset value for each divided region. The adaptive offset filter 87 supplies the filtered image to the adaptive loop filter 88.

[0182] Note that the quadtree structure and the offset value of each divided region are calculated by the adaptive offset filter 72 of the image encoding device 60, and are encoded and transmitted as adaptive offset parameters. Then, the offset value of each divided region and the quadtree structure encoded by the image encoding device 60 are received by the image decoding device 80 as adaptive offset parameters, decoded by the reversible decoding unit 82, and used by the adaptive offset filter 87.

[0183] The adaptive loop filter 88 performs filter processing on the image filtered by the adaptive offset filter 87 for each processing unit using the filter coefficient, and supplies the filter processing result to the frame memory 91 and the screen rearrangement buffer 89 .

[0184] Note that although Fig.10 Although not shown in the example, in the image decoding device 80, filter coefficients are calculated for each LUC by the adaptive loop filter 73 of the image encoding device 60, and the filter coefficients are encoded and transmitted as adaptive loop filter parameters and decoded and used by the reversible decoding unit 82.

[0185] The screen rearrangement buffer 89 rearranges the image and supplies the rearranged image to the D / A conversion unit 90. That is, by Figure 8 The order of frames rearranged into encoding order in the picture rearrangement buffer 62 is rearranged in the original display order.

[0186] The D / A conversion unit 90 D / A converts the image (decoded picture) supplied from the screen rearrangement buffer 89 and outputs the D / A converted image to a display (not shown) for display. Note that the digital data image may be output as is without providing the D / A conversion unit 90.

[0187] The output of the adaptive loop filter 88 is also supplied to the frame memory 91 .

[0188] The frame memory 91, the selection unit 92, the intra-frame prediction unit 93, the motion prediction / compensation unit 94 and the selection unit 95 respectively correspond to the frame memory 74, the selection unit 75, the intra-frame prediction unit 76, the motion prediction / compensation unit 77 and the predicted image selection unit 78 of the image encoding device 60.

[0189] The selection unit 92 reads the inter-processed image and the reference image from the frame memory 91, and supplies the read images to the motion prediction / compensation unit 94. Furthermore, the selection unit 92 reads the image for intra-prediction from the frame memory 91, and supplies the read images to the intra-prediction unit 93.

[0190] Information indicating the intra prediction mode obtained by decoding the header information and the like are appropriately supplied from the reversible decoding unit 82 to the intra prediction unit 93. Based on the information, the intra prediction unit 93 generates a predicted image from the reference image acquired from the frame memory 91, and supplies the generated predicted image to the selection unit 95.

[0191] Information obtained by decoding the header information (prediction mode information, motion vector information, reference frame information, flags, various parameters, and the like) is supplied from the reversible decoding unit 82 to the motion prediction / compensation unit 94 .

[0192] Based on this information supplied from the reversible decoding unit 82 , the motion prediction / compensation unit 94 generates a predicted image from the reference image acquired from the frame memory 91 , and supplies the generated predicted image to the selection unit 95 .

[0193] The selection unit 95 selects the predicted image generated by the motion prediction / compensation unit 94 or the intra prediction unit 93 , and supplies the selected predicted image to the calculation unit 85 .

[0194] In this way, the image decoding device 80 is configured, and the reversible decoding unit 82 corresponds to Figure 1 The decoding unit 25, and the adaptive loop filter 88 corresponds to Figure 1 The filter unit 26. Therefore, as described above, the image decoding device 80 can reduce the installation load required for image processing.

[0195] <Operation of Image Decoding Device>

[0196] Will refer to Fig.11 An example of the flow of a decoding process performed by the image decoding device 80 as described above is described.

[0197] When the decoding process is started, in step S51, the accumulation buffer 81 receives and accumulates the transmitted coded stream (data). In step S52, the reversible decoding unit 82 decodes the coded data supplied from the accumulation buffer 81. Figure 8 The I picture, P picture and B picture encoded by the reversible encoding unit 66 are decoded.

[0198] Before decoding a picture, parameter information such as motion vector information, reference frame information, and prediction mode information (intra-prediction mode or inter-prediction mode) is also decoded.

[0199] When the prediction mode information is intra prediction mode information, the prediction mode information is supplied to the intra prediction unit 93. When the prediction mode information is inter prediction mode information, motion vector information corresponding to the prediction mode information is supplied to the motion prediction / compensation unit 94. In addition, the parameters of the deblocking filter and the adaptive offset parameters are also decoded and supplied to the deblocking filter 86 and the adaptive offset filter 87, respectively.

[0200] In step S53 , the intra prediction unit 93 or the motion prediction / compensation unit 94 respectively performs a predicted image generation process corresponding to the prediction mode information supplied from the reversible decoding unit 82 .

[0201] That is, when the intra prediction mode information is supplied from the reversible decoding unit 82, the intra prediction unit 93 generates an intra prediction image of the intra prediction mode. When the inter prediction mode information is supplied from the reversible decoding unit 82, the motion prediction / compensation unit 94 performs motion prediction / compensation processing in the inter prediction mode to generate an inter prediction image.

[0202] Through this processing, the predicted image (intra-prediction image) generated by the intra-prediction unit 93 or the predicted image (inter-prediction image) generated by the motion prediction / compensation unit 94 is supplied to the selection unit 95 .

[0203] In step S54, the selection unit 95 selects a predicted image. That is, the predicted image generated by the intra prediction unit 93 or the predicted image generated by the motion prediction / compensation unit 94 is supplied. Therefore, the supplied predicted image is selected and supplied to the calculation unit 85, and is added to the output of the inverse orthogonal transform unit 84 in step S57 described later.

[0204] In the above step S52, the conversion coefficient decoded by the reversible decoding unit 82 is also supplied to the inverse quantization unit 83. In step S55, the inverse quantization unit 83 performs Figure 8 The transform coefficients decoded by the reversible decoding unit 82 are inversely quantized according to characteristics corresponding to the characteristics of the quantization unit 65 in FIG.

[0205] In step S56, the inverse orthogonal transform unit 84 performs the Figure 8 The transform coefficients inversely quantized by the inverse quantization unit 83 are inversely orthogonally transformed according to the characteristics corresponding to the characteristics of the orthogonal transform unit 64 in FIG. Figure 8 The difference information corresponding to the input of the orthogonal transform unit 64 (the output of the calculation unit 63) is decoded.

[0206] In step S57, the calculation unit 85 is selected in the process of step S54 described above, and adds the predicted image input via the selection unit 95 to the difference information. Thus, the original image is decoded.

[0207] In step S58, the deblocking filter 86 performs a deblocking filter process on the image output from the calculation unit 85. In this case, as a threshold value for determination regarding the deblocking filter, parameters β and Tc extended from β and Tc defined by the HEVC method are used. The filtered image from the deblocking filter 86 is output to the adaptive offset filter 87. Note that in the deblocking filter process, each of the offsets of the parameters β and Tc of the deblocking filter supplied from the reversible decoding unit 82 is also used.

[0208] In step S59, the adaptive offset filter 87 performs an adaptive offset filter process. Through this process, the adaptive offset filter 87 performs a filter process on the image filtered by the deblocking filter 86 by using a quadtree structure in which the type of offset filter is determined for each divided area and an offset value for each divided area. The filtered image is fed to the adaptive loop filter 88.

[0209] In step S60, the adaptive loop filter 88 performs adaptive loop filtering processing on the image filtered by the adaptive offset filter 87. The adaptive loop filter 88 performs filtering processing on the input image of each processing unit using the filter coefficient calculated for each processing unit, and supplies the filtering processing result to the screen rearrangement buffer 89 and the frame memory 91.

[0210] In step S61, the frame memory 91 stores the filtered image.

[0211] In step S62, the screen rearrangement buffer 89 rearranges the image after the adaptive loop filter 88, and then supplies the image to the D / A conversion unit 90. That is, the order of frames rearranged into the encoding order by the screen rearrangement buffer 62 of the image encoding device 60 is rearranged in the original display order.

[0212] In step S63 , the D / A conversion unit 90 D / A converts the image rearranged by the screen rearrangement buffer 89 , and outputs the D / A converted image to a display (not shown) for display.

[0213] When the process of step S63 ends, the decoding process ends.

[0214] In the above decoding process, in step S52, a bit stream including a parameter set is decoded, in which the maximum number of parameter sets referenced by the adaptive loop filter is defined as a fixed value, and when the adaptive loop filtering process in step S60 is performed, the adaptive loop filter is applied to the decoded image with reference to the decoded parameter set.

[0215] <Computer Configuration Example>

[0216] Next, the above-described series of processing (image processing method) can be executed by hardware or software. When the series of processing is executed by software, a program constituting the software is installed on a general-purpose computer or the like.

[0217] Fig.12 : is a block diagram showing a configuration example of an embodiment of a computer on which a program for executing the above-described series of processes is installed.

[0218] The program may be recorded in advance on the hard disk 105 or the ROM 103 which is a recording medium built in the computer.

[0219] Alternatively, the program may be stored (recorded) in a removable recording medium 111 driven by the drive 109. Such a removable recording medium 111 may be provided as so-called packaged software. Here, examples of the removable recording medium 111 include a flexible disk, a compact disk read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a magnetic disk, a semiconductor memory, and the like.

[0220] Note that, as described above, the program can be installed on the computer not only from the removable recording medium 111 but also downloaded to the computer via a communication network or a broadcasting network and installed on the built-in hard disk 105. That is, for example, the program can be wirelessly transmitted from a download site to the computer via an artificial satellite for digital satellite broadcasting, or the program can be transmitted to the computer by line via a network such as a local area network (LAN) or the Internet.

[0221] The computer has a built-in central processing unit (CPU) 102 , and an input / output interface 110 is connected to the CPU 102 via a bus 101 .

[0222] When the user inputs a command via the input / output interface 110 by operating the input unit 107 or the like, the CPU 102 accordingly executes a program stored in the read only memory (ROM) 103. Alternatively, the CPU 102 loads a program stored in the hard disk 105 into the random access memory (RAM) 104 and executes the loaded program.

[0223] Thus, the CPU 102 executes the processing according to the above-described flowchart or the processing executed according to the configuration of the above-described block diagram. Then, the CPU 102 outputs the processing result from the output unit 106 by the communication unit 108, or transmits the processing result from the communication unit 108 via, for example, the input / output interface 110 as needed, and also records the processing result on the hard disk 105.

[0224] Note that the input unit 107 is composed of a keyboard, a mouse, a microphone, etc. In addition, the output unit 106 is composed of a liquid crystal display (LCD), a speaker, and the like.

[0225] Here, in this specification, the processing performed by a computer according to a program does not necessarily have to be performed in a time series according to the order described in the flowchart. That is, the processing performed by a computer according to a program also includes processing performed in parallel or individually (for example, parallel processing or processing performed by an object).

[0226] In addition, the program may be processed by one computer (processor), or may be distributed and processed by a plurality of computers. In addition, the program may be transmitted to a remote computer and executed.

[0227] In addition, in this specification, a system refers to a group of multiple components (devices, modules (parts), etc.), and it is not important whether all the components are in the same housing. Therefore, any one of multiple devices housed in separate housings and connected via a network and one device housing multiple modules in one housing are both systems.

[0228] In addition, for example, a configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configuration described above as a plurality of devices (or processing units) may be collectively configured as one device (or processing unit). In addition, of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). In addition, when the configuration and operation of the entire system are substantially the same, a portion of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0229] In addition, for example, the present technology may be configured as cloud computing in which one function is shared and jointly processed by a plurality of devices via a network.

[0230] In addition, for example, the above-mentioned program can be executed in any device. In this case, the device can have necessary functions (functional blocks, etc.) so that necessary information can be obtained.

[0231] In addition, for example, each step described in the above flowchart can be performed by one device, and can also be shared and performed by multiple devices. In addition, when one step includes multiple processes, the multiple processes included in one step can be performed by one device or shared and performed by multiple devices. In other words, the multiple processes included in one step can be performed as processes of multiple steps. Conversely, the processes described as multiple steps can be performed together as one step.

[0232] Note that the program executed by the computer may cause the processing of the steps for describing the program to be performed in a time sequence in the order described in this specification, or may be performed in parallel or individually at a desired timing, such as when a call is made. That is, the processing of each step may be performed in an order different from the above order as long as there is no contradiction. In addition, the processing of the steps for describing the program may be performed in parallel with the processing of another program, or may be performed in combination with the processing of another program.

[0233] It should be noted that the present technology described in this specification can be implemented independently of each other as long as there is no contradiction. Of course, any multiple present technologies can be used in combination. For example, some or all of the technologies described in any one of the embodiments can be combined with some or all of the technologies described in other embodiments. In addition, part or all of any of the above technologies can be performed in combination with other technologies not described above.

[0234] <Configuration combination example>

[0235] Note that the present technology may also be configured as follows. (1)

[0237] An image processing device, comprising:

[0238] a decoding unit configured to decode a bitstream including a parameter set in which a maximum number of parameter sets referenced by an adaptive loop filter is defined as a fixed value to generate a decoded image; and

[0239] A filter unit is configured to apply the adaptive loop filter to a decoded image generated by the decoding unit by referring to the parameter set decoded by the decoding unit. (2)

[0241] The image processing device according to (1) above, wherein:

[0242] The maximum number is a fixed value that does not depend on the number of blocks that divide the slice. (3)

[0244] The image processing device according to (1) or (2) above, wherein:

[0245] The maximum number is a fixed value defined for each level. (4)

[0247] The image processing device according to any one of (1) to (3) above, wherein:

[0248] The parameter set is an APS (Adaptive Parameter Set). (5)

[0250] An image processing device according to any one of (1) to (4) above, wherein:

[0251] The maximum number is a fixed value defined for each aps_params_type. (6)

[0253] An image processing device according to any one of (1) to (5) above, wherein:

[0254] The maximum number is defined for each aps_params_type and is a fixed value that is different for each level. (7)

[0256] An image processing method, comprising:

[0257] a decoding process of decoding a bitstream including parameter sets in which a maximum number of parameter sets referenced by an adaptive loop filter is defined as a fixed value to generate a decoded image; and

[0258] The adaptive loop filter is applied to a decoded image generated in the decoding process by referring to the parameter set decoded in the decoding process. (8)

[0260] An image processing device, comprising:

[0261] a setting unit configured to set a parameter set in which a maximum number of parameter sets referenced by the adaptive loop filter is defined as a fixed value; and

[0262] An encoding unit is configured to encode the image to generate a bit stream including the parameter set set by the setting unit. (9)

[0264] The image processing device according to (8) above further includes:

[0265] a filter unit configured to apply the adaptive loop filter to a local coded image when encoding is performed in the encoding unit,

[0266] The encoding unit encodes the image using the filtered image to which the adaptive loop filter is applied by the filter unit. (10)

[0268] An image processing method, comprising:

[0269] a setting process of setting a parameter set in which a maximum number of parameter sets to be referenced by an adaptive loop filter is defined as a fixed value; and

[0270] An encoding process of encoding an image to generate a bit stream including the parameter set set in the setting process.

[0271] Note that the embodiments of the present technology are not limited to the above-described embodiments, and various changes can be made without departing from the gist of the present disclosure. In addition, the effects described in this specification are merely examples and are not limited, and other effects can be obtained.

[0272] Reference numerals list

[0273] 11Image Processing System

[0274] 12Image encoding device

[0275] 13 Image decoding device

[0276] 21 Setting Unit

[0277] 22 coding units

[0278] 23 filter units

[0279] 24 Database

[0280] 25 decoding units

[0281] 26 filter units

[0282] 27 Database

Claims

1. An image processing device, comprising: a decoding unit configured to decode a bitstream including a parameter set in which a maximum number of parameter sets used as references for an adaptive loop filter is defined as a fixed value to generate a decoded image; as well as A filter unit is configured to apply the adaptive loop filter to a decoded image generated by the decoding unit by referring to the parameter set decoded by the decoding unit.

2. The image processing device according to claim 1, wherein: The maximum number is a fixed value that does not depend on the number of blocks that divide the slice.

3. The image processing device according to claim 1, wherein: The maximum number is a fixed value defined for each level.

4. The image processing device according to claim 1, wherein: The parameter set is an APS (Adaptive Parameter Set).

5. The image processing device according to claim 1, wherein: The maximum number is a fixed value defined for each aps_params_type.

6. The image processing device according to claim 1, wherein: The maximum number is defined for each aps_params_type and is a fixed value that is different for each level.

7. An image processing method, comprising: A decoding process for decoding a bitstream comprising parameter sets to generate a decoded image, wherein a maximum number of parameter sets used as references for an adaptive loop filter is defined as a fixed value; as well as The adaptive loop filter is applied to a decoded image generated in the decoding process by referring to a parameter set decoded in the decoding process.

8. An image processing device, comprising: a setting unit configured to set a parameter set in which a maximum number of parameter sets used as references for the adaptive loop filter is defined as a fixed value; as well as An encoding unit is configured to encode the image to generate a bit stream including the parameter set set by the setting unit.

9. The image processing apparatus according to claim 8, further comprising: a filter unit configured to apply the adaptive loop filter to a local coded image when encoding is performed in the encoding unit, The encoding unit encodes the image using the filtered image to which the adaptive loop filter is applied by the filter unit.

10. An image processing method, comprising: a setting process of setting a parameter set in which a maximum number of parameter sets used as references for an adaptive loop filter is defined as a fixed value; as well as An encoding process of encoding an image to generate a bit stream including the parameter set set in the setting process.

Images