Image processing apparatus and image processing method

Abstract

To reduce a mounting load required for image processing.SOLUTION: A decoding unit decodes a bitstream including a parameter set in which a maximum number of the number of parameter sets referenced as an adaptive loop filter is defined as a fixed value to generate a decoded image. A filter unit references the parameter set decoded by the decoding unit and applies the adaptive loop filter to the decoded image generated by the decoding unit. The present technology can be applied to, for example, an image processing system that performs image processing using an adaptive loop filter.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus and an image processing method, and more particularly, to an image processing apparatus and an image processing method capable of reducing the implementation load required for image processing.

Background Art

[0002] Conventionally, as disclosed in Non-Patent Document 1, it has been possible to switch a parameter set (up to 25 classes per set) of an adaptive loop filter (ALF) in units of a coding tree unit (CTU).

[0003] For example, the maximum number of parameter sets that can be switched is 22 types, the number of fixed filters (Fixed filter) whose coefficients are determined by the standard is 16 types, and the number of user-specified filters transmitted by the user with an adaptation parameter set (APS) is 6 types. Also, APS is used to transmit parameters.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, in the current APS specifications, since the degree of freedom is too high (including ambiguous points), unless restrictions are imposed on the usage method, the decoder side always has to decode the maximum number of APS IDs (= 32). For this reason, there has been a concern that the implementation load would become high conventionally.

[0006] This disclosure has been made in view of such a situation, and enables reduction of the implementation load required for image processing.

Means for Solving the Problem

[0010] In the One side image processing apparatus of the present disclosure, a filter unit that applies the adaptive loop filter to a locally decoded image generated by local decoding when encoding an image is provided according to a parameter set defined as a fixed value defined for each aps_params_type which is the type of APS, which is the maximum number of parameter sets referred to as the adaptive loop filter.

[0011] In the One side image processing method of the present disclosure, applying the adaptive loop filter to a locally decoded image generated by local decoding when encoding an image is included according to a parameter set defined as a fixed value defined for each aps_params_type which is the type of APS, which is the maximum number of parameter sets referred to as the adaptive loop filter.

[0012] In the One side of the present disclosure, the adaptive loop filter is applied to a locally decoded image generated by local decoding when encoding an image according to a parameter set defined as a fixed value defined for each aps_params_type which is the type of APS, which is the maximum number of parameter sets referred to as the adaptive loop filter.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0014] <Literature etc. Supporting Technical Content and Technical Terms> The scope disclosed in this specification is not limited to the content of the embodiments. The content of the following reference documents REF1 to REF6, which were known at the time of filing, is also incorporated herein by reference. That is, the content described in reference documents REF1 to REF6 is also used as a basis for judging the support requirements. Also, for technical terms such as Parsing, Syntax, Semantics, etc., even if they are not directly defined in the detailed description of the invention, they are within the scope of this disclosure and are considered to meet the support requirements of the claims.

[0015] REF1 Recommendation ITU-T H.264 (04 / 2017) “Advanced video coding for generic audiovisual services”, April 2017 REF2 Recommendation ITU-T H.265 (02 / 2018) “High efficiency video coding”, February 2018 REF3 Benjamin Bross, Jianle Chen, Shan Liu, Versatile Video Coding (Draft 5), JVET-N1001-v7 (version 7 - date 2019-05-29) REF4: Jianle Chen, Yan Ye, Seung Hwan Kim, Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5), JVETN1002-v1 REF5: Ye-Kui Wang, Hendry, Jianle Chen, Peng Yin, Taoran Lu, Fangjun Pu, Sean McCarthy, AHG17: Signalling of reshaper parameters in APS, JVET-N0117-v1 (version 1 - date 2019-03-12) REF6: Nael Ouedraogo, Eric Nassor, Jonathan Taquet, Gerald Kergourlay, Frederic Maze, [AHG17 / AHG12] On APS id for bitstream merging for VVC, JVET-N0191-v1 (version 1 - date 2019-03-12)

[0016] <Term> In this application, the following terms are defined as follows.

[0017] <Block> The "block" (not the block indicating the processing unit) used in the description as a partial area or processing unit of an image (picture) indicates any partial area in the picture, unless otherwise specified, and its size, shape, characteristics, etc. are not limited. For example, the "block" includes any partial area (processing unit) such as TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding TreeBlock), CTU (Coding Tree Unit), transform block, sub-block, macro-block, tile, or slice, etc.

[0018] <Specification of block size> Also, when specifying the size of such a block, not only can the block size be directly specified, but it may also be specified indirectly. For example, the block size may be specified using identification information for identifying the size. Also, for example, the block size may be specified by a ratio or difference from the size of a reference block (such as an LCU or SCU, etc.). For example, when transmitting information specifying the block size as a syntax element or the like, as that information, information indirectly specifying the size as described above may be used. By doing so, the amount of information of that information can be reduced, and in some cases, the coding efficiency can be improved. Also, the specification of this block size includes the specification of the range of the block size (for example, the specification of the allowable range of the block size, etc.).

[0019] <Unit of Information and Processing> The data unit in which various information is set and the data unit to which various processes are applied are each arbitrary and not limited to the examples described above. For example, these information and processes may be set for each of TU (Transform Unit), TB (Transform Block), PU (Prediction Unit), PB (Prediction Block), CU (Coding Unit), LCU (Largest Coding Unit), sub-block, block, tile, slice, picture, sequence, or component, or the data of those data units may be targeted. Of course, this data unit can be set for each piece of information and process, and it is not necessary for the data units of all information and processes to be unified. Note that the storage location of this information is arbitrary and may be stored in the header or parameter set of the data unit described above. Also, it may be stored in multiple locations.

[0020] <Control Information> Control information related to this technology may be transmitted from the encoding side to the decoding side. For example, it may be transmitted to control whether to permit (or prohibit) the application of the above-described technology (e.g., enabled_flag). Also, for example, it may be transmitted to control information indicating the target (or non-target) to which the above-described technology is applied. For example, it may be transmitted to control information specifying the block size (upper limit or lower limit, or both), frame, component, or layer, etc., to which this technology is applied (or to permit or prohibit the application).

[0021] <Flag> Note that in this specification, a "flag" is information for identifying a plurality of states, and includes not only information used for identifying two states of true (1) or false (0), but also information capable of identifying three or more states. Therefore, the values that this "flag" can take may be, for example, two values of 1 / 0, or three or more values. That is, the number of bits constituting this "flag" is arbitrary and may be 1 bit or a plurality of bits. Also, since identification information (including flags) is assumed not only in the form of including the identification information in a bit stream, but also in the form of including the difference information of the identification information with respect to a certain reference information in the bit stream, in this specification, "flags" and "identification information" include not only the information itself, but also the difference information with respect to the reference information.

[0022] <Associate metadata> Also, various types of information (such as metadata) related to the encoded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the encoded data. Here, the term "associate" means, for example, making it possible to use (link) the other data when processing one data. That is, the data associated with each other may be grouped as one data or may be individual data respectively. For example, the information associated with the encoded data (image) may be transmitted on a transmission path different from that of the encoded data (image). Also, for example, the information associated with the encoded data (image) may be recorded on a recording medium different from that of the encoded data (image) (or a different recording area of the same recording medium). Note that this "association" may be a part of the data instead of the whole data. For example, an image and the information corresponding to the image may be associated with each other in any unit such as a plurality of frames, one frame, or a part within a frame.

[0023] Note that in this specification, terms such as "synthesize", "multiplex", "add", "integrate", "include", "store", "insert", "plug in", "insert" mean, for example, grouping a plurality of things into one, such as grouping the encoded data and the metadata into one data, and mean one method of the above-mentioned "associate". Also, in this specification, encoding includes not only the entire process of converting an image into a bitstream but also a part of the process. For example, it includes not only processes including prediction processing, orthogonal transformation, quantization, arithmetic coding, etc., but also processes collectively referred to as quantization and arithmetic coding, processes including prediction processing, quantization, and arithmetic coding, etc. Similarly, decoding includes not only the entire process of converting a bitstream into an image but also a part of the process. For example, it includes not only processes including inverse arithmetic decoding, inverse quantization, inverse orthogonal transformation, prediction processing, etc., but also processes including inverse arithmetic decoding and inverse quantization, processes including inverse arithmetic decoding, inverse quantization, and prediction processing, etc.

[0024] Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

[0025] <Configuration Example of Image Processing System> FIG. 1 is a block diagram showing a configuration example of an embodiment of an image processing system to which the present technology is applied.

[0026] As shown in FIG. 1, the image processing system 11 includes an image encoding device 12 and an image decoding device 13. For example, in the image processing system 11, an image input to the image encoding device 12 is encoded, and a bitstream obtained by the encoding is transmitted to the image decoding device 13, and a decoded image decoded from the bitstream is output by the image decoding device 13.

[0027] The image encoding device 12 has a setting unit 21, an encoding unit 22, a filter unit 23, and a database 24, and the image decoding device 13 has a decoding unit 25, a filter unit 26, and a database 27.

[0028] The setting unit 21 sets a parameter set in which the maximum number of parameter sets referred to as an adaptive loop filter is defined as a fixed value.

[0029] The encoding unit 22 encodes an image and generates a bitstream including the parameter set set by the setting unit 21. At this time, the encoding unit 22 encodes the image using the filtered image to which the adaptive loop filter is applied by the filter unit 23.

[0030] The filter unit 23 applies an adaptive loop filter to the local encoded image when encoding is performed by the encoding unit 22.

[0031] The database 24 stores various parameters such as filter coefficients required when the encoding unit 22 encodes an image.

[0032] The decoding unit 25 decodes a bit stream including a parameter set in which the maximum number of parameter sets referred to as an adaptive loop filter is defined as a fixed value, and generates a decoded image.

[0033] The filter unit 26 applies an adaptive loop filter to the decoded image generated by the decoding unit 25 with reference to the parameter set decoded by the decoding unit 25.

[0034] The database 27 stores various parameters such as filter coefficients required when the decoding unit 25 decodes an image.

[0035] Here, in the adaptive loop filter applied in the filter unit 23 and the filter unit 26, for one slice, up to 6 sets of filter coefficients can be stored in the APS with the current filter coefficient and the filter coefficient in the time direction. This is a design assuming that there are 5 temporal levels (temporal_id) from 0 to 4, and a total of 6 sets of filter coefficients are stored in the APS with 1 set of filter coefficients for each of those levels and 1 set of current filter coefficients.

[0036] On the other hand, when one slice is divided into a plurality of tile groups (tile_of_group), data for (6 sets × a plurality of tile groups) needs to be transmitted, and the number of APS IDs increases in proportion to the number of tile groups. Therefore, not only does the processing amount for parsing the APS in the image decoding apparatus 13 increase, but it also becomes necessary to increase the buffer size of the database 27 for storing the parameters parsed from the APS.

[0037] With reference to FIG. 2, an example of using the ALF_APS will be described.

[0038] As shown in A of FIG. 2, in the adaptive loop filter, it is possible to refer to a maximum of six APSs in units of slices. On the other hand, when a slice is divided into a plurality of tiles, it is possible to refer to a maximum of six APSs in units of tiles.

[0039] Therefore, as shown in B of FIG. 2, when a slice is divided into six tiles, it becomes 6 APSs × 6 tiles = 36 APSs, and the number of APSs may exceed the maximum number of APS IDs (= 32).

[0040] Also, depending on the use case (e.g., 8K vs. HD, 2D vs. omnidirectional image, etc.), the number of appropriate APS IDs to be used differs. From this, without some kind of usage restriction, the image decoder 13 would have to prepare a database 27 with a buffer size capable of always expanding the maximum number of APS IDs, resulting in a high implementation load required for image processing.

[0041] Therefore, in the image processing system 11, in order to reduce the implementation load required for image processing, the maximum number of parameter sets (hereinafter referred to as the maximum APS number) to be referred to as the adaptive loop filter is defined as a fixed value.

[0042] First, in the first defined example, the maximum APS number is defined for each aps_params_type that specifies the type of APS parameters.

[0043] For example, as shown in FIG. 3, as aps_params_type, 0 and 1 are already used to specify types of APS parameters, and 2 to 7 are reserved. And in the case of aps_params_type in which ALF parameters are used (aps_params_type = 0), the maximum APS number is defined as 6.

[0044] That is, regardless of the number of tile_of_group, the maximum number of APS IDs that can be referenced as an adaptive loop filter is set to a fixed value. That is, the maximum number of APSs is defined to be a fixed value independent of the number of tiles. However, the APS IDs referenced in each tile_of_group may be different from each other. This defines the maximum number of APS IDs for the adaptive loop filter. In other words, in the first defined example, for each aps_params_type, the maximum number of APS IDs to be used is defined.

[0045] Next, in the second defined example, according to the level, it is defined that the maximum number of APSs is bit-aligned. By defining the maximum number of APSs for each level in this way, it is possible to set the APS specifications suitable for the use case.

[0046] FIG. 4 shows an example of the second defined example in which the maximum number of APSs is defined to be bit-aligned according to the level.

[0047] In the example shown in FIG. 4, the maximum number of APSs for levels from 1 to 4.1 is 8 (3 bits), the maximum number of APSs for levels from 5 to 5.2 is 16 (4 bits), and the maximum number of APSs for levels from 6 to 6.2 is 32 (5 bits), which are defined in byte units.

[0048] Also, for the omnidirectional image (for each projection format), the maximum number of APSs is defined in the same way.

[0049] Furthermore, as a third defined example combining the first defined example and the second defined example, according to the level, the maximum number of APSs may be defined as a multiple of the unit usage number set for each aps_params_type. That is, the maximum number of APSs is defined for each aps_params_type and is a fixed value different for each level. Thereby, the maximum number of APS IDs that can be used for each aps_params_type can be changed according to the level. That is, the maximum number of APS IDs is defined by "tool (function)" × "parallelization requirement (Level)".

[0050] FIG. 5 shows an example of a third defined example that combines the first defined example and the second defined example.

[0051] In the example shown in FIG. 5, the maximum number of APSs from level 1 to 4.1 is defined as 1 times the unit usage number set for each aps_params_type. Also, the maximum number of APSs from level 5 to 5.2 is defined as 2 times the unit usage number set for each aps_params_type, and the maximum number of APSs from level 6 to 6.2 is defined as 4 times the unit usage number set for each aps_params_type.

[0052] Specifically, as shown in FIG. 6, for the maximum number of APSs in an HD image (level = 4.x), the maximum number of APSs in a 4K image (level = 5.x) is defined to be 2 times the unit usage number set for each aps_params_type.

[0053] As described above, in the image processing system 11, when encoding and decoding an image, a parameter set in which the maximum number of APSs referred to as an adaptive loop filter is defined as a fixed value can be used. In this way, by setting the maximum number of APSs as a fixed value, for example, in the image decoding device 13, it is possible to avoid making the buffer size of the database 27 for storing the parameters parsed from the APS larger than necessary, and compared with the conventional case, the implementation load required for image processing can be reduced.

[0054] Here, in this specification, an image means an original image before encoding, a decoded image means an image output after decoding processing, a local decoded image means an image output after being locally decoded when encoding an image, encoded data means data after an image (texture) is encoded, and a bitstream (or encoded bitstream, encoded stream) means data including encoded data in which parameters required for encoding or decoding are encoded in addition to the encoded data.

[0055] Also, a parameter (encoding parameter) is a general term for data necessary for encoding or decoding, and typically includes the syntax of a bit stream, a parameter set, etc. Further, the parameter (encoding parameter) shall also include variables used in the derivation process. In the present disclosure, recognition data for recognizing a plurality of patterns can also be set as the syntax of a bit stream. In this case, the decoder can perform processing more efficiently by parsing and referring to the identification data.

[0056] <Configuration example of a computer-based system> FIG. 7 is a block diagram showing a configuration example of an embodiment of a computer-based system to which the present technology is applied.

[0057] FIG. 7 is a block diagram showing a configuration example of a network system in which one or more computers, servers, etc. are connected via a network. Note that the hardware and software environment shown in the embodiment of FIG. 7 is shown as an example capable of providing a platform for implementing the software and / or method according to the present disclosure.

[0058] As shown in FIG. 7, the network system 31 includes a computer 32, a network 33, a remote computer 34, a web server 35, a cloud storage server 36, and a computer server 37. Here, in the present embodiment, a plurality of instances are executed by one or more of the functional blocks shown in FIG. 1.

[0059] Also, in FIG. 7, the detailed configuration of computer 32 is illustrated. Note that the functional blocks shown within computer 32 are illustrated for establishing exemplary functions and are not limited to such a configuration. Also, the detailed configurations of remote computer 34, web server 35, cloud storage server 36, and computer server 37 are not illustrated, but these include configurations similar to the functional blocks shown within computer 32.

[0060] As computer 32, a personal computer, desktop computer, laptop computer, tablet computer, netbook computer, personal digital assistant, smartphone, or other programmable electronic device capable of communicating with other devices on a network can be used.

[0061] And computer 32 is configured to include bus 41, processor 42, memory 43, non-volatile storage 44, network interface 46, peripheral device interface 47, and display interface 48. Each of these functions can be implemented in certain embodiments in individual electronic subsystems (integrated circuit chips or combinations of chips and related devices), or in other embodiments, some of the functions may be combined and implemented in a single chip (system on chip or SoC (System on Chip)).

[0062] Bus 41 can employ various proprietary or industry-standard high-speed parallel or serial peripheral interconnect buses.

[0063] Processor 42 can employ one or more single or multi-chip microprocessors designed and / or manufactured as such.

[0064] Memory 43 and non-volatile storage 44 are storage media readable by computer 32. For example, memory 43 can employ any suitable volatile storage device such as DRAM (Dynamic Random Access Memory) or SRAM (Static RAM). Non-volatile storage 44 can employ at least one or more of a flexible disk, a hard disk, an SSD (Solid State Drive), a ROM (Read Only Memory), an EPROM (Erasable and Programmable Read Only Memory), flash memory, a compact disk (CD or CD-ROM), a DVD (Digital Versatile Disc), a card-type memory, or a stick-type memory.

[0065] Also, program 45 is stored in non-volatile storage 44. Program 45 is, for example, a set of machine-readable instructions and / or data used to create, manage, and control a specific software function. In a configuration where memory 43 is much faster than non-volatile storage 44, program 45 can be transferred from non-volatile storage 44 to memory 43 before being executed by processor 42.

[0066] Computer 32 can communicate and interact with other computers via network 33 through network interface 46. Network 33 can adopt a configuration including a LAN (Local Area Network), a WAN (Wide Area Network) such as the Internet, or a combination of a LAN and a WAN, with a wired, wireless, or optical fiber connection. Generally, network 33 consists of any combination of connections and protocols that support communication between two or more computers and related devices.

[0067] The peripheral device interface 47 can perform input and output of data with other devices that can be locally connected to the computer 32. For example, the peripheral device interface 47 provides a connection to an external device 51. As the external device 51, a keyboard, a mouse, a keypad, a touch screen, and / or other appropriate input devices are used. The external device 51 may include, for example, a portable computer-readable storage medium such as a thumb drive, a portable optical disk or magnetic disk, and a memory card.

[0068] In an embodiment of the present disclosure, for example, the software and data used to execute the program 45 may be stored in such a portable computer-readable storage medium. In such an embodiment, the software may be directly loaded into the non-volatile storage 44 or into the memory 43 via the peripheral device interface 47. The peripheral device interface 47 may use an industry standard such as RS-232 or USB (Universal Serial Bus) for connection to the external device 51.

[0069] The display interface 48 can connect the computer 32 to a display 52, and using the display 52, a command line or a graphical user interface can be presented to the user of the computer 32. For example, the display interface 48 can adopt an industry standard such as VGA (Video Graphics Array), DVI (Digital Visual Interface), DisplayPort, HDMI (High-Definition Multimedia Interface) (registered trademark).

[0070] <Configuration Example of Image Encoding Device> FIG. 8 shows the configuration of an embodiment of an image encoding device as an image processing device to which the present disclosure is applied.

[0071] The image encoding device 60 shown in FIG. 8 encodes image data using prediction processing. Here, as the encoding method, for example, the HEVC (High Efficiency Video Coding) method or the like is used.

[0072] The image encoding device 60 in FIG. 8 includes an A / D conversion unit 61, a screen rearrangement buffer 62, an arithmetic unit 63, an orthogonal conversion unit 64, a quantization unit 65, a reversible encoding unit 66, and an accumulation buffer 67. The image encoding device 60 also includes an inverse quantization unit 68, an inverse orthogonal conversion unit 69, an arithmetic unit 70, a deblocking filter 71, an adaptive offset filter 72, an adaptive loop filter 73, a frame memory 74, a selection unit 75, an intra prediction unit 76, a motion prediction / compensation unit 77, a predicted image selection unit 78, and a rate control unit 79.

[0073] The A / D conversion unit 61 performs A / D conversion on the input image data (Picture(s)) and supplies it to the screen rearrangement buffer 62. Note that, instead of providing the A / D conversion unit 61, a configuration may be adopted in which digital data images are input.

[0074] The screen rearrangement buffer 62 stores the image data supplied from the A / D conversion unit 61 and rearranges the images of the frames in the stored display order into the order of frames for encoding according to the GOP (Group of Picture) structure. The screen rearrangement buffer 62 outputs the image with the frame order rearranged to the arithmetic unit 63, the intra prediction unit 76, and the motion prediction / compensation unit 77.

[0075] The arithmetic unit 63 subtracts the predicted image supplied from the intra prediction unit 76 or the motion prediction / compensation unit 77 via the predicted image selection unit 78 from the image output from the screen rearrangement buffer 62, and outputs the difference information to the orthogonal conversion unit 64.

[0076] For example, in the case of an intra-coded image, the arithmetic unit 63 subtracts the predicted image supplied from the intra prediction unit 76 from the image output from the screen rearrangement buffer 62. Also, for example, in the case of an inter-coded image, the arithmetic unit 63 subtracts the predicted image supplied from the motion prediction / compensation unit 77 from the image output from the screen rearrangement buffer 62.

[0077] The orthogonal transformation unit 64 performs an orthogonal transformation such as a discrete cosine transform or a Karhunen-Loeve transform on the differential information supplied from the arithmetic unit 63, and supplies the transformation coefficients to the quantization unit 65.

[0078] The quantization unit 65 quantizes the transformation coefficients output from the orthogonal transformation unit 64. The quantization unit 65 supplies the quantized transformation coefficients to the reversible coding unit 66.

[0079] The reversible coding unit 66 performs reversible coding such as variable length coding or arithmetic coding on the quantized transformation coefficients.

[0080] The reversible coding unit 66 acquires parameters such as information indicating the intra prediction mode from the intra prediction unit 76, and acquires parameters such as information indicating the inter prediction mode and motion vector information from the motion prediction / compensation unit 77.

[0081] The reversible coding unit 66 encodes the quantized transformation coefficients and also encodes each acquired parameter (syntax element), and makes them part of the header information of the encoded data (multiplexes them). The reversible coding unit 66 supplies the encoded data obtained by encoding to the storage buffer 67 for storage.

[0082] For example, in the reversible coding unit 66, reversible coding processing such as variable length coding or arithmetic coding is performed. Examples of variable length coding include CAVLC (Context-Adaptive Variable Length Coding). Examples of arithmetic coding include CABAC (Context-Adaptive Binary Arithmetic Coding).

[0083] The accumulation buffer 67 temporarily holds the encoded stream (Encoded Data) supplied from the reversible encoding unit 66 and outputs it as an encoded image at a predetermined timing to, for example, a recording device or a transmission path (not shown) in the subsequent stage. That is, the accumulation buffer 67 is also a transmission unit that transmits the encoded stream.

[0084] Also, the quantization coefficients quantized in the quantization unit 65 are supplied to the inverse quantization unit 68. The inverse quantization unit 68 inverse-quantizes the quantized coefficients in a method corresponding to the quantization by the quantization unit 65. The inverse quantization unit 68 supplies the obtained coefficients to the inverse orthogonal transformation unit 69.

[0085] The inverse orthogonal transformation unit 69 inverse-orthogonally transforms the supplied coefficients in a method corresponding to the orthogonal transformation process by the orthogonal transformation unit 64. The inverse-orthogonally transformed output (restored difference information) is supplied to the arithmetic unit 70.

[0086] The arithmetic unit 70 adds the inverse-orthogonal transformation result supplied from the inverse orthogonal transformation unit 69, that is, the restored difference information, to the predicted image supplied from the intra prediction unit 76 or the motion prediction / compensation unit 77 via the prediction image selection unit 78 to obtain a locally decoded image (decoded image).

[0087] For example, when the difference information corresponds to an image for which intra encoding is performed, the arithmetic unit 70 adds the predicted image supplied from the intra prediction unit 76 to the difference information. Also, for example, when the difference information corresponds to an image for which inter encoding is performed, the arithmetic unit 70 adds the predicted image supplied from the motion prediction / compensation unit 77 to the difference information.

[0088] The decoded image that is the addition result is supplied to the deblocking filter 71 and the frame memory 74.

[0089] The deblocking filter 71 performs deblocking filter processing as appropriate on the image from the arithmetic unit 70 to suppress block distortion in the decoded image, and supplies the filter processing result to the adaptive offset filter 72. The deblocking filter 71 has parameters β and Tc obtained based on the quantization parameter QP. The parameters β and Tc are thresholds (parameters) used for determination regarding the deblocking filter.

[0090] Note that β and Tc, which are parameters of the deblocking filter 71, are extended from β and Tc defined in the HEVC format. Each offset of the parameters β and Tc is encoded as a parameter of the deblocking filter in the reversible encoding unit 66 and transmitted to the image decoding apparatus 80 in FIG. 10 described later.

[0091] The adaptive offset filter 72 performs offset filter (SAO: Sample adaptive offset) processing mainly for suppressing ringing on the image after filtering by the deblocking filter 71.

[0092] There are a total of nine types of offset filters, including two types of band offsets, six types of edge offsets, and no offset. The adaptive offset filter 72 uses a quad-tree structure in which the type of offset filter is determined for each divided region and the offset value for each divided region, and performs filter processing on the image after filtering by the deblocking filter 71. The adaptive offset filter 72 supplies the image after the filter processing to the adaptive loop filter 73.

[0093] Note that in the image encoding apparatus 60, the quad-tree structure and the offset value for each divided region are calculated and used by the adaptive offset filter 72. The calculated quad-tree structure and the offset value for each divided region are encoded as adaptive offset parameters in the reversible encoding unit 66 and transmitted to the image decoding apparatus 80 in FIG. 10 described later.

[0094] The adaptive loop filter 73 performs an adaptive loop filter (ALF: Adaptive Loop Filter) process for each processing unit on the image after filtering by the adaptive offset filter 72 using filter coefficients. In the adaptive loop filter 73, for example, a two-dimensional Wiener filter is used as the filter. Of course, filters other than the Wiener filter may be used. The adaptive loop filter 73 supplies the filter processing result to the frame memory 74.

[0095] Although not shown in the example of FIG. 8, in the image encoding apparatus 60, the filter coefficients are calculated and used by the adaptive loop filter 73 so as to minimize the residual with the original image from the screen rearrangement buffer 62 for each processing unit. The calculated filter coefficients are encoded in the reversible encoding unit 66 as adaptive loop filter parameters and transmitted to the image decoding apparatus 80 of FIG. 10 described later.

[0096] The frame memory 74 outputs the stored reference image to the intra prediction unit 76 or the motion prediction / compensation unit 77 via the selection unit 75 at a predetermined timing.

[0097] For example, in the case of an image for which intra encoding is performed, the frame memory 74 supplies the reference image to the intra prediction unit 76 via the selection unit 75. Also, for example, in the case of inter encoding, the frame memory 74 supplies the reference image to the motion prediction / compensation unit 77 via the selection unit 75.

[0098] When the reference image supplied from the frame memory 74 is an image for which intra encoding is performed, the selection unit 75 supplies the reference image to the intra prediction unit 76. Also, when the reference image supplied from the frame memory 74 is an image for which inter encoding is performed, the selection unit 75 supplies the reference image to the motion prediction / compensation unit 77.

[0099] The intra prediction unit 76 performs intra prediction (intra-frame prediction) that generates a predicted image using pixel values within the screen. The intra prediction unit 76 performs intra prediction in a plurality of modes (intra prediction modes).

[0100] The intra prediction unit 76 generates predicted images in all intra prediction modes, evaluates each predicted image, and selects an optimal mode. When the intra prediction unit 76 selects an optimal intra prediction mode, it supplies the predicted image generated in that optimal mode to the arithmetic unit 63 and the arithmetic unit 70 via the predicted image selection unit 78.

[0101] Also, as described above, the intra prediction unit 76 appropriately supplies parameters such as intra prediction mode information indicating the adopted intra prediction mode to the reversible coding unit 66.

[0102] The motion prediction / compensation unit 77 performs motion prediction on an image for which inter coding is performed, using the input image supplied from the screen rearrangement buffer 62 and the reference image supplied from the frame memory 74 via the selection unit 75. Also, the motion prediction / compensation unit 77 performs motion compensation processing according to the motion vector detected by the motion prediction, and generates a predicted image (inter prediction image information).

[0103] The motion prediction / compensation unit 77 performs inter prediction processing for all candidate inter prediction modes, and generates a predicted image. The motion prediction / compensation unit 77 supplies the generated predicted image to the arithmetic unit 63 and the arithmetic unit 70 via the predicted image selection unit 78. Also, the motion prediction / compensation unit 77 supplies parameters such as inter prediction mode information indicating the adopted inter prediction mode and motion vector information indicating the calculated motion vector to the reversible coding unit 66.

[0104] The predicted image selection unit 78 supplies the output of the intra prediction unit 76 to the arithmetic unit 63 and the arithmetic unit 70 in the case of an image to be intra-coded, and supplies the output of the motion prediction / compensation unit 77 to the arithmetic unit 63 and the arithmetic unit 70 in the case of an image to be inter-coded.

[0105] The rate control unit 79 controls the rate of the quantization operation of the quantization unit 65 so that overflow or underflow does not occur based on the compressed image stored in the accumulation buffer 67.

[0106] The image encoding device 60 is configured in this way. The reversible encoding unit 66 corresponds to the encoding unit 22 in FIG. 1, and the adaptive loop filter 73 corresponds to the filter unit 23 in FIG. 1 and has the function as the setting unit 21. Therefore, as described above, the image encoding device 60 can reduce the implementation load required for image processing.

[0107] <Operation of the image encoding device> With reference to FIG. 9, the flow of the encoding process executed by the image encoding device 60 as described above will be described.

[0108] In step S31, the A / D conversion unit 61 performs A / D conversion on the input image.

[0109] In step S32, the screen rearrangement buffer 62 stores the image A / D-converted by the A / D conversion unit 61 and rearranges the order of display of each picture to the order of encoding.

[0110] When the image to be processed supplied from the screen rearrangement buffer 62 is the image of the block to be intra-processed, the decoded image to be referred to is read from the frame memory 74 and supplied to the intra prediction unit 76 via the selection unit 75.

[0111] Based on these images, in step S33, the intra prediction unit 76 intra-predicts the pixels of the block to be processed in all intra prediction modes that are candidates. Note that, as the decoded pixels to be referred to, pixels that have not been filtered by the deblocking filter 71 are used.

[0112] Through this process, intra prediction is performed in all candidate intra prediction modes, and cost function values are calculated for all candidate intra prediction modes. Then, based on the calculated cost function values, an optimal intra prediction mode is selected, and the predicted image generated by the intra prediction in the optimal intra prediction mode and its cost function value are supplied to the predicted image selection unit 78.

[0113] When the image to be processed supplied from the screen rearrangement buffer 62 is an image to be inter-processed, the reference image is read from the frame memory 74 and supplied to the motion prediction / compensation unit 77 via the selection unit 75. Based on these images, in step S34, the motion prediction / compensation unit 77 performs motion prediction / compensation processing.

[0114] Through this process, motion prediction processing is performed in all candidate inter prediction modes, cost function values are calculated for all candidate inter prediction modes, and based on the calculated cost function values, an optimal inter prediction mode is determined. Then, the predicted image generated by the optimal inter prediction mode and its cost function value are supplied to the predicted image selection unit 78.

[0115] In step S35, the predicted image selection unit 78 determines one of the optimal intra prediction mode and the optimal inter prediction mode as the optimal prediction mode based on each cost function value output from the intra prediction unit 76 and the motion prediction / compensation unit 77. Then, the predicted image selection unit 78 selects the predicted image in the determined optimal prediction mode and supplies it to the arithmetic units 63 and 70. This predicted image is used in the arithmetic operations in steps S36 and S41 described later.

[0116] Note that the selection information of this predicted image is supplied to the intra prediction unit 76 or the motion prediction / compensation unit 77. When the predicted image in the optimal intra prediction mode is selected, the intra prediction unit 76 supplies information indicating the optimal intra prediction mode (that is, parameters related to intra prediction) to the reversible encoding unit 66.

[0117] When the predicted image of the optimal inter prediction mode is selected, the motion prediction / compensation unit 77 outputs information indicating the optimal inter prediction mode and information corresponding to the optimal inter prediction mode (i.e., parameters related to motion prediction) to the reversible coding unit 66. Examples of the information corresponding to the optimal inter prediction mode include motion vector information and reference frame information.

[0118] In step S36, the arithmetic unit 63 calculates the difference between the image rearranged in step S32 and the predicted image selected in step S35. The predicted image is supplied to the arithmetic unit 63 via the prediction image selection unit 78 from the motion prediction / compensation unit 77 in the case of inter prediction and from the intra prediction unit 76 in the case of intra prediction.

[0119] The amount of difference data is smaller than that of the original image data. Therefore, the amount of data can be compressed compared to the case of directly coding the image.

[0120] In step S37, the orthogonal transformation unit 64 orthogonally transforms the difference information supplied from the arithmetic unit 63. Specifically, orthogonal transformations such as discrete cosine transform and Karhunen - Loeve transform are performed, and the transformation coefficients are output.

[0121] In step S38, the quantization unit 65 quantizes the transformation coefficients. During this quantization, the rate is controlled as will be described in the process of step S49.

[0122] The difference information quantized as described above is locally decoded as follows. That is, in step S39, the inverse quantization unit 68 inverse - quantizes the transformation coefficients quantized by the quantization unit 65 with characteristics corresponding to the characteristics of the quantization unit 65. In step S40, the inverse orthogonal transformation unit 69 inverse - orthogonally transforms the transformation coefficients inverse - quantized by the inverse quantization unit 68 with characteristics corresponding to the characteristics of the orthogonal transformation unit 64.

[0123] In step S41, the arithmetic unit 70 adds the predicted image input via the predicted image selection unit 78 to the locally decoded differential information to generate a locally decoded (i.e., locally decoded) image (the image corresponding to the input to the arithmetic unit 63).

[0124] In step S42, the deblocking filter 71 performs deblocking filter processing on the image output from the arithmetic unit 70. At this time, as the determination threshold values for the deblocking filter, the parameter β and Tc extended from β and Tc defined in the HEVC method are used. The image after filtering from the deblocking filter 71 is output to the adaptive offset filter 72.

[0125] Note that the offsets of the parameters β and Tc that are input by the user operating the operation unit or the like and used in the deblocking filter 71 are supplied to the reversible encoding unit 66 as the parameters of the deblocking filter.

[0126] In step S43, the adaptive offset filter 72 performs adaptive offset filter processing. By this processing, using the quad-tree structure in which the type of offset filter is determined for each divided region and the offset value for each divided region, filter processing is performed on the image after filtering by the deblocking filter 71. The image after filtering is supplied to the adaptive loop filter 73.

[0127] Note that the determined quad-tree structure and the offset value for each divided region are supplied to the reversible encoding unit 66 as the adaptive offset parameters.

[0128] In step S44, the adaptive loop filter 73 performs adaptive loop filter processing on the image after filtering by the adaptive offset filter 72. For example, for the image after filtering by the adaptive offset filter 72, filter processing is performed on the image for each processing unit using filter coefficients, and the filter processing result is supplied to the frame memory 74.

[0129] In step S45, the frame memory 74 stores the filtered image. Note that the frame memory 74 is also supplied with and stores an image that has not been filtered by the deblocking filter 71, the adaptive offset filter 72, and the adaptive loop filter 73 from the arithmetic unit 70.

[0130] On the other hand, the quantization coefficients quantized in step S38 described above are also supplied to the reversible encoding unit 66. In step S46, the reversible encoding unit 66 encodes the quantization coefficients output from the quantization unit 65 and the supplied parameters. That is, the difference image is reversibly encoded and compressed by variable length encoding, arithmetic encoding, or the like. Here, examples of the parameters to be encoded include parameters of the deblocking filter, parameters of the adaptive offset filter, parameters of the adaptive loop filter, quantization parameters, motion vector information, reference frame information, prediction mode information, and the like.

[0131] In step S47, the accumulation buffer 67 accumulates the encoded difference image (i.e., the encoded stream) as a compressed image. The compressed image accumulated in the accumulation buffer 67 is appropriately read out and transmitted to the decoding side via a transmission path.

[0132] In step S48, the rate control unit 79 controls the rate of the quantization operation of the quantization unit 65 so that overflow or underflow does not occur based on the compressed image accumulated in the accumulation buffer 67.

[0133] When the process of step S48 ends, the encoding process ends.

[0134] In the encoding process as described above, when performing the adaptive loop filter process in step S44, a parameter set in which the maximum number of parameter sets referred to as the adaptive loop filter is defined as a fixed value is set, and in step S46, a bit stream including the set parameter set is generated.

[0135] <Configuration Example of Image Decoding Device> FIG. 10 shows the configuration of an embodiment of an image decoding device as an image processing device to which the present disclosure is applied. The image decoding device 80 shown in FIG. 10 is a decoding device corresponding to the image encoding device 60 of FIG. 8.

[0136] The encoded stream (Encoded Data) encoded by the image encoding device 60 is transmitted via a predetermined transmission path to the image decoding device 80 corresponding to this image encoding device 60 and is to be decoded.

[0137] As shown in FIG. 10, the image decoding device 80 includes an accumulation buffer 81, an invertible decoding unit 82, an inverse quantization unit 83, an inverse orthogonal transformation unit 84, an arithmetic unit 85, a deblocking filter 86, an adaptive offset filter 87, an adaptive loop filter 88, a screen rearrangement buffer 89, a D / A conversion unit 90, a frame memory 91, a selection unit 92, an intra prediction unit 93, a motion prediction / compensation unit 94, and a selection unit 95.

[0138] The accumulation buffer 81 is also a receiving unit that receives the transmitted encoded data. The accumulation buffer 81 receives and accumulates the transmitted encoded data. This encoded data is encoded by the image encoding device 60. The invertible decoding unit 82 decodes the encoded data read from the accumulation buffer 81 at a predetermined timing in a manner corresponding to the encoding method of the invertible encoding unit 66 in FIG. 8.

[0139] The invertible decoding unit 82 supplies parameters such as information indicating the decoded intra prediction mode to the intra prediction unit 93 and supplies parameters such as information indicating the inter prediction mode and motion vector information to the motion prediction / compensation unit 94. Further, the invertible decoding unit 82 supplies the decoded deblocking filter parameters to the deblocking filter 86 and supplies the decoded adaptive offset parameters to the adaptive offset filter 87.

[0140] The inverse quantization unit 83 inverse quantizes the coefficient data (quantized coefficients) obtained by decoding by the reversible decoding unit 82 in a method corresponding to the quantization method of the quantization unit 65 in FIG. 8. That is, the inverse quantization unit 83 inverse quantizes the quantized coefficients in the same manner as the inverse quantization unit 68 in FIG. 8 using the quantization parameters supplied from the image encoding device 60.

[0141] The inverse quantization unit 83 supplies the inverse quantized coefficient data, that is, the orthogonal transform coefficients, to the inverse orthogonal transform unit 84. The inverse orthogonal transform unit 84 inverse orthogonally transforms the orthogonal transform coefficients in a method corresponding to the orthogonal transform method of the orthogonal transform unit 64 in FIG. 8, and obtains decoded residual data corresponding to the residual data before orthogonal transformation in the image encoding device 60.

[0142] The decoded residual data obtained by inverse orthogonal transformation is supplied to the arithmetic unit 85. Also, a predicted image is supplied to the arithmetic unit 85 from the intra prediction unit 93 or the motion prediction / compensation unit 94 via the selection unit 95.

[0143] The arithmetic unit 85 adds the decoded residual data and the predicted image to obtain decoded image data corresponding to the image data before the predicted image is subtracted by the arithmetic unit 63 of the image encoding device 60. The arithmetic unit 85 supplies the decoded image data to the deblocking filter 86.

[0144] The deblocking filter 86 suppresses the block distortion of the decoded image by appropriately performing deblocking filter processing on the image from the arithmetic unit 85, and supplies the filter processing result to the adaptive offset filter 87. The deblocking filter 86 is basically configured in the same manner as the deblocking filter 71 in FIG. 8. That is, the deblocking filter 86 has parameters β and Tc obtained based on the quantization parameters. The parameters β and Tc are threshold values used for determination regarding the deblocking filter.

[0145] Note that the parameters β and Tc of the deblocking filter 86 are extended from β and Tc defined in the HEVC standard. The offsets of the parameters β and Tc of the deblocking filter encoded by the image encoding device 60 are received as parameters of the deblocking filter in the image decoding device 80, decoded by the reversible decoding unit 82, and used by the deblocking filter 86.

[0146] The adaptive offset filter 87 performs offset filter (SAO) processing mainly for suppressing ringing on the image after filtering by the deblocking filter 86.

[0147] The adaptive offset filter 87 uses a quad-tree structure in which the type of offset filter is determined for each divided region and offset values for each divided region to perform filtering on the image after filtering by the deblocking filter 86. The adaptive offset filter 87 supplies the image after the filtering process to the adaptive loop filter 88.

[0148] Note that this quad-tree structure and the offset values for each divided region are calculated by the adaptive offset filter 72 of the image encoding device 60, encoded as adaptive offset parameters, and sent. The quad-tree structure and the offset values for each divided region encoded by the image encoding device 60 are received as adaptive offset parameters in the image decoding device 80, decoded by the reversible decoding unit 82, and used by the adaptive offset filter 87.

[0149] The adaptive loop filter 88 performs filtering for each processing unit using filter coefficients on the image after filtering by the adaptive offset filter 87, and supplies the filter processing result to the frame memory 91 and the screen rearrangement buffer 89.

[0150] Although not shown in the example of FIG. 10, in the image decoding apparatus 80, the filter coefficients are calculated for each LUC by the adaptive loop filter 73 of the image encoding apparatus 60, and the encoded and transmitted ones are decoded by the inverse decoding unit 82 and used as adaptive loop filter parameters.

[0151] The screen rearrangement buffer 89 rearranges the images and supplies them to the D / A conversion unit 90. That is, the order of the frames rearranged for the encoding order by the screen rearrangement buffer 62 in FIG. 8 is rearranged to the original display order.

[0152] The D / A conversion unit 90 performs D / A conversion on the images (Decoded Picture(s)) supplied from the screen rearrangement buffer 89 and outputs them to a display (not shown) for display. Note that a configuration may be adopted in which the images are output as digital data without providing the D / A conversion unit 90.

[0153] The output of the adaptive loop filter 88 is further supplied to the frame memory 91.

[0154] The frame memory 91, the selection unit 92, the intra prediction unit 93, the motion prediction / compensation unit 94, and the selection unit 95 respectively correspond to the frame memory 74, the selection unit 75, the intra prediction unit 76, the motion prediction / compensation unit 77, and the predicted image selection unit 78 of the image encoding apparatus 60.

[0155] The selection unit 92 reads out the inter-processed image and the reference image from the frame memory 91 and supplies them to the motion prediction / compensation unit 94. The selection unit 92 also reads out the image used for intra prediction from the frame memory 91 and supplies it to the intra prediction unit 93.

[0156] To the intra prediction unit 93, information indicating the intra prediction mode obtained by decoding the header information and the like are appropriately supplied from the inverse decoding unit 82. Based on this information, the intra prediction unit 93 generates a predicted image from the reference image acquired from the frame memory 91 and supplies the generated predicted image to the selection unit 95.

[0157] The motion prediction / compensation unit 94 is supplied with information (prediction mode information, motion vector information, reference frame information, flags, various parameters, etc.) obtained by decrypting the header information from the reversible decoding unit 82.

[0158] Based on the information supplied from the reversible decoding unit 82, the motion prediction / compensation unit 94 generates a predicted image from the reference image acquired from the frame memory 91, and supplies the generated predicted image to the selection unit 95.

[0159] The selection unit 95 selects the predicted image generated by the motion prediction / compensation unit 94 or the intra prediction unit 93, and supplies it to the arithmetic unit 85.

[0160] The image decoding apparatus 80 is configured in this way. The reversible decoding unit 82 corresponds to the decoding unit 25 in FIG. 1, and the adaptive loop filter 88 corresponds to the filter unit 26 in FIG. 1. Therefore, as described above, the image decoding apparatus 80 can reduce the implementation load required for image processing.

[0161] <Operation of the Image Decoding Apparatus> With reference to FIG. 11, an example of the flow of the decoding process executed by the image decoding apparatus 80 as described above will be described.

[0162] When the decoding process is started, in step S51, the accumulation buffer 81 receives and accumulates the transmitted encoded stream (data). In step S52, the reversible decoding unit 82 decodes the encoded data supplied from the accumulation buffer 81. The I picture, P picture, and B picture encoded by the reversible encoding unit 66 in FIG. 8 are decoded.

[0163] Prior to the decoding of the picture, information on parameters such as motion vector information, reference frame information, and prediction mode information (intra prediction mode or inter prediction mode) is also decoded.

[0164] When the prediction mode information is intra prediction mode information, the prediction mode information is supplied to the intra prediction unit 93. When the prediction mode information is inter prediction mode information, the motion vector information corresponding to the prediction mode information and the like is supplied to the motion prediction / compensation unit 94. Also, the parameters of the deblocking filter and the adaptive offset parameters are also decoded and supplied to the deblocking filter 86 and the adaptive offset filter 87, respectively.

[0165] In step S53, the intra prediction unit 93 or the motion prediction / compensation unit 94 performs a prediction image generation process corresponding to the prediction mode information supplied from the reversible decoding unit 82, respectively.

[0166] That is, when the intra prediction mode information is supplied from the reversible decoding unit 82, the intra prediction unit 93 generates an intra prediction image in the intra prediction mode. When the inter prediction mode information is supplied from the reversible decoding unit 82, the motion prediction / compensation unit 94 performs motion prediction / compensation processing in the inter prediction mode and generates an inter prediction image.

[0167] By this process, the prediction image (intra prediction image) generated by the intra prediction unit 93 or the prediction image (inter prediction image) generated by the motion prediction / compensation unit 94 is supplied to the selection unit 95.

[0168] In step S54, the selection unit 95 selects a prediction image. That is, the prediction image generated by the intra prediction unit 93 or the prediction image generated by the motion prediction / compensation unit 94 is supplied. Therefore, the supplied prediction image is selected and supplied to the arithmetic unit 85, and is added to the output of the inverse orthogonal transformation unit 84 in step S57 described later.

[0169] In step S52 described above, the transformation coefficients decoded by the reversible decoding unit 82 are also supplied to the inverse quantization unit 83. In step S55, the inverse quantization unit 83 inverse quantizes the transformation coefficients decoded by the reversible decoding unit 82 with characteristics corresponding to the characteristics of the quantization unit 65 in FIG. 8.

[0170] In step S56, the inverse orthogonal transformation unit 84 performs an inverse orthogonal transformation on the transform coefficients inverse quantized by the inverse quantization unit 83 with characteristics corresponding to those of the orthogonal transformation unit 64 in FIG. 8. As a result, the differential information corresponding to the input (output of the arithmetic unit 63) of the orthogonal transformation unit 64 in FIG. 8 is decoded.

[0171] In step S57, the arithmetic unit 85 adds the predicted image selected in the process of step S54 described above and input via the selection unit 95 to the differential information. As a result, the original image is decoded.

[0172] In step S58, the deblocking filter 86 performs deblocking filter processing on the image output from the arithmetic unit 85. At this time, as the determination threshold values for the deblocking filter, the parameter β and Tc extended from β and Tc defined by the HEVC method are used. The image after filtering from the deblocking filter 86 is output to the adaptive offset filter 87. In addition, in the deblocking filter processing, the offsets of the parameters β and Tc of the deblocking filter supplied from the reversible decoding unit 82 are also used.

[0173] In step S59, the adaptive offset filter 87 performs adaptive offset filter processing. By this processing, using the quad-tree structure in which the type of offset filter is determined for each divided region and the offset value for each divided region, filter processing is performed on the image after filtering by the deblocking filter 86. The image after filtering is supplied to the adaptive loop filter 88.

[0174] In step S60, the adaptive loop filter 88 performs adaptive loop filter processing on the image after filtering by the adaptive offset filter 87. The adaptive loop filter 88 performs filter processing on the input image for each processing unit using the filter coefficients calculated for each processing unit, and supplies the filter processing result to the screen rearrangement buffer 89 and the frame memory 91.

[0175] In step S61, the frame memory 91 stores the filtered image.

[0176] In step S62, after the screen rearrangement buffer 89 rearranges the image after the adaptive loop filter 88, it supplies the image to the D / A conversion unit 90. That is, the order of the frames rearranged for encoding by the screen rearrangement buffer 62 of the image encoding device 60 is rearranged to the original display order.

[0177] In step S63, the D / A conversion unit 90 performs D / A conversion on the image rearranged by the screen rearrangement buffer 89 and outputs the image to a display (not shown), and the image is displayed.

[0178] When the process of step S63 ends, the decoding process ends.

[0179] In the decoding process as described above, in step S52, a bitstream including a parameter set in which the maximum number of parameter sets referred to as an adaptive loop filter is defined as a fixed value is decoded, and when performing the adaptive loop filter process in step S60, the decoded parameter set is referred to, and an adaptive loop filter is applied to the decoded image.

[0180] <Configuration example of computer> Next, the above-described series of processes (image processing method) can be performed by hardware or by software. When a series of processes are performed by software, the program constituting the software is installed in a general-purpose computer or the like.

[0181] FIG. 12 is a block diagram showing a configuration example of an embodiment of a computer in which a program for executing the above-described series of processes is installed.

[0182] The program can be pre-recorded in a hard disk 105 or a ROM 103 as a recording medium built into the computer.

[0183] Alternatively, the program can be stored (recorded) in a removable recording medium 111 driven by a drive 109. Such a removable recording medium 111 can be provided as so-called package software. Here, examples of the removable recording medium 111 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, a semiconductor memory, and the like.

[0184] In addition to being installed from the removable recording medium 111 as described above into the computer, the program can be downloaded to the computer via a communication network or a broadcast network and installed in the built-in hard disk 105. That is, the program can be wirelessly transferred to the computer from, for example, a download site via an artificial satellite for digital satellite broadcasting, or can be wiredly transferred to the computer via a network such as a LAN (Local Area Network) or the Internet.

[0185] The computer incorporates a CPU (Central Processing Unit) 102, and an input / output interface 110 is connected to the CPU 102 via a bus 101.

[0186] When a command is input by the user operating the input unit 107 via the input / output interface 110, the CPU 102 executes the program stored in the ROM (Read Only Memory) 103 accordingly. Alternatively, the CPU 102 loads and executes the program stored in the hard disk 105 into the RAM (Random Access Memory) 104.

[0187] As a result, the CPU 102 performs the processes according to the above-described flowchart or the processes performed according to the configuration of the above-described block diagram. Then, the CPU 102 outputs the processing result from the output unit 106 via the input / output interface 110 as necessary, or transmits it from the communication unit 108, or records it on the hard disk 105, etc.

[0188] Note that the input unit 107 is composed of a keyboard, a mouse, a microphone, etc. The output unit 106 is composed of an LCD (Liquid Crystal Display), a speaker, etc.

[0189] Here, in this specification, the processes performed by the computer according to the program do not necessarily have to be performed in time series in the order described as a flowchart. That is, the processes performed by the computer according to the program also include processes that are executed in parallel or individually (for example, parallel processing or object-based processing).

[0190] Also, the program may be processed by one computer (processor) or may be distributedly processed by a plurality of computers. Further, the program may be transferred to a remote computer and executed.

[0191] Furthermore, in this specification, a system means a collection of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and one device in which a plurality of modules are housed in one housing are both systems.

[0192] Further, for example, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configurations described as a plurality of devices (or processing units) above may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, if the overall configuration and operation of the system are substantially the same, a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or another processing unit).

[0193] Further, for example, the present technology can adopt a cloud computing configuration in which one function is shared and jointly processed by a plurality of devices via a network.

[0194] Further, for example, the above-described program can be executed on any device. In that case, it suffices if the device has the necessary functions (function blocks, etc.) and can obtain the necessary information.

[0195] Further, for example, each step described in the above flowchart can be executed by one device or can be shared and executed by a plurality of devices. Furthermore, when a plurality of processes are included in one step, the plurality of processes included in that one step can be executed by one device or can be shared and executed by a plurality of devices. In other words, the plurality of processes included in one step can be executed as the processes of a plurality of steps. Conversely, the processes described as a plurality of steps can be combined and executed as one step.

[0196] Note that the program executed by the computer may be such that the processing of the steps of describing the program is executed in time series along the order described in this specification, or may be executed in parallel, or may be executed individually at a necessary timing such as when a call is made. That is, as long as there is no contradiction, the processing of each step may be executed in an order different from the above-described order. Further, the processing of the steps of describing this program may be executed in parallel with the processing of other programs, or may be executed in combination with the processing of other programs.

[0197] Note that the present technology described multiple times in this specification can be implemented independently and individually as long as there is no contradiction. Of course, any plurality of the present technologies can also be implemented in combination. For example, part or all of the present technology described in any one of the embodiments can also be implemented in combination with part or all of the present technology described in other embodiments. Further, part or all of any of the above-described present technologies can also be implemented in combination with other technologies not described above.

[0198] <Example of configuration combination> Note that the present technology can also have the following configuration. (1) A decoding unit that decodes a bitstream including a parameter set in which the maximum number of parameter sets referred to as an adaptive loop filter is defined as a fixed value, and generates a decoded image, A filter unit that applies the adaptive loop filter to the decoded image generated by the decoding unit with reference to the parameter set decoded by the decoding unit An image processing apparatus comprising: (2) The maximum number is a fixed value that does not depend on the number of tiles for dividing a slice The image processing apparatus according to (1) above. (3) The maximum number is a fixed value defined for each level The image processing apparatus according to (1) or (2) above. (4) The parameter set is an APS (Adaptation parameter set). The image processing apparatus according to any one of (1) to (3) above. (5) The maximum number is a fixed value defined for each aps_params_type. The image processing apparatus according to any one of (1) to (4) above. (6) The maximum number is a fixed value defined for each aps_params_type and different for each level. The image processing apparatus according to any one of (1) to (5) above. (7) A decoding step of decoding a bit stream including a parameter set in which the maximum number of parameter sets referred to as an adaptive loop filter is defined as a fixed value to generate a decoded image; Applying the adaptive loop filter to the decoded image generated in the decoding step with reference to the parameter set decoded in the decoding step; An image processing method including the above. (8) A setting unit that sets a parameter set in which the maximum number of parameter sets referred to as an adaptive loop filter is defined as a fixed value; An encoding unit that encodes an image and generates a bit stream including the parameter set set by the setting unit; An image processing apparatus including the above. (9) The apparatus further includes a filter unit that applies the adaptive loop filter to a local encoded image when encoding is performed in the encoding unit, The encoding unit encodes the image using the filtered image to which the adaptive loop filter has been applied by the filter unit. The image processing apparatus according to (8) above. (10) A setting step of setting a parameter set in which the maximum number of parameter sets referred to as an adaptive loop filter is defined as a fixed value, An encoding step of encoding an image to generate a bitstream including the parameter set set in the setting step, and An image processing method including.

[0199] Note that the present embodiment is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present disclosure. Also, the effects described in this specification are merely examples and are not limiting, and there may be other effects.

Description of Signs

[0200] 11 Image processing system, 12 Image encoding device, 13 Image decoding device, 21 Setting unit, 22 Encoding unit, 23 Filter unit, 24 Database, 25 Decoding unit, 26 Filter unit, 27 Database

Claims

1. A filter unit that applies the adaptive loop filter to a locally decoded image generated by local decoding when encoding an image according to a parameter set defined as a fixed value defined for each aps_params_type, which is the type of APS (Adaptation Parameter Set), where the maximum number of parameter sets referred to as the adaptive loop filter is defined An image processing apparatus comprising the same.

2. The maximum number is a fixed value that does not depend on the number of tiles that divide a slice The image processing apparatus according to Claim 1.

3. The maximum number is a fixed value defined for each level The image processing apparatus according to Claim 1.

4. The maximum number is a fixed value defined for each aps_params_type and different for each level The image processing apparatus according to Claim 1.

5. Applying the adaptive loop filter to a locally decoded image generated by local decoding when encoding an image according to a parameter set defined as a fixed value defined for each aps_params_type, which is the type of APS (Adaptation Parameter Set), where the maximum number of parameter sets referred to as the adaptive loop filter is defined An image processing method including the same.

Images