Decoding device, encoding device, decoding method, and encoding method

Abstract

A circuit included in this decoding device: acquires, from a bit stream, first presence flag information indicating whether or not first video usability information is described in the bit stream; acquires the first video usability information from the bit stream if the value of the first presence flag information is true; acquires, from the bit stream, first extension flag information indicating whether or not second video usability information is described in the bit stream; acquires the second video usability information from the bit stream if the value of the first extension flag information is true; acquires, from the bit stream, second extension flag information indicating whether or not the bit stream includes a description of first extension information for extending other information of a Network Abstraction Layer (NAL) unit in which the second video usability information is described; and acquires the first extension information from the bit stream if the value of the second extension flag information is true.

Description

Decoding device, encoding device, decoding method, and encoding method

[0001] The present disclosure relates to a decoding device, an encoding device, a decoding method, and an encoding method.

[0002] Patent Literature 1 discloses an image processing system according to the background art. The image processing system includes an encoder (encoding device) and a decoder (decoding device). Input images of various modalities are input to the encoding device. The encoding device extracts features from the input images and inputs the extracted features to the decoding device. The decoding device performs an image analysis task based on the input features, thereby outputting a segmentation map.

[0003] However, in the background art, no consideration has been given to a method for extending VUI (video usability information) parameters in encoding devices and decoding devices compatible with the existing video encoding standards AVC (H.264) or HEVC (H.265).

[0004] US Patent Application Publication No. 2024 / 0046453

[0005] An object of the present disclosure is to provide a decoding device, an encoding device, a decoding method, and an encoding method that are capable of appropriately extending VUI parameters in existing video encoding standards.

[0006] A decoding device according to one aspect of the present disclosure comprises a circuit and a memory connected to the circuit, wherein the circuit acquires from the bitstream first presence flag information indicating whether first video usability information is described in the bitstream, and if the first presence flag information is a true value, acquires the first video usability information from the bitstream; acquires from the bitstream first extension flag information indicating whether second video usability information is described in the bitstream, and if the first extension flag information is a true value, acquires the second video usability information from the bitstream; acquires from the bitstream second extension flag information indicating whether first extension information that extends other information of a NAL (Network Abstraction Layer) unit in which the second video usability information is described is described in the bitstream; and if the second extension flag information is a true value, acquires the first extension information from the bitstream.

[0007] FIG. 1 is a simplified diagram showing the configuration of an image processing system according to an embodiment of the present disclosure. FIG. 2 is a simplified diagram showing the configuration of a circuit included in an encoding device. FIG. 3 is a flowchart showing processing executed by a circuit included in an encoding device. FIG. 4 is a simplified diagram showing the configuration of a bitstream. FIG. 5 is a simplified diagram showing an example of the syntax of VUI parameters. FIG. 6 is a diagram showing an example of the correspondence between values ​​of vui_modality_type and image types. FIG. 7 is a simplified diagram showing an example of the syntax of VUI parameters. FIG. 8 is a simplified diagram showing an example of the syntax of VUI parameters. FIG. 9 is a diagram showing an example of the correspondence between index values ​​and wavelength information. FIG. 10 is a simplified diagram showing an example of the syntax of VUI parameters. FIG. 11 is a simplified diagram showing an example of the syntax of VUI parameters. FIG. 12 is a diagram showing an example of the correspondence between values ​​of representation type information and interpretations of each value. FIG. 13 is a simplified diagram showing an example of the syntax of VUI parameters. FIG. 14 is a diagram showing an example of the correspondence between values ​​of interpolation information and interpretations of each value. FIG. 15 is a diagram showing an example of the configuration of a bitstream. 1 is a diagram showing, in a simplified form, the configuration of a circuit included in a decoding device. FIG. 2 is a flowchart showing processing executed by a circuit included in a decoding device. FIG. 3 is a diagram showing, in a simplified form, an example of table information. FIG. 4 is a diagram showing, in a simplified form, an example of table information. FIG. 5 is a block diagram showing, in a simplified form, an example of the functional configuration of an encoding unit. FIG. 6 is a diagram showing, in a simplified form, an example of the hierarchical structure of data in a stream. FIG. 7 is a block diagram showing, in a simplified form, a first example of the syntax of SPS in HEVC. FIG. 8 is a diagram showing, in a simplified form, an example of the syntax of first video usability information in HEVC or AVC. FIG. 9 is a diagram showing, in a simplified form, a second example of the syntax of SPS in HEVC. FIG. 10 is a diagram showing, in a simplified form, a first example of the syntax of SPS in AVC. FIG. 11 is a diagram showing, in a simplified form, a second example of the syntax of SPS in AVC. 1A and 1B are simplified diagrams illustrating a first example of syntax for secondary video usability information in HEVC or AVC, and a second example of syntax for secondary video usability information in HEVC or AVC.A simplified diagram showing a third example of the syntax of the second video usability information in HEVC or AVC.

[0008] (Findings underlying the present disclosure) An image processing system according to the background art includes an encoding device and a decoding device. Input images of various modalities are input to the encoding device. The encoding device extracts features from the input images and inputs the extracted features to the decoding device. The decoding device performs an image analysis task based on the input features, thereby outputting a segmentation map.

[0009] The task processing performed by the decoding device includes human vision and machine tasks. Human vision is the viewing or viewing of video images by a human, such as an operator or user. Machine tasks include various types of task processing using an AI model, such as object detection, object tracking, object segmentation, action recognition, or pose estimation.

[0010] In the background art, image-related information such as modality information indicating an image type is not transmitted from an encoding device to a decoding device. Therefore, an optimal task process corresponding to the image type of an input image may not be selected, resulting in reduced accuracy in the execution of the task process. Therefore, in the present disclosure, an encoding device transmits image-related information such as modality information to a decoding device by including it in a bitstream, and the decoding device executes the optimal task process according to the image type indicated by the modality information, thereby improving the execution accuracy.

[0011] The header area of ​​a bitstream includes video usability information (VUI), and the VUI parameters provide information about image characteristics such as the image aspect ratio. With the increasing number of video applications, the next-generation video coding standard, VVC (H.266), is considering extending image-related information such as modality information into the VUI parameters. However, existing video coding standards, AVC (H.264) and HEVC (H.265), do not allow direct extension of VUI parameters. Therefore, encoding devices and decoding devices compatible with AVC or HEVC cannot properly process image-related information such as modality information extended to the VUI parameters.

[0012] In order to solve such problems, the present inventors have found that the above problems can be solved by appropriately hierarchizing the syntax of SPS (Sequence Parameter Set) using extension flag information, and have come up with the present disclosure.

[0013] Next, each aspect of the present disclosure will be described.

[0014] A decoding device according to a first aspect of the present disclosure comprises a circuit and a memory connected to the circuit, wherein the circuit acquires, from a bitstream, first presence flag information indicating whether or not first video usability information is described in the bitstream, and if the first presence flag information is a true value, acquires the first video usability information from the bitstream; acquires, from the bitstream, first extension flag information indicating whether or not second video usability information is described in the bitstream, and if the first extension flag information is a true value, acquires the second video usability information from the bitstream; acquires, from the bitstream, second extension flag information indicating whether or not first extension information that extends other information of a NAL (Network Abstraction Layer) unit in which the second video usability information is described is described in the bitstream; and if the second extension flag information is a true value, acquires the first extension information from the bitstream.

[0015] According to the first aspect, the video usability information can be appropriately extended by the first extension flag information and the second video usability information, and other information of the NAL unit in which the second video usability information is described can be appropriately extended by the second extension flag information and the first extension information.

[0016] In the decoding device of the second aspect of the present disclosure, in the first aspect, the circuit may obtain the first presence flag information, the first video usability information, the first extension flag information, the second video usability information, the second extension flag information, and the first extension information from the same NAL unit.

[0017] According to the second aspect, it is possible to support HEVC (H.265), which is an existing video encoding standard.

[0018] In the decoding device of the third aspect of the present disclosure, in the second aspect, the circuit may obtain the second video usability information from the NAL unit when the first extension flag information is a true value and the first presence flag information is a true value.

[0019] According to the third aspect, when the first presence flag information is a false value (i.e., when the bitstream does not include the first video usability information), the amount of coding can be reduced by not including the second video usability information in the bitstream.

[0020] In a decoding device according to a fourth aspect of the present disclosure, in a first aspect, the circuit may obtain the first presence flag information and the first video usability information from a first NAL unit, and obtain the first extension flag information, the second video usability information, the second extension flag information, and the first extension information from a second NAL unit.

[0021] According to the fourth aspect, it is possible to support AVC (H.264), which is an existing video encoding standard.

[0022] In the decoding device of the fifth aspect of the present disclosure, in the fourth aspect, the circuit obtains second presence flag information from the second NAL unit, which indicates whether the first video usability information is described in the bitstream, and obtains the second video usability information from the second NAL unit when the first extension flag information is a true value and the second presence flag information is a true value.

[0023] According to the fifth aspect, when the second presence flag information is a false value (i.e., when the first video usability information is not included in the bitstream), the amount of coding can be reduced by not including the second video usability information in the bitstream.

[0024] In the decoding device of the sixth aspect of the present disclosure, in the first aspect, the circuit obtains image-related information related to images included in the bitstream from the second video usability information, and when second extension information that further extends the second video usability information is described in the second video usability information, obtains size information indicating the size of the second extension information from the second video usability information, and obtains the second extension information from the second video usability information based on the size information.

[0025] According to the sixth aspect, the video usability information can be further extended by the second extension information, and the second extension information can be appropriately acquired from the second video usability information by the size information.

[0026] In the decoding device of the seventh aspect of the present disclosure, in the first aspect, the circuit obtains image-related information related to images included in the bitstream from the second video usability information, obtains third extension flag information from the second video usability information indicating whether second extension information that further extends the second video usability information is described in the second video usability information, and obtains the second extension information from the second video usability information when the third extension flag information is a true value.

[0027] According to the seventh aspect, the video usability information can be further extended by the third extension flag information and the second extension information.

[0028] In the decoding device according to the eighth aspect of the present disclosure, in the first aspect, the first extension information may have a bit string that continues up to the end bit of the NAL unit that stores the first extension information.

[0029] According to the eighth aspect, the first extension information can be easily and appropriately acquired from the NAL unit that stores the first extension information.

[0030] A decoding device according to a ninth aspect of the present disclosure is, in the seventh aspect, preferably, wherein the second extension information has a bit string that continues up to the end bit of the NAL unit that stores the second extension information.

[0031] According to the ninth aspect, the second extension information can be easily and appropriately acquired from the NAL unit that stores the second extension information.

[0032] In the decoding device of the tenth aspect of the present disclosure, in the first aspect, the second video usability information includes image-related information related to images contained in the bitstream, and in the decoding device that cannot interpret the image-related information, if the first extension flag information is a true value, the circuit may ignore the second video usability information, the second extension flag information, and the first extension information.

[0033] According to the tenth aspect, the second video usability information, the second extension flag information, and the first extension information are ignored, thereby making it possible to prevent the decoding device from performing unnecessary processing.

[0034] In the decoding device of the eleventh aspect of the present disclosure, in the first aspect, the second video usability information includes image-related information related to an image contained in the bitstream, and in the decoding device that can interpret the image-related information but cannot interpret the first extension information, if the first extension flag information is a true value, the circuit obtains the second video usability information and the second extension flag information from the bitstream, and if the second extension flag information is a true value, the circuit ignores the first extension information.

[0035] According to the eleventh aspect, the first extension information is ignored, thereby making it possible to prevent the decoding device from performing unnecessary processing.

[0036] A coding device according to a twelfth aspect of the present disclosure comprises a circuit and a memory connected to the circuit, wherein the circuit encodes into the bitstream first presence flag information indicating whether or not first video usability information is described in the bitstream, and if the first presence flag information is a true value, encodes the first video usability information into the bitstream, encodes into the bitstream first extension flag information indicating whether or not second video usability information is described in the bitstream, and if the first extension flag information is a true value, encodes into the bitstream the second video usability information, encodes into the bitstream second extension flag information indicating whether or not first extension information that extends other information of a NAL (Network Abstraction Layer) unit in which the second video usability information is described is described in the bitstream, and if the second extension flag information is a true value, encodes into the bitstream the first extension information.

[0037] According to the twelfth aspect, the video usability information can be appropriately extended by the first extension flag information and the second video usability information. Also, the second extension flag information and the first extension information can be appropriately extended to other information of the NAL unit in which the second video usability information is described.

[0038] In the encoding device of the thirteenth aspect of the present disclosure, in the twelfth aspect, the circuit may encode the first presence flag information, the first video usability information, the first extension flag information, the second video usability information, the second extension flag information, and the first extension information into the same NAL unit.

[0039] According to the thirteenth aspect, it is possible to support HEVC (H.265), which is an existing video encoding standard.

[0040] In the encoding device of the 14th aspect of the present disclosure, in the 13th aspect, the circuit may encode the second video usability information into the NAL unit when the first extension flag information is a true value and the first presence flag information is a true value.

[0041] According to the fourteenth aspect, when the first presence flag information is a false value (i.e., when the bitstream does not include the first video usability information), the amount of coding can be reduced by not including the second video usability information in the bitstream.

[0042] In the encoding device of the 15th aspect of the present disclosure, in the 12th aspect, the circuit may encode the first presence flag information and the first video usability information into a first NAL unit, and encode the first extension flag information, the second video usability information, the second extension flag information, and the first extension information into a second NAL unit.

[0043] According to the fifteenth aspect, it is possible to support AVC (H.264), which is an existing video encoding standard.

[0044] In the encoding device of the 16th aspect of the present disclosure, in the 15th aspect, the circuit encodes second presence flag information indicating whether the first video usability information is described in the bitstream into the second NAL unit, and when the first extension flag information is a true value and the second presence flag information is a true value, encodes the second video usability information into the second NAL unit.

[0045] According to the 16th aspect, when the second presence flag information is a false value (i.e., when the first video usability information is not included in the bitstream), the amount of coding can be reduced by not including the second video usability information in the bitstream.

[0046] In the encoding device of the 17th aspect of the present disclosure, in the 12th aspect, the circuit encodes image-related information related to an image included in the bitstream into the second video usability information, and when second extension information that further extends the second video usability information is described in the second video usability information, encodes size information indicating the size of the second extension information into the second video usability information, and encodes the second extension information into the second video usability information based on the size information.

[0047] According to the seventeenth aspect, the second video usability information can be further extended by the second extension information, and the size information allows the decoding device to appropriately acquire the second extension information from the second video usability information.

[0048] In the encoding device of the 18th aspect of the present disclosure, in the 12th aspect, the circuit encodes image-related information related to an image included in the bitstream into the second video usability information, encodes third extension flag information into the second video usability information indicating whether second extension information that further extends the second video usability information is described in the second video usability information, and if the third extension flag information is a true value, encodes the second extension information into the second video usability information.

[0049] According to the eighteenth aspect, the second video usability information can be further extended by the third extension flag information and the second extension information.

[0050] In the encoding device according to the 19th aspect of the present disclosure, in the 12th aspect, the first extension information may have a bit string that continues up to the end bit of the NAL unit that stores the first extension information.

[0051] According to the nineteenth aspect, the first extension information can be simply and appropriately encoded into the NAL unit storing the first extension information.

[0052] In the encoding device according to the 20th aspect of the present disclosure, in the 18th aspect, the second extension information may have a bit string that continues up to the end bit of the NAL unit that stores the second extension information.

[0053] According to the twentieth aspect, the second extension information can be simply and appropriately encoded into the NAL unit storing the second extension information.

[0054] In the encoding device of the 21st aspect of the present disclosure, in the 12th aspect, the second video usability information includes image-related information related to images included in the bitstream, and in the encoding device that cannot interpret the image-related information, the circuit encodes the first extension flag information that is a false value into the bitstream, and does not encode the second video usability information, the second extension flag information, and the first extension information into the bitstream.

[0055] According to the twenty-first aspect, the second video usability information, the second extension flag information, and the first extension information are not coded, thereby making it possible to prevent the coding device from performing unnecessary processing.

[0056] In the encoding device of the 22nd aspect of the present disclosure, in the 12th aspect, the second video usability information includes image-related information related to images included in the bitstream, and in the encoding device that can interpret the image-related information but cannot interpret the first extended information, the circuit encodes the first extended flag information that is a true value, the second extended flag information that is a false value, and the second video usability information into the bitstream, and does not encode the first extended information into the bitstream.

[0057] According to the twenty-second aspect, the first extension information is not coded, so that the coding device can avoid performing unnecessary processing.

[0058] A decoding method according to a 23rd aspect of the present disclosure includes a decoding device that acquires, from a bitstream, first presence flag information indicating whether or not first video usability information is described in the bitstream, and acquires the first video usability information from the bitstream if the first presence flag information is a true value; acquires, from the bitstream, first extension flag information indicating whether or not second video usability information is described in the bitstream, and acquires, from the bitstream if the first extension flag information is a true value, the second video usability information from the bitstream; acquires, from the bitstream, second extension flag information indicating whether or not first extension information that extends other information of a NAL (Network Abstraction Layer) unit in which the second video usability information is described is described in the bitstream, and acquires, from the bitstream if the second extension flag information is a true value, the first extension information from the bitstream.

[0059] According to the 23rd aspect, the video usability information can be appropriately extended by the first extension flag information and the second video usability information. Also, the second extension flag information and the first extension information can be appropriately extended to other information of the NAL unit in which the second video usability information is described.

[0060] An encoding method according to a 24th aspect of the present disclosure includes an encoding device encoding, into a bitstream, first presence flag information indicating whether or not first video usability information is described in the bitstream, and if the first presence flag information is a true value, encoding the first video usability information into the bitstream; encoding, into the bitstream, first extension flag information indicating whether or not second video usability information is described in the bitstream, and if the first extension flag information is a true value, encoding the second video usability information into the bitstream; encoding, into the bitstream, second extension flag information indicating whether or not first extension information that extends other information of a NAL (Network Abstraction Layer) unit in which the second video usability information is described is described in the bitstream; and if the second extension flag information is a true value, encoding the first extension information into the bitstream.

[0061] According to the 24th aspect, the video usability information can be appropriately extended by the first extension flag information and the second video usability information. Also, the second extension flag information and the first extension information can be appropriately extended to other information of the NAL unit in which the second video usability information is described.

[0062] (Embodiments of the Present Disclosure) Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that elements with the same reference numerals in different drawings indicate the same or corresponding elements.

[0063] Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, components, steps, and step orders shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not recited in the independent claims that represent the highest concepts are described as optional components. Furthermore, in all embodiments, the respective contents can be substituted or combined. Note that these general or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or by any combination of a system, a method, an integrated circuit, a computer program, or a recording medium.

[0064] 1 is a diagram showing a simplified configuration of an image processing system according to an embodiment of the present disclosure. The image processing system includes an encoding device 1, a decoding device 2, and a transmission path NW.

[0065] Image data D1 is input from an external device to the encoding device 1. The external device may include a camera or the like that captures moving images. The external device inputs image data D1 of the captured moving images to the encoding device 1.

[0066] The encoding device 1 generates a bit stream BS based on the image data D1. Here, a bit stream refers to a data string or flow of digital data. The bit stream (or simply stream) may be a single stream, or may be composed of multiple streams divided into multiple layers. Furthermore, the bit stream may be transmitted by serial communication over a single transmission path, or by packet communication over multiple transmission paths. The encoding device 1 transmits the generated bit stream BS to the decoding device 2 via the transmission path NW. The decoding device 2 receives the bit stream BS.

[0067] The decoding device 2 decodes image data D1 from the bitstream BS and performs task processing based on the decoded image data D1. The task processing includes human vision and machine tasks. Human vision is the viewing or viewing of moving images by a human being such as an operator or a user. Machine tasks include various types of task processing such as object detection, object tracking, object segmentation, action recognition, or pose estimation using an artificial intelligence (AI) model, which is a machine-learned estimation model. The task processing unit that performs human vision includes a display device such as a liquid crystal display or an organic EL display. The task processing unit that performs machine tasks includes an inference device using AI.

[0068] The transmission path NW may be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof. The transmission path NW is preferably a private network or the like in which secure communication is ensured by access restrictions.

[0069] The encoding device 1 includes a circuit 11 and a memory 12 connected to the circuit 11. The circuit 11 includes a processor such as a CPU. The memory 12 includes any recording medium such as a ROM, RAM, HDD, SSD, or semiconductor memory. The memory 12 stores data to be processed by the circuit 11 or data in the middle of processing.

[0070] The decoding device 2 includes a circuit 21 and a memory 22 connected to the circuit 21. The circuit 21 includes a processor such as a CPU. The memory 22 includes any recording medium such as a ROM, a RAM, a HDD, an SSD, or a semiconductor memory. The memory 22 stores data to be processed by the circuit 21 or data in the middle of processing.

[0071] 2 is a simplified diagram showing the configuration of the circuit 11 included in the encoding device 1. The circuit 11 includes an acquisition unit 31, a setting unit 32, an encoding unit 33, and a transmission unit .

[0072] Next, the encoding unit 33 according to this embodiment will be described. Fig. 21 is a block diagram showing an example of the functional configuration of the encoding unit 33 according to this embodiment. The encoding unit 33 encodes an image in units of blocks.

[0073] 21 , the encoding unit 33 includes a dividing unit 102, a subtraction unit 104, a transform unit 106, a quantization unit 108, an entropy encoding unit 110, an inverse quantization unit 112, an inverse transform unit 114, an addition unit 116, a block memory 118, a loop filter 120, a frame memory 122, an intra prediction unit 124, an inter prediction unit 126, a prediction control unit 128, and a prediction parameter generation unit 130. Note that the intra prediction unit 124 and the inter prediction unit 126 are configured as part of a prediction processing unit 125.

[0074] For example, the components of the encoding unit 33 shown in FIG. 21 are implemented by the circuit 11 and memory 12 shown in FIG.

[0075] The circuit 11 is configured to include a processor such as a CPU. The circuit 11 may be a dedicated or general-purpose electronic circuit for encoding images, or may be a collection of multiple electronic circuits. Furthermore, for example, the circuit 11 may fulfill the roles of multiple components of the encoding unit 33 shown in FIG. 21 , excluding the component for storing information.

[0076] The memory 12 may be a dedicated or general-purpose electronic circuit for storing information, or may be a collection of multiple electronic circuits. The memory 12 may be externally connected to the circuit 11 or may be built into the circuit 11. The memory 12 may be a magnetic disk, an optical disk, or the like, and may be expressed as a storage or a recording medium, or the like. The memory 12 may be a non-volatile memory or a volatile memory.

[0077] The memory 12 may store an image to be encoded, or a stream corresponding to the encoded image, or may store a program for the processor to execute the image encoding process.

[0078] Furthermore, the memory 12 may serve as a component for storing information among the multiple components included in the encoding unit 33 shown in Fig. 21. Specifically, the memory 12 may serve as the block memory 118 and the frame memory 122 shown in Fig. 21. More specifically, the memory 12 may store a reconstructed image (specifically, a reconstructed block or a reconstructed picture, etc.).

[0079] 21 may be omitted from the encoding unit 33, or some of the processes executed by the components may be omitted. Alternatively, some of the components shown in FIG. 21 may be implemented in another device, or some of the processes executed by the components may be executed by another device.

[0080] FIG. 3 is a flowchart showing the process executed by the circuit 11 included in the encoding device 1.

[0081] First, in step SP11, the acquisition unit 31 acquires image data D11 representing the image Q to be processed, which is input from an external device. The image data D11 corresponds to the image data D1 shown in FIG.

[0082] Next, in step SP12, the setting unit 32 sets parameters P in association with the image Q. The parameters P include a first parameter P1, a second parameter P2, a third parameter P3, and a fourth parameter P4. The first parameter P1 includes modality information indicating the image type of the image Q. The second parameter P2 includes wavelength information indicating the wavelength band of the optical radiation of the image Q. The third parameter P3 includes color information of the image Q. The fourth parameter P4 includes extension bit information corresponding to the spare image type when the modality information indicates that the image type is a spare image type for extension. The setting unit 32 inputs data D12 including the setting information of the parameters P to the encoding unit 33.

[0083] The first parameter P1 includes modality information indicating the image type of the image Q. The image type includes, for example, at least one of a visible light image, an infrared image, and an ultraviolet image. The visible light image includes a natural image or an RGB image, and is used to provide detailed color information in human vision or machine tasks. The infrared image includes an image captured using an infrared camera, and is used to capture images in the dark. The ultraviolet image includes an image captured using an ultraviolet camera, and is used to detect scratches on products. The modality information indicates that the image type is at least one of a visible light image, an infrared image, and an ultraviolet image. The modality information also indicates that the image type is at least one of an undefined image type and a spare image type for extension. In other words, the modality information indicates that the image type is at least one of a visible light image, an infrared image, an ultraviolet image, an undefined image type, and a spare image type for extension. Note that the modality information may include other image types. The setting unit 32 may set the first parameter P1 by image analysis based on the image data D11, or may set the first parameter P1 based on setting information input by an operator of the encoding device 1.

[0084] Next, in step SP13, the encoding unit 33 encodes the image Q represented by the image data D11 input from the acquisition unit 31 into a bit stream BS.

[0085] Next, in step SP14, the encoding unit 33 encodes the parameters P indicated by the data D12 input from the setting unit 32 into a bit stream BS. Here, encoding the parameters P into the bit stream BS may be rephrased as saving the parameters P in the bit stream BS or storing the parameters P in the bit stream BS. Note that the order of execution of steps SP13 and SP14 may be reversed from the example in FIG. 3, or steps SP13 and SP14 may be executed simultaneously.

[0086] Next, in step SP15, the transmitting unit 34 transmits the bit stream BS input from the encoding unit 33 to the decoding device 2 via the transmission path NW.

[0087] 4 is a simplified diagram showing the structure of a bitstream BS. The bitstream BS has a header area 41 and a payload area 42. The encoding unit 33 stores encoded data of an image Q in the payload area 42, and stores encoded data of a parameter P associated with the image Q in the header area 41.

[0088] The encoding unit 33 may encode the encoded data of the parameter P in a predetermined area 43 in the header area 41. The predetermined area 43 may be VUI (video usability information) or SEI (supplemental enhancement information). However, the predetermined area 43 is not limited to VUI or SEI, and may also be VPS, SPS, PPS, PH, SH, APS, a tile header, a system layer header, or the like.

[0089] 22 is a diagram showing an example of a hierarchical structure of data in a stream. The stream includes, for example, a video sequence. As shown in (A) of FIG. 22, the video sequence includes, for example, a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), supplemental enhancement information (SEI), and multiple pictures.

[0090] In a video composed of multiple layers, the VPS includes coding parameters common to multiple layers and coding parameters related to multiple layers or individual layers included in the video.

[0091] The SPS includes parameters used for a sequence, i.e., encoding parameters that the decoding device 2 refers to in order to decode the sequence. The encoding parameters may indicate, for example, the width or height of a picture. Note that there may be multiple SPSs.

[0092] The PPS includes parameters used for a picture, i.e., encoding parameters that the decoding device 2 references to decode each picture in a sequence. The encoding parameters may include, for example, a reference value of the quantization width used to decode the picture and a flag indicating the application of weighted prediction. Note that there may be multiple PPSs. The SPS and PPS may also be simply referred to as parameter sets.

[0093] As shown in (B) of Fig. 22, a picture includes a picture header and one or more slices. The picture header includes coding parameters that are referenced by the decoding device 2 to decode the one or more slices.

[0094] As shown in Fig. 22C, a slice includes a slice header and one or more bricks. The slice header includes coding parameters that are referenced by the decoding device 2 to decode the one or more bricks.

[0095] As shown in FIG. 22(D), a brick includes one or more CTUs (Coding Tree Units).

[0096] Note that a picture may not contain slices, but may contain tile groups instead, where each tile group contains one or more tiles. Also, a brick may contain slices.

[0097] A CTU is also called a superblock or a basic division unit. As shown in (E) of Fig. 22, a CTU includes a CTU header and one or more coding units (CUs). The CTU header includes coding parameters that the decoding device 2 references to decode the one or more CUs.

[0098] A CU may be divided into multiple smaller CUs. Furthermore, as shown in (F) of FIG. 22 , a CU includes a CU header, prediction information, and residual coefficient information. The prediction information is information for predicting a CU. The residual coefficient information is information indicating a prediction residual. A CU is basically the same as a PU (Prediction Unit) or a TU (Transform Unit), but may include multiple TUs smaller than a CU. A CU may be processed for each VPDU (Virtual Pipeline Decoding Unit) that constitutes the CU. A VPDU is a fixed unit that can be processed in one stage, for example, when performing pipeline processing in hardware.

[0099] Note that a stream may not have some of the layers among the layers shown in Fig. 22. Furthermore, the order of these layers may be changed, and any layer may be replaced with another layer.

[0100] A picture currently being processed by a device such as the encoding device 1 or the decoding device 2 is called a current picture. If the processing is encoding, the current picture is synonymous with a picture to be encoded, and if the processing is decoding, the current picture is synonymous with a picture to be decoded. Furthermore, a block (CU or a block of a CU) currently being processed by a device such as the encoding device 1 or the decoding device 2 is called a current block. If the processing is encoding, the current block is synonymous with a block to be encoded, and if the processing is decoding, the current block is synonymous with a block to be decoded.

[0101] FIG. 5 is a simplified diagram showing an example of the syntax of VUI parameters including a first parameter P1 and a third parameter P3.

[0102] The first parameter P1 includes flag information (vui_modality_info_present_flag) indicating whether modality information is described in the bitstream BS. When the value of vui_modality_info_present_flag is 1, it indicates that modality information is present in the VUI parameters, and when the value of vui_modality_info_present_flag is 0, it indicates that modality information is not present in the VUI parameters. When vui_modality_info_present_flag indicates that modality information is described in the bitstream BS, the modality information is represented as the value of the identifier vui_modality_type included in the VUI parameters.

[0103] FIG. 6 is a diagram showing an example of the correspondence between the value of vui_modality_type and the image type. As shown in FIG. 6, a value of 0 for vui_modality_type indicates that the image type of image Q is not defined (i.e., undefined). A value of 1 for vui_modality_type indicates that the image type of image Q is a visible light image. A value of 2 for vui_modality_type indicates that the image type of image Q is an infrared image. A value of 3 for vui_modality_type indicates that the image type of image Q is an ultraviolet image. Other values ​​of vui_modality_type (4-255 in this example) indicate a spare area reserved for future extended use. Note that the number of image types defined in FIG. 6 may be increased depending on the image Q to be processed. Furthermore, the absence of vui_modality_type may mean that the value of vui_modality_type is 0. Furthermore, when the value of vui_modality_type is 0, wavelength information indicating a wide wavelength band spanning multiple image types, such as a hyperspectral image, may be defined by user settings or the like.

[0104] As shown in Fig. 5, the third parameter P3 includes flag information (vui_colour_description_present_flag) indicating whether color information is described in the bitstream BS. A value of 1 in vui_colour_description_present_flag indicates that color information is present in the VUI parameters, and a value of 0 in vui_colour_description_present_flag indicates that color information is not present in the VUI parameters. Although not shown, the color information includes vui_colour_primaries, vui_transfer_characteristics, vui_matrix_coeffs, etc. Note that if the image type is an infrared image or an ultraviolet image, the third parameter P3 does not need to include color information. In the encoding process, if the image type indicated by the modality information is a visible light image, the encoding device 1 sets the value of vui_colour_description_present_flag to 1 and describes color information in the VUI parameters, whereas if the image type indicated by the modality information is an infrared image or an ultraviolet image, the encoding device 1 sets the value of vui_colour_description_present_flag to 0 and does not describe color information in the VUI parameters. Furthermore, in the decoding process, if the image type indicated by the modality information is a visible light image, the decoding device 2 acquires color information from the bitstream BS, whereas if the image type indicated by the modality information is an infrared image or an ultraviolet image, it is not necessary to acquire color information from the bitstream BS, or even if color information is acquired from the bitstream BS, the value of the color information may be ignored.

[0105] FIG. 7 is a simplified diagram showing an example of the syntax of VUI parameters including the fourth parameter P4.

[0106] The fourth parameter P4 includes extension bit information corresponding to the reserved image type when the modality information indicates that the image type is a reserved image type for extension. The extension bit information includes information indicating the bit length of the reserved image type (vui_modality_type_extension_bits) and information indicating the content of the reserved image type (vui_reserved_modality_type_extension). A value of vui_modality_type_extension_bits of 0 indicates that vui_reserved_modality_type_extension is not described in the bitstream BS. On the other hand, a value of vui_modality_type_extension_bits greater than 0 indicates that vui_reserved_modality_type_extension is described in the bitstream BS and its bit length. The maximum value of vui_modality_type_extension_bits can be set arbitrarily, for example, to 2048.

[0107] 8 is a simplified diagram showing a first example of the syntax of VUI parameters including a second parameter P2, which includes wavelength information indicating the wavelength band of the light radiation of the image Q.

[0108] 8, the second parameter P2 includes flag information (vui_spectrum_range_present_flag) indicating whether wavelength information is described in the bitstream BS. When the value of vui_spectrum_range_present_flag is 0, it indicates that the spectral band of the optical radiation wavelength represented by the CLVS (Coded Layer Video Sequence) image does not exist in the VUI parameters, and when the value of vui_spectrum_range_present_flag is 1, it indicates that the spectral band of the optical radiation wavelength represented by the CLVS image exists in the VUI parameters.

[0109] When the value of vui_spectrum_range_present_flag is 1, the wavelength information includes information indicating the minimum value (minimum wavelength) and maximum value (maximum wavelength) of the wavelength band of image Q. Both the minimum wavelength and the maximum wavelength are wavelengths in meters.

[0110] The information indicating the minimum value of the wavelength band includes information (vui_min_wavelength_mantissa) indicating the mantissa of the minimum value in exponential notation using a predetermined base (e.g., 10), and information (vui_min_wavelength_exponent_plus15) indicating the value obtained by adding 15 to the exponent of the minimum value. Here, if vui_min_wavelength_mantissa is 0 or does not exist, this means that the minimum value of the wavelength band is unknown or unspecified and is not defined, or that it is defined by a method other than exponential notation. In this case, the decoding device 2 may not acquire vui_min_wavelength_exponent_plus15 from the bitstream BS during the decoding process, or may ignore the value of vui_min_wavelength_exponent_plus15 even if it is acquired from the bitstream BS.

[0111] The information indicating the maximum value of the wavelength band includes information (vui_max_wavelength_mantissa) indicating the mantissa of the maximum value in exponential notation using a predetermined base (e.g., 10), and information (vui_max_wavelength_exponent_plus15) indicating the value obtained by adding 15 to the exponent of the maximum value. Here, if vui_max_wavelength_mantissa is 0 or does not exist, this means that the maximum value of the wavelength band is unknown or unspecified and therefore not defined, or that it is defined using a method other than exponential notation. In this case, the decoding device 2 does not need to acquire vui_max_wavelength_exponent_plus15 from the bitstream BS, and even if it acquires vui_max_wavelength_exponent_plus15 from the bitstream BS, it may ignore its value.

[0112] Both vui_min_wavelength_mantissa and vui_max_wavelength_mantissa have three or more significant digits in decimal notation. In the example shown in Figure 8, both vui_min_wavelength_mantissa and vui_max_wavelength_mantissa have a binary bit length of 10 bits, but may have any bit length of 11 bits or more. Similarly, in the example shown in Figure 8, both vui_min_wavelength_exponent_plus15 and vui_max_wavelength_exponent_plus15 have a binary bit length of 5 bits, but may have any bit length of 6 bits or more.

[0113] In the encoding device 1, the minimum value of the wavelength band indicated by vui_min_wavelength_mantissa and vui_min_wavelength_exponent_plus15 is set to be equal to or greater than the shortest wavelength corresponding to the image type indicated by the modality information. Similarly, in the encoding device 1, the maximum value of the wavelength band indicated by vui_max_wavelength_mantissa and vui_max_wavelength_exponent_plus15 is set to be equal to or less than the longest wavelength corresponding to the image type indicated by the modality information.

[0114] FIG. 9 is a simplified diagram showing a second example of the syntax of the VUI parameters including the second parameter P2.

[0115] In the example shown in FIG. 9, the second parameter P2 includes information (vui_spectrum_range) indicating an index value of the wavelength information.

[0116] 10 is a diagram showing an example of the correspondence between index values ​​and wavelength information. A minimum value (minimum wavelength) and a maximum value (maximum wavelength) of a wavelength band are predefined corresponding to each of a plurality of index values, and are shared by the encoding device 1 and the decoding device 2. In the example shown in FIG. 10, index values ​​2 to 6 correspond to the ultraviolet wavelength region, index value 7 corresponds to the visible light wavelength region, and index values ​​8 to 10 correspond to the infrared wavelength region.

[0117] For example, if the index value is 2, the minimum wavelength is 1*10 －９ (m), the maximum wavelength is 1*10 －７ (m), and if the index value is, for example, 3, the minimum wavelength is 1*10 －７ (m), the maximum wavelength is 19*10 －８ (m) is defined as:

[0118] Furthermore, when the index value is a certain value (0 in the example shown in FIG. 10), it means that the minimum wavelength and the maximum wavelength are not defined.

[0119] If the index value is another specific value (1 in the example shown in FIG. 10), this means that the desired minimum and maximum wavelengths are defined by user settings or the like.

[0120] Note that other index values ​​(11-255 in this example) indicate spare areas reserved for future extended use.

[0121] In the encoding device 1, the index value is set according to the wavelength band corresponding to the image type indicated by the modality information. For example, if vui_modality_type is 1 (visible light image), vui_spectrum_range is set to 0, 1, or 7. If vui_modality_type is 2 (infrared image), vui_spectrum_range is set to 0, 1, or any of 8 to 10. If vui_modality_type is 3 (ultraviolet image), vui_spectrum_range is set to 0, 1, or any of 2 to 6.

[0122] FIG. 11 is a simplified diagram showing a third example of the syntax of VUI parameters including the second parameter P2.

[0123] In the example shown in Figure 9, if the index value of vui_spectrum_range is set to 1 (user-defined), the minimum value of the wavelength band may be defined by vui_min_wavelength_mantissa and vui_min_wavelength_exponent_plus15, as in the example shown in Figure 8, and the maximum value of the wavelength band may be defined by vui_max_wavelength_mantissa and vui_max_wavelength_exponent_plus15.

[0124] FIG. 12 is a simplified diagram showing a first modified example of the syntax of VUI parameters.

[0125] If the image type indicated by the modality information is an infrared image, the encoding device 1 may further encode representation type information (vui_infrared_radiance_representation_type) indicating the representation type of infrared radiance into the bitstream BS.

[0126] FIG. 13 is a diagram showing an example of the correspondence between values ​​of the expression type information and interpretations of each value.

[0127] If vui_infrared_radiance_representation_type is 0 or does not exist, it means that the representation type of the infrared radiation intensity is unknown or unspecified and the representation type is not defined, or is defined by a method other than intensity value representation or temperature value representation.

[0128] If the value of vui_infrared_radiance_representation_type is 1, this means that the representation type of infrared radiation intensity is intensity value representation. In the intensity value representation, each pixel value of image Q indicates an intensity value of infrared radiation intensity.

[0129] If the value of vui_infrared_radiance_representation_type is 2, this means that the representation type of the infrared radiation intensity is temperature value representation. In the temperature value representation, each pixel value of image Q indicates a temperature value converted from the intensity value of the infrared radiation intensity.

[0130] Any other value of vui_infrared_radiance_representation_type (3 in this example) indicates a spare area reserved for future expansion.

[0131] 12, when the value of vui_infrared_radiance_representation_type is 2 (i.e., temperature value representation), the encoding device 1 describes temperature information in the VUI parameters. The temperature information includes the minimum and maximum temperatures of the temperature value, which is a floating-point value.

[0132] The information indicating the minimum temperature includes information (vui_min_temperature_sign) indicating the positive or negative sign of the minimum temperature in exponential notation using a predetermined base (for example, 2), information indicating the exponent of the minimum temperature (vui_min_temperature_exponent), information indicating the value obtained by subtracting 1 from the bit length of the mantissa of the minimum temperature (vui_min_temperature_mantissa_len_minus1), and information indicating the mantissa of the minimum temperature (vui_min_temperature_mantissa). Note that vui_min_temperature_mantissa_len_minus1 may be omitted by setting the bit length of the mantissa of the minimum temperature to a fixed value.

[0133] If the value of vui_min_temperature_sign is 0, it indicates that the sign of the minimum temperature is positive, and if the value of vui_min_temperature_sign is 1, it indicates that the sign of the minimum temperature is negative.

[0134] The information indicating the maximum temperature includes information (vui_max_temperature_sign) indicating the sign of the maximum temperature in exponential notation using a predetermined base (for example, 2), information (vui_max_temperature_exponent) indicating the exponent of the maximum temperature, information (vui_max_temperature_mantissa_len_minus1) indicating the value obtained by subtracting 1 from the bit length of the mantissa of the maximum temperature, and information (vui_max_temperature_mantissa) indicating the mantissa of the maximum temperature. Note that vui_max_temperature_mantissa_len_minus1 may be omitted by setting the bit length of the mantissa of the maximum temperature to a fixed value.

[0135] If the value of vui_max_temperature_sign is 0, it indicates that the sign of the maximum temperature is positive, and if the value of vui_max_temperature_sign is 1, it indicates that the sign of the maximum temperature is negative.

[0136] FIG. 14 is a simplified diagram showing a second modified example of the syntax of the VUI parameters.

[0137] The encoding device 1 may further encode flag information (vui_false_colour_representation_present_flag) indicating whether or not pixel values ​​of the image Q are represented using a false colour representation into the bitstream BS.

[0138] If the value of vui_false_colour_representation_present_flag is 1, it indicates that the pixel values ​​of image Q are represented using pseudo-colour representation, and if the value of vui_false_colour_representation_present_flag is 0, it indicates that the pixel values ​​of image Q are not represented using pseudo-colour representation.

[0139] When the pixel values ​​of image Q are expressed using pseudo-color representation, generation information for generating table information that associates each pixel value of image Q with a color code of the pseudo-color representation is described in the VUI parameters.

[0140] The generation information includes information (vui_num_hex_colour_code_minus2) indicating the value obtained by subtracting 2 from the number of color codes encoded in the bitstream BS, information (vui_num_value_minus2) indicating the value obtained by subtracting 2 from the number of pairs of pixel values ​​and color codes, interpolation information (vui_colour_interpolation_type) for interpolating values ​​between the pairs, and information (vui_hex_colour_code[i]) indicating the i-th hexadecimal color code. Each hexadecimal color code is 24 bits long, with the first 8 bits representing the depth of red, the middle 8 bits representing the depth of green, and the last 8 bits representing the depth of blue. For example, if vui_hex_colour_code is 000000, it indicates black (RGB: 0,0,0), and if vui_hex_colour_code is FFFFFF, it indicates white (RGB: 255,255,255).

[0141] For example, if the value of vui_num_hex_colour_code_minus2 is 0, it means that the number of colour codes coded in the bitstream BS is 2.

[0142] For example, if the value of vui_num_value_minus2 is 4, it means that the number of pairs of pixel values ​​and color codes is 6. The number of pairs being 6 means that the range of color components can be divided into 5 equal intervals.

[0143] FIG. 15 is a diagram showing an example of the correspondence between the values ​​of the interpolation information and the interpretation of each value.

[0144] If vui_colour_interpolation_type is 0 or not present, it means that the interpolation type is unknown or unspecified and not defined, or is defined by a method other than linear or bilinear interpolation.

[0145] If vui_colour_interpolation_type is 1, it means that the interpolation type is linear.

[0146] If vui_colour_interpolation_type is 2, it means that the interpolation type is bilinear interpolation.

[0147] Other values ​​of vui_colour_interpolation_type (3-15 in this example) indicate a spare area reserved for future expansion.

[0148] FIG. 16 is a diagram showing an example of the structure of the bit stream BS.

[0149] The bitstream BS is a multi-layer image layer L １ ~L ｍ 16 shows only one access unit. An access unit is the minimum processing unit of a time attribute, and corresponds to, for example, one frame of a video. An access unit is also a collection of multiple NAL units, each corresponding to, for example, one frame of a video. A bitstream BS is configured to include multiple access units that are consecutive in time.

[0150] The first image layer L, which is the lowest layer １ The payload area 42 of the image Q Ｌ１ The coded data of the second image layer L is stored. ２ The payload area 42 of the image Q Ｌ２ Similarly, the encoded data of the m-th image layer L ｍ The payload area 42 of the image Q Ｌｍ The encoded data is stored.

[0151] Image Q Ｌ１ ~Q Ｌｍ may have different image types or wavelength bands. For example, in a three-layer multi-layer structure with m=3, image Q Ｌ１ is the visible light image, and image Q Ｌ２ is the infrared image, and image Q Ｌ３The encoding device 1 generates a bitstream BS having a multi-layer structure by encoding, for example, an output signal from a multispectral camera.

[0152] Image Q Ｌ１ ~Q Ｌｍ If there is a correlation between the image layers L １ ~L ｍ Images may be referenced between images. Ｌ１ ~Q Ｌｍ If there is no correlation between the image layers L １ ~L ｍ Images do not need to be referenced between them.

[0153] Image layer L １ The header area 41 of the image Q Ｌ１ The parameter P associated with Ｌ１ The encoded data of the parameter P is stored. Ｌ１ is image Q Ｌ１ This includes modality information and wavelength information.

[0154] Image layer L ２ The header area 41 of the image Q Ｌ２ The parameter P associated with Ｌ２ The encoded data of the parameter P is stored. Ｌ２ is image Q Ｌ２ This includes modality information and wavelength information.

[0155] Image layer L ｍ The header area 41 of the image Q Ｌｍ The parameter P associated with Ｌｍ The encoded data of the parameter P is stored. Ｌｍ is image Q Ｌｍ This includes modality information and wavelength information.

[0156] 17 is a simplified diagram showing the configuration of the circuit 21 included in the decoding device 2. The circuit 21 includes a receiving unit 51, a decoding unit 52, a switching unit 53, and a plurality of n task processing units 54 (n is a natural number of 2 or more). １ ~54 ｎ The task processing unit 54 １ ~54 ｎThe tasks performed by include human vision and machine tasks, including various types of tasks using AI models, such as object detection, object tracking, object segmentation, action recognition, or pose estimation.

[0157] Next, the decoding unit 52 according to this embodiment will be described. Fig. 23 is a block diagram showing an example of the functional configuration of the decoding unit 52 according to this embodiment. The decoding unit 52 decodes a stream, which is an encoded image, in units of blocks.

[0158] 23 , the decoding unit 52 includes an entropy decoding unit 202, an inverse quantization unit 204, an inverse transform unit 206, an adder 208, a block memory 210, a loop filter 212, a frame memory 214, an intra prediction unit 216, an inter prediction unit 218, a prediction control unit 220, a prediction parameter generation unit 222, and a partition determination unit 224. Note that the intra prediction unit 216 and the inter prediction unit 218 are configured as part of a prediction processing unit 215.

[0159] For example, the components of the decoding unit 52 shown in FIG. 23 are implemented by the circuit 21 and memory 22 shown in FIG.

[0160] The circuit 21 is configured to include a processor such as a CPU. The circuit 21 may be a dedicated or general-purpose electronic circuit for decoding streams, or may be a collection of multiple electronic circuits. Furthermore, for example, the circuit 21 may fulfill the roles of multiple components of the decoding unit 52 shown in FIG. 23 , excluding the component for storing information.

[0161] The memory 22 may be a dedicated or general-purpose electronic circuit for storing information, or may be a collection of multiple electronic circuits. The memory 22 may be externally connected to the circuit 21 or may be built into the circuit 21. The memory 22 may be a magnetic disk, an optical disk, or the like, and may be expressed as a storage or a recording medium, or the like. The memory 22 may be a non-volatile memory or a volatile memory.

[0162] The memory 22 may store a stream to be decoded or a decoded image, or may store a program for the processor to execute the decoding process of the stream.

[0163] Furthermore, the memory 22 may serve as a component for storing information among the multiple components included in the decoding unit 52 shown in Fig. 23. Specifically, the memory 22 may serve as the block memory 210 and the frame memory 214 shown in Fig. 23. More specifically, the memory 22 may store a reconstructed image (specifically, a reconstructed block or a reconstructed picture, etc.).

[0164] 23 may be omitted from the decoding unit 52, or some of the processes executed by the components may be omitted. Alternatively, some of the components shown in FIG. 23 may be implemented in another device, or some of the processes executed by the components may be executed by another device.

[0165] The inverse quantization unit 204, inverse transform unit 206, addition unit 208, block memory 210, frame memory 214, intra prediction unit 216, inter prediction unit 218, prediction control unit 220, and loop filter 212 included in the decoding unit 52 shown in Figure 23 perform processing similar to that of the inverse quantization unit 112, inverse transform unit 114, addition unit 116, block memory 118, frame memory 122, intra prediction unit 124, inter prediction unit 126, prediction control unit 128, and loop filter 120 included in the encoding unit 33 shown in Figure 21.

[0166] FIG. 18 is a flowchart showing the processing executed by the circuit 21 included in the decoding device 2.

[0167] First, in step SP21, the receiving unit 51 receives the bit stream BS transmitted by the encoding device 1 from the transmission path NW.

[0168] Next, in step SP22, the decoding unit 52 obtains image Q by decoding it from the payload area 42 of the bitstream BS input from the receiving unit 51. Note that the decoding may include extraction. The decoding unit 52 outputs image data D21 of image Q. The image data D21 corresponds to the image data D11 shown in FIG. 2.

[0169] As shown in FIG. 16, the bitstream BS includes multiple image layers L １ ~L ｍ In the case where the image has a multi-layer structure including a plurality of images Q having different image types or wavelength bands, the decoding unit 52 Ｌ１ ~Q Ｌｍ multiple image layers L １ ~L ｍ You can also get it from.

[0170] Next, in step SP23, the decoding unit 52 obtains the parameters P by decoding them from the header region 41 (or a predetermined region 43 within the header region 41) of the bit stream BS input from the receiving unit 51. As described above, the parameters P include a first parameter P1, a second parameter P2, a third parameter P3, and a fourth parameter P4. The first parameter P1 includes modality information indicating the image type of the image Q. The second parameter P2 includes wavelength information indicating the wavelength band of the optical radiation of the image Q. The third parameter P3 includes color information of the image Q. The fourth parameter P4 includes extension bit information corresponding to the spare image type when the modality information indicates that the image type is a spare image type for extension. Note that the order of execution of steps SP22 and SP23 may be reversed from the example in FIG. 18, or they may be performed simultaneously.

[0171] 12 is included in the parameter P, and if the modality information indicates that the image type is an infrared image (i.e., the value of vui_modality_type is 2), the decoding unit 52 further acquires the above-mentioned expression type information indicating the expression type of infrared radiation intensity from the bitstream BS. Also, if the expression type information indicates that the pixel value indicates a temperature value (i.e., the expression is a temperature value), the decoding unit 52 further acquires temperature information including the minimum and maximum temperatures of the temperature values ​​from the bitstream BS.

[0172] 14 is included in the parameter P, and if the flag information indicates that the pixel values ​​are represented using pseudo-color representation (i.e., if the value of vui_false_colour_representation_present_flag is 1), the decoding unit 52 further acquires from the bitstream BS the generation information for generating table information that associates each pixel value of the image Q with a color code of the pseudo-color representation. The decoding unit 52 also generates the table information based on the acquired generation information.

[0173] 19 and 20 are simplified diagrams showing examples of table information.

[0174] 19 and 20, the number of pairs of pixel values ​​and color codes is 6 (i.e., the value of vui_num_value_minus2 is 4), the image type is an infrared image, and the interpolation type may be linear or bilinear.

[0175] In the example shown in FIG. 19, the table information indicates the correspondence between hexadecimal color codes, RGB color codes, and gray shades.

[0176] 20, the table information shows the correspondence between hex color codes, shades of gray, and temperatures, with shades of gray being mapped to temperatures. For example, black represents a temperature of 300K, gray represents a temperature of 330K, and white represents a temperature of 350K.

[0177] Next, in step SP24, the switching unit 53 switches the task processing unit 54 based on the modality information input from the decoding unit 52. １ ~54 ｎ The switching unit 53 switches between a plurality of image types and a plurality of task processing units 54. １ ~54 ｎ The switching unit 53 holds table information (not shown) in which a correspondence relationship between the task and the task is preset. １ ~54 ｎ Among these, one task processing unit 54 corresponding to the image type indicated by the modality information is selected. １ ~54 ｎ Select .

[0178] Referring to FIG. 18, next, in step SP25, the one task processing unit 54 selected in step SP24 １ ~54 ｎ executes task processing based on image data D21 input from the decoding unit 52 via the switching unit 53.

[0179] According to the encoding device 1 and decoding device 2 of this embodiment, modality information indicating the image type of image Q and wavelength information indicating the wavelength band of the optical radiation of image Q can be included in the bit stream BS and transmitted from the encoding device 1 to the decoding device 2, thereby making it possible to improve the accuracy of task processing execution by the decoding device 2.

[0180] Furthermore, according to the encoding device 1 and the decoding device 2 of this embodiment, the modality information indicates that the image type is at least one of a visible light image, an infrared image, an ultraviolet image, undefined, and a spare image type for extension. Therefore, since the modality information includes modality information indicating that the image type is undefined or that the image type is a spare image type for extension, it is possible to deal with cases where the image characteristics are unknown in the encoding device 1 and the image type cannot be identified, or with future extensions of the image type, etc.

[0181] (Development of the Present Disclosure) The encoding device 1 and the decoding device 2 according to the above embodiment are devices that support VVC (H.266), a next-generation video coding standard. In VVC, it is being considered to extend VUI parameters to include image-related information such as modality information. However, existing video coding standards AVC (H.264) and HEVC (H.265) do not allow direct extension of VUI parameters. Therefore, if the encoding device 1 and the decoding device 2 are devices that support AVC or HEVC, they cannot properly process image-related information such as modality information extended to VUI parameters. Therefore, in this development, the syntax of the Sequence Parameter Set (SPS) is appropriately hierarchized using extension flag information, thereby enabling the encoding device 1 and the decoding device 2, which cannot directly extend VUI parameters, to properly process image-related information such as modality information. In addition, the information is not limited to SPS, but may be VUI, VPS, SPS, PPS, PH, SH, APS, SEI, tile header, system layer header, etc.

[0182] 1, the encoding device 1 according to this development includes a circuit 11 and a memory 12 connected to the circuit 11. The decoding device 2 includes a circuit 21 and a memory 22 connected to the circuit 21.

[0183] FIG. 24 is a simplified diagram illustrating a first example of the syntax of SPS in HEVC.

[0184] The NAL (Network Abstraction Layer) unit of the SPS includes first presence flag information (vui_parameters_present_flag), first video usability information (vui_parameters()), first extension flag information (sps_vui_extension_flag), second video usability information (sps_vui_extension()), second extension flag information (sps_extension_3bits), and first extension information (sps_extension_data_flag).

[0185] The first presence flag information indicates whether the first video usability information is described in the bitstream BS. If the first presence flag information is a true value (1), it indicates that the first video usability information is described in the bitstream BS, and if the first presence flag information is a false value (0), it indicates that the first video usability information is not described in the bitstream BS.

[0186] 25 is a simplified diagram illustrating an example of the syntax of first video usability information in HEVC or AVC. The first video usability information is VUI parameters including information on the aspect ratio of an image included in the bitstream BS (sar_width, sar_height), part of the color information of the image (color_primaries, transfer_characteristics, matrix_coeffs), etc.

[0187] 24, the first extension flag information indicates whether or not second video usability information that extends the video usability information is described in the bitstream BS. If the first extension flag information is a true value (1), it indicates that the second video usability information is described in the bitstream BS, and if the first extension flag information is a false value (0), it indicates that the second video usability information is not described in the bitstream BS.

[0188] The second video usability information includes related information (image related information) about images included in the bitstream BS, such as modality information, etc. The second video usability information will be described in detail later.

[0189] The second extension flag information indicates a reserved area reserved for future extension use. The second extension flag information indicates whether first extension information that extends other information of the NAL unit in which the second video usability information is described is described in the bitstream BS. If the second extension flag information is a true value (1), it indicates that the first extension information is described in the bitstream BS. If the second extension flag information is a false value (0) or if the second extension flag information is not present, it indicates that the first extension information is not described in the bitstream BS. If the first extension information is included at the end of the NAL unit that stores the first extension information, the first extension information has a bit string that continues up to the end bit of the NAL unit.

[0190] If the encoding device 1 supports HEVC or AVC but does not support the second video usability information, the encoding unit 33 encodes the first extension flag information, which is a false value (0), into the bitstream BS, and does not encode the second video usability information, the second extension flag information, and the first extension information into the bitstream BS.

[0191] Furthermore, when the decoding device 2 supports HEVC or AVC but does not support the second video usability information, if the first extension flag information is a true value (1), the decoding unit 52 ignores the second video usability information, the second extension flag information, and the first extension information in the decoding process. Ignoring in the decoding process includes not decoding or discarding the decoded result.

[0192] If the encoding device 1 supports the second video usability information but cannot interpret the first extension information, the encoding unit 33 encodes the first extension flag information that is a true value (1), the second extension flag information that is a false value (0), and the second video usability information into a bitstream BS, and does not encode the first extension information into the bitstream BS.

[0193] In addition, when the decoding device 2 supports the second video usability information but cannot interpret the first extension information, if the first extension flag information is a true value (1), the decoding unit 52 decodes the second video usability information and the second extension flag information from the bitstream BS, and if the second extension flag information is a true value (1), the decoding unit 52 ignores the first extension information in the decoding process.

[0194] 2 and 3, the setting unit 32 sets parameters P such as SPS in step SP12, and the encoding unit 33 encodes the parameters P into a bit stream BS in step SP14. Also, as shown in Figures 17 and 18, the decoding unit 52 decodes the parameters P from the bit stream BS in step SP23.

[0195] Here, the encoding unit 33 encodes the first presence flag information into a bitstream BS. If the first presence flag information is a true value (1), the encoding unit 33 encodes the first video usability information into the bitstream BS, and if the first presence flag information is a false value (0), the encoding unit 33 does not encode the first video usability information into the bitstream BS. Furthermore, the decoding unit 52 decodes the first presence flag information from the bitstream BS. If the first presence flag information is a true value (1), the decoding unit 52 decodes the first video usability information from the bitstream BS, and if the first presence flag information is a false value (0), the decoding unit 52 does not decode the first video usability information from the bitstream BS.

[0196] Furthermore, the encoding unit 33 encodes the first extension flag information into the bitstream BS. If the first extension flag information is a true value (1), the encoding unit 33 encodes the second video usability information into the bitstream BS, and if the first extension flag information is a false value (0), the encoding unit 33 does not encode the second video usability information into the bitstream BS. The decoding unit 52 decodes the first extension flag information from the bitstream BS. If the first extension flag information is a true value (1), the decoding unit 52 decodes the second video usability information from the bitstream BS, and if the first extension flag information is a false value (0), the decoding unit 52 does not decode the second video usability information from the bitstream BS.

[0197] Furthermore, the encoding unit 33 encodes the second extension flag information into the bit stream BS. If the second extension flag information is a true value (1), the encoding unit 33 encodes the first extension information into the bit stream BS, and if the second extension flag information is a false value (0), the encoding unit 33 does not encode the first extension information into the bit stream BS. The decoding unit 52 decodes the second extension flag information from the bit stream BS. If the second extension flag information is a true value (1), the decoding unit 52 decodes the first extension information from the bit stream BS, and if the second extension flag information is a false value (0), the decoding unit 52 does not decode the first extension information from the bit stream BS.

[0198] FIG. 26 is a simplified diagram illustrating a second example of the syntax of SPS in HEVC.

[0199] The encoding unit 33 encodes the second video usability information into the bitstream BS when the first extension flag information is a true value (1) and the first presence flag information is also a true value (1). The decoding unit 52 decodes the second video usability information from the bitstream BS when the first extension flag information is a true value (1) and the first presence flag information is also a true value (1). This makes it possible to prevent inconsistent processing, such as encoding and decoding only the second video usability information while not encoding and decoding the first video usability information.

[0200] 27 and 28 are simplified diagrams illustrating a first example of the syntax of SPS in AVC.

[0201] As shown in Figure 27, the first NAL unit of the SPS includes first presence flag information (vui_parameters_present_flag) and first video usability information (vui_parameters()). As shown in Figure 28, the second NAL unit of the SPS includes first extension flag information (additional_extension_flag), second video usability information (sps_vui_extension()), second extension flag information (additional_extension2_flag), and first extension information (additional_extension2_data_flag).

[0202] The encoding unit 33 encodes the first presence flag information and the first video usability information into a first NAL unit. The encoding unit 33 also encodes the first extension flag information, the second video usability information, the second extension flag information, and the first extension information into a second NAL unit. The decoding unit 52 decodes the first presence flag information and the first video usability information from the first NAL unit. The decoding unit 52 also decodes the first extension flag information, the second video usability information, the second extension flag information, and the first extension information from the second NAL unit.

[0203] FIG. 29 is a simplified diagram illustrating a second example of the syntax of SPS in AVC. The syntax of the first NAL unit is the same as that of FIG. 27. As shown in FIG. 29, the second NAL unit further includes second presence flag information (vui_parameters_present_flag) in addition to the first extension flag information, second video usability information, second extension flag information, and first extension information that are the same as those in FIG. 28. The second presence flag information, like the first presence flag information, indicates whether the first video usability information is described in the bitstream BS. If the second presence flag information is a true value (1), this indicates that the first video usability information is described in the bitstream BS, and if the second presence flag information is a false value (0), this indicates that the first video usability information is not described in the bitstream BS.

[0204] The encoding unit 33 encodes the second video usability information into the bitstream BS when the second extension flag information is a true value (1) and the first presence flag information is a true value (1). The decoding unit 52 decodes the second video usability information from the bitstream BS when the second extension flag information is a true value (1) and the first presence flag information is a true value (1). This makes it possible to prevent inconsistent processing, such as encoding and decoding only the second video usability information without encoding and decoding the first video usability information.

[0205] FIG. 30 is a simplified diagram illustrating a first example of the syntax of the second video usability information in HEVC or AVC.

[0206] The second video usability information includes image-related information, first size information (modality_type_extension_bits), extension modality information (reserved_modality_type_extension), second size information (vui_extension_bits), and second extension information (vui_reserved_extension).

[0207] The image-related information includes modality information (modality_type) and wavelength information (min_wavelength_mantissa, max_wavelength_mantissa, min_wavelength_exponent, max_wavelength_exponent), etc. However, the image-related information is not limited to modality information and wavelength information, etc., and may be any information related to an image included in the bitstream BS. For example, the image-related information may be phase information notified from the encoder to the decoder to avoid misalignment of the display position in an encoding technique (RPR) in which an image reduced on the encoder side is enlarged on the decoder side for display. The phase information may be set separately for each of the horizontal and vertical directions of the image.

[0208] The first size information indicates the size (number of bits in this example) of the extended modality information. The extended modality information extends the modality information. The second size information indicates the size (number of bits in this example) of the second extended information. The second extended information further extends the second video usability information.

[0209] The first size information having a value greater than 0 indicates that extended modality information is present in the second video usability information and indicates the number of bits of the extended modality information. The first size information having a value of 0 indicates that extended modality information is not present in the second video usability information. The value of the first size information ranges from 0 to, for example, 2048.

[0210] Similarly, second size information greater than 0 bits indicates the presence of second extension information in the second video usability information and the number of bits of the second extension information. Second size information of 0 indicates the absence of second extension information in the second video usability information. The value of the second size information ranges from 0 to, for example, 2048.

[0211] The encoding unit 33 encodes the related information relating to the image contained in the bitstream BS into second video usability information.

[0212] Furthermore, when setting the extended modality information, the encoding unit 33 encodes first size information indicating the size of the extended modality information into second video usability information and encodes the extended modality information into the second video usability information based on the first size information. The decoding unit 52 decodes the first size information from the second video usability information and decodes the extended modality information from the second video usability information based on the first size information.

[0213] On the other hand, if the extended modality information is not set or cannot be interpreted, the encoding unit 33 encodes the 0-bit first size information into the second video usability information and does not encode the extended modality information into the second video usability information. If the decoding unit 52 decodes the 0-bit first size information from the second video usability information, the decoding unit 52 does not decode the extended modality information from the second video usability information.

[0214] When the second extension information is set, the encoding unit 33 encodes second size information indicating the size of the second extension information into the second video usability information and encodes the second extension information into the second video usability information based on the second size information. The decoding unit 52 decodes the second size information from the second video usability information and decodes the second extension information from the second video usability information based on the second size information.

[0215] On the other hand, if the second extension information is not set or cannot be interpreted, the encoding unit 33 encodes 0-bit second size information into the second video usability information and does not encode the second extension information into the second video usability information. If the decoding unit 52 decodes 0-bit second size information from the second video usability information, the decoding unit 52 does not decode the second extension information from the second video usability information. This enables appropriate encoding and decoding when further information is added to the second video usability information through a future standard update by describing information about modality in the bitstream as extended modality information and describing other video usability information in the bitstream as second extension information. On the other hand, even if further information is added to the second video usability information through a future standard update, existing decoding devices that do not support the above future standard updates can consistently decode the bitstream by ignoring the extended modality information and the second extension information, even if they are described in the bitstream.

[0216] FIG. 31 is a simplified diagram illustrating a second example of the syntax of the second video usability information in HEVC or AVC.

[0217] The second video usability information includes third extension flag information (vui_extension_flag) instead of the second size information shown in Fig. 30. If the third extension flag information is a true value (1), it indicates that the second extension information is described in the bitstream BS, and if the third extension flag information is a false value (0) or if the third extension flag information does not exist, it indicates that the second extension information is not described in the bitstream BS.

[0218] FIG. 32 is a simplified diagram illustrating a third example of the syntax of the second video usability information in HEVC or AVC.

[0219] As in the second example shown in Figure 31, the second video usability information includes third extension flag information (vui_extension_flag). If the third extension flag information is a true value (1), it indicates that the second extension information is described in the bitstream BS, and if the third extension flag information is a false value (0) or if the third extension flag information is not present, it indicates that the second extension information is not described in the bitstream BS. If the second extension information is included at the end of the NAL unit that stores the second extension information, the second extension information has a bit string that continues up to the end bit of the NAL unit.

[0220] According to the encoding device 1 and the decoding device 2 of this development, the video usability information can be appropriately extended by the first extension flag information and the second video usability information. Also, the extension flag information can be appropriately extended by the second extension flag information and the first extension information.

[0221] The present disclosure is particularly useful when applied to an image processing system that includes an encoding device that encodes an image into a bitstream and transmits the bitstream, and a decoding device that decodes an image from the received bitstream.

Claims

1. A decoding device comprising a circuit and a memory connected to the circuit, wherein the circuit: obtains first presence flag information indicating whether first video user utility information is described in a bitstream from the bitstream; when the first presence flag information is a true value, obtains the first video user utility information from the bitstream; obtains first extension flag information indicating whether second video user utility information is described in the bitstream from the bitstream; when the first extension flag information is a true value, obtains the second video user utility information from the bitstream; obtains second extension flag information indicating whether first extension information for extending other information of a NAL (Network Abstraction Layer) unit in which the second video user utility information is described is described in the bitstream from the bitstream; and when the second extension flag information is a true value, obtains the first extension information from the bitstream.

2. The decoding device according to claim 1, wherein the circuit obtains the first presence flag information, the first video user utility information, the first extension flag information, the second video user utility information, the second extension flag information, and the first extension information from the same NAL unit.

3. The decoding device according to claim 2, wherein the circuit obtains the second video user utility information from the NAL unit when the first extension flag information is a true value and the first presence flag information is a true value.

4. The decoding device according to claim 1, wherein the circuit obtains the first presence flag information and the first video user utility information from a first NAL unit, and obtains the first extension flag information, the second video user utility information, the second extension flag information, and the first extension information from a second NAL unit.

5. The circuit obtains second presence flag information indicating whether the first video user facility information is described in the bit stream from the second NAL unit, and when the first extension flag information is a true value and the second presence flag information is a true value, obtains the second video user facility information from the second NAL unit. The decoding apparatus according to claim 4.

6. The circuit obtains image-related information related to an image included in the bit stream from the second video user facility information, and when second extension information for further expanding the second video user facility information is described in the second video user facility information, obtains size information indicating the size of the second extension information from the second video user facility information, and obtains the second extension information from the second video user facility information based on the size information. The decoding apparatus according to claim 1.

7. The circuit obtains image-related information related to an image included in the bit stream from the second video user facility information, obtains third extension flag information indicating whether second extension information for further expanding the second video user facility information is described in the second video user facility information from the second video user facility information, and when the third extension flag information is a true value, obtains the second extension information from the second video user facility information. The decoding apparatus according to claim 1.

8. The first extension information has a bit string continuous up to the end bit of the NAL unit storing the first extension information. The decoding apparatus according to claim 1.

9. The second extension information has a bit string continuous up to the end bit of the NAL unit storing the second extension information. The decoding apparatus according to claim 7.

10. The second video user facility information includes image-related information related to an image included in the bit stream, and in the decoding apparatus that cannot interpret the image-related information, when the first extension flag information is a true value, the circuit ignores the second video user facility information, the second extension flag information, and the first extension information. The decoding apparatus according to claim 1.

11. The second video user-ability information includes image-related information related to an image included in the bitstream. In the decoder that can interpret the image-related information but cannot interpret the first extension information, when the first extension flag information is a true value, the circuit obtains the second video user-ability information and the second extension flag information from the bitstream, and when the second extension flag information is a true value, ignores the first extension information. The decoder according to claim 1.

12. An encoding device comprising: a circuit; and a memory connected to the circuit, wherein the circuit encodes first presence flag information indicating whether first video user-ability information is described in a bitstream into the bitstream, encodes the first video user-ability information into the bitstream when the first presence flag information is a true value, encodes first extension flag information indicating whether second video user-ability information is described in the bitstream into the bitstream, encodes the second video user-ability information into the bitstream when the first extension flag information is a true value, encodes second extension flag information indicating whether first extension information for extending other information of a NAL (Network Abstraction Layer) unit in which the second video user-ability information is described is described in the bitstream into the bitstream, and encodes the first extension information into the bitstream when the second extension flag information is a true value.

13. The encoding device according to claim 12, wherein the circuit encodes the first presence flag information, the first video user-ability information, the first extension flag information, the second video user-ability information, the second extension flag information, and the first extension information into the same NAL unit.

14. The encoding device according to claim 13, wherein the circuit encodes the second video user-ability information into the NAL unit when the first extension flag information is a true value and the first presence flag information is a true value.

15. The circuit encodes the first presence flag information and the first video user capability information into a first NAL unit, and encodes the first extension flag information, the second video user capability information, the second extension flag information, and the first extension information into a second NAL unit. The encoding apparatus according to claim 12.

16. The circuit encodes second presence flag information indicating whether the first video user capability information is described in the bitstream into the second NAL unit, and encodes the second video user capability information into the second NAL unit when the first extension flag information is a true value and the second presence flag information is a true value. The encoding apparatus according to claim 15.

17. The circuit encodes image-related information related to an image included in the bitstream into the second video user capability information, and encodes size information indicating the size of the second extension information into the second video user capability information when the second extension information for further expanding the second video user capability information is described in the second video user capability information, and encodes the second extension information into the second video user capability information based on the size information. The encoding apparatus according to claim 12.

18. The circuit encodes image-related information related to an image included in the bitstream into the second video user capability information, encodes third extension flag information indicating whether the second extension information for further expanding the second video user capability information is described in the second video user capability information into the second video user capability information, and encodes the second extension information into the second video user capability information when the third extension flag information is a true value. The encoding apparatus according to claim 12.

19. The first extension information has a bit string continuous up to the end bit of the NAL unit storing the first extension information. The encoding apparatus according to claim 12.

20. The second extension information has a bit string continuous up to the end bit of the NAL unit storing the second extension information. The encoding apparatus according to claim 18.

21. The second video user ability information includes image-related information related to an image included in the bitstream. In the encoding device that cannot interpret the image-related information, the circuit encodes the first extended flag information, which is a false value, into the bitstream, and does not encode the second video user ability information, the second extended flag information, and the first extended information into the bitstream. The encoding device according to claim 12.

22. The second video user ability information includes image-related information related to an image included in the bitstream. In the encoding device that can interpret the image-related information but cannot interpret the first extended information, the circuit encodes the first extended flag information, which is a true value, the second extended flag information, which is a false value, and the second video user ability information into the bitstream, and does not encode the first extended information into the bitstream. The encoding device according to claim 12.

23. A decoding method, wherein a decoding device obtains first existence flag information indicating whether first video user ability information is described in a bitstream from the bitstream, obtains the first video user ability information from the bitstream when the first existence flag information is a true value, obtains first extended flag information indicating whether second video user ability information is described in the bitstream from the bitstream, obtains the second video user ability information from the bitstream when the first extended flag information is a true value, obtains second extended flag information indicating whether first extended information for extending other information of a NAL (Network Abstraction Layer) unit in which the second video user ability information is described is described in the bitstream from the bitstream, and obtains the first extended information from the bitstream when the second extended flag information is a true value.

24. The symbolization device encodes first existence flag information indicating whether first video user usability information is described in the bitstream into the bitstream. When the first existence flag information is true, the symbolization device encodes the first video user usability information into the bitstream. The symbolization device encodes first extension flag information indicating whether second video user usability information is described in the bitstream into the bitstream. When the first extension flag information is true, the symbolization device encodes the second video user usability information into the bitstream. The symbolization device encodes second extension flag information indicating whether first extension information for extending other information of a NAL (Network Abstraction Layer) unit in which the second video user usability information is described is described in the bitstream into the bitstream. When the second extension flag information is true, the symbolization device encodes the first extension information into the bitstream. Symbolization device.

Images