Image encoding device, image decoding device, image encoding method, and image decoding method

Abstract

An image decoding device is provided with: a circuit that, during operation, acquires a first image by decoding a first bit stream, acquires specification information for specifying a specific region in the first image, and acquires a second image including image data of the specific region, and that, when decoding a second bit stream, acquires a second image including image data of the specific region; and generating a third image on the basis of the first image, the designation information, and the second image.

Description

Technical Field

[0001] The present disclosure relates to an image encoding device, an image decoding device, an image encoding method, and an image decoding method. Background Art

[0002] Patent Document 1 discloses a video encoding and decoding method using an adaptive coupled pre-filter and post-filter.

[0003] Patent Document 2 discloses a method for encoding image data to be loaded into an artificial intelligence (AI) integrated circuit.

[0004] Prior Art Literature

[0005] Patent Literature

[0006] Patent Document 1: U.S. Patent No. 9883207

[0007] Patent Document 2: U.S. Patent No. 10452955 Summary of the invention

[0008] An object of the present disclosure is to balance the protection of personal privacy information and the execution of machine tasks or human vision using the privacy information on the image decoding device side during transmission of an image from an image encoding device to an image decoding device.

[0009] An image decoding device involved in one method of the present disclosure includes a circuit and a memory connected to the circuit. When the circuit is in operation, it obtains a first image by decoding a first bit stream, obtains designated information that specifies a specific area in the first image and a second image including image data of the specific area by decoding a second bit stream, and generates a third image based on the first image, the designated information and the second image. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a diagram schematically showing the configuration of an image processing system according to an embodiment of the present disclosure.

[0011] Figure 2 It is a flowchart showing the flow of processing performed by the image encoding device.

[0012] Figure 3 is a diagram showing an example of an input image.

[0013] Figure 4 FIG. 1 is a diagram showing a bounding box as an example of a specific area.

[0014] Figure 5A It is a diagram showing an example of designation information.

[0015] Figure 5B It is a diagram showing an example of designation information.

[0016] Figure 5C It is a diagram showing an example of designation information.

[0017] Figure 5D It is a diagram showing an example of designation information.

[0018] Figure 5E It is a diagram showing an example of designation information.

[0019] Fig. 5F It is a diagram showing an example of designation information.

[0020] Figure 6 This is a diagram showing an example of the first image.

[0021] Fig. 7A This is a diagram showing an example of the second image.

[0022] Figure 7B This is a diagram showing an example of the second image.

[0023] Figure 7C This is a diagram showing an example of the second image.

[0024] Fig.7D This is a diagram showing an example of the second image.

[0025] Fig. 7E This is a diagram showing an example of the second image.

[0026] Figure 7F This is a diagram showing an example of the second image.

[0027] Figure 7G This is a diagram showing an example of the second image.

[0028] Figure 7H This is a diagram showing an example of the second image.

[0029] Figure 8 This is a diagram showing a first example of the second image in which a plurality of bounding boxes partially overlap.

[0030] Fig.9A This is a diagram showing a second example of the second image in which a plurality of bounding boxes partially overlap.

[0031] Fig. 9B This is a diagram showing a second example of the second image in which a plurality of bounding boxes partially overlap.

[0032] Fig. 10AThis is a diagram showing a first example of the data structure of a bit stream.

[0033] Fig. 10B This is a diagram showing a first example of the data structure of a bit stream.

[0034] Fig.11 This is a flowchart showing the flow of processing performed by the image decoding device.

[0035] Fig.12 This is a diagram showing a simplified implementation example of an image encoding device.

[0036] Fig.13 This is a diagram showing a simplified implementation example of an image decoding device. DETAILED DESCRIPTION

[0037] (Knowledge that forms the basis of this disclosure)

[0038] Existing encoding methods aim to provide the best image under bit rate constraints in order to achieve human vision.

[0039] With the development of rich sensors and applications based on machine learning or neural networks, many intelligent platforms that process large amounts of data have been realized, including connected cars, video surveillance or smart cities, etc. Since a large amount of data is always generated, the existing methods of including humans in the pipeline are inefficient and impractical in terms of latency and scale.

[0040] Furthermore, in transmission and archiving systems, there is a need for more compact data representation and low-latency solutions, so VCM (Video Coding for Machines) was introduced.

[0041] In some cases, machines can communicate with each other and perform tasks without human intervention, or a specific stream after decompression needs additional processing by a human, such as in a surveillance camera where a human "supervisor" searches for specific people or scenes in the image.

[0042] In other cases, both humans and machines use corresponding bit streams. In the case of connected cars, humans can use features in image correction functions, and machines can use features in object detection and segmentation.

[0043] The general system structure includes a pair of an image encoding device and an image decoding device. The input of the system is a dynamic image, a static image or a feature quantity. Examples of machine tasks include object detection, object segmentation, object tracking, action recognition, posture estimation or any combination thereof. Human vision may be one of the use cases that can be used together with machine tasks.

[0044] Based on the prior art, when the captured image includes private information such as a person's face or a vehicle's license plate, the image including the private information is transmitted from the image encoding device to the image decoding device even if the information is not required for the machine task, so there is a possibility that the private information will be leaked. On the other hand, it is also possible to transmit an image obtained by performing a concealment process such as blurring the private information from the image encoding device to the image decoding device, but there is a problem that the private information cannot be used when the machine task or human vision requires the private information.

[0045] In order to solve this problem, the inventors have come to the understanding that the above-mentioned problem can be solved by transmitting a bit stream of an image including privacy information and a bit stream of an image not including privacy information separately from an image encoding device to an image decoding device, and synthesizing them in the image decoding device when privacy information is needed, thereby conceiving of the present disclosure.

[0046] Next, each aspect of the present disclosure will be described.

[0047] The image decoding device involved in the first embodiment of the present invention includes a circuit and a memory connected to the circuit. When the circuit is in operation, it obtains a first image by decoding a first bit stream, obtains designated information that specifies a specific area in the first image and a second image including image data of the specific area by decoding a second bit stream, and generates a third image based on the first image, the designated information and the second image.

[0048] According to the first embodiment, since the first bit stream does not include private information, personal private information can be protected. In addition, the second image including the private information is transmitted as the second bit stream, and the third image is generated based on the second image, thereby enabling machine tasks or human vision using the private information to be performed on the image decoding device side.

[0049] In the image decoding device according to the second aspect of the present disclosure, in the first aspect, the specific area in the first image may include an image in which concealment processing has been performed on a privacy area including private information.

[0050] According to the second aspect, it is possible to appropriately protect the private information included in the specific region in the first image through concealment processing.

[0051] In the image decoding device according to the third aspect of the present disclosure, in the second aspect, the second image and the third image may include an image in which the concealment process is not performed on the privacy area included in the specific area.

[0052] According to the third aspect, since the second image and the third image include an image in which the privacy area included in the specific area is not concealed, it is possible to appropriately perform machine tasks or human vision using the privacy information on the image decoding device side.

[0053] The image decoding device involved in the fourth aspect of the present disclosure is, in the first aspect, an image in which concealment processing is performed on a privacy area including privacy information in the specific area within the first image, and the second image includes an image in which concealment processing is not performed on the privacy area included in the specific area. In generating the third image, when using the image in which the privacy information is concealed, the first image is used to generate the third image, and when using the image in which the privacy information is not concealed, the first image and the second image are used to generate the third image.

[0054] According to the fourth aspect, the image decoding device can switch and execute a machine task or human vision using an image whose private information is concealed and a machine task or human vision using an image whose private information is not concealed.

[0055] In the image decoding device according to the fifth aspect of the present disclosure, in any one of the first to fourth aspects, the designation information may include information indicating the position and size of the specific area in the first image.

[0056] According to the fifth aspect, the position and size of the specific area in the first image can be appropriately determined by the designation information.

[0057] In the image decoding device according to the sixth aspect of the present disclosure, in the fifth aspect, the designation information may further include information indicating a correspondence relationship between the specific area in the first image and the second image.

[0058] According to the sixth aspect, the correspondence relationship between the specific area in the first image and the second image can be appropriately determined using the designation information.

[0059] In the image decoding device according to the seventh aspect of the present disclosure, in any one of the first to sixth aspects, the circuit may acquire the designation information by decoding a header region of the second bit stream.

[0060] According to the seventh aspect, since the designation information is stored in the header area of ​​the second bit stream, the designation information can be easily decoded from the second bit stream.

[0061] The image decoding device involved in the 8th mode of the present disclosure may be, in any one of the 1st to 7th modes, that the information representing the position and size of the specific area within the first image in the designated information is stored in the header area of ​​the first bit stream, and the circuit obtains the information by decoding the header area of ​​the first bit stream.

[0062] According to the eighth aspect, information originally included in the header area of ​​the first bit stream can be used as the designation information.

[0063] The image decoding device involved in the 9th mode of the present disclosure may be, in any one of the 1st to 8th modes, that the second image includes difference image data between the specific area in the first image and the specific area in the third image as the image data.

[0064] According to the ninth aspect, compared with the case where image data of a specific region that has not been subjected to concealment processing is transmitted, the amount of code transmitted from the image encoding device to the image decoding device can be reduced.

[0065] In the image decoding device according to the tenth aspect of the present disclosure, in any one of the first to eighth aspects, the second image may include image data of the specific area in the third image as the image data.

[0066] According to the tenth aspect, since the circuit only needs to replace the image data of the specific area in the first image with the image data of the second image, the processing load of the circuit can be reduced.

[0067] In the image decoding device involved in the 11th mode of the present disclosure, in any one of the 1st to 10th modes, the specific area may include a plurality of specific areas, and the picture of the second bit stream may include the image data of a specific area among the plurality of specific areas.

[0068] According to the eleventh aspect, since the picture in the second bit stream includes only image data of one specific region, it is easy to associate each specific region in the first image with each second image.

[0069] In the image decoding device according to the twelfth aspect of the present disclosure, in any one of the first to tenth aspects, the specific area may include a plurality of specific areas, and the picture of the second bit stream may include the image data of the plurality of specific areas.

[0070] According to the twelfth aspect, since the picture of the second bit stream includes image data of a plurality of specific regions, it is possible to reduce the number of pictures of the second bit stream to be transmitted from the image encoding device to the image decoding device.

[0071] In the image decoding device according to the 13th aspect of the present disclosure, in any one of the 1st to 12th aspects, the specific area may be included in one or more coding unit blocks constituting a picture of the second bit stream.

[0072] According to the thirteenth aspect, since the specific region is included in one or more coding unit blocks, the specific region can be appropriately decoded in the circuit.

[0073] In the image decoding device according to the fourteenth aspect of the present disclosure, in any one of the first to twelfth aspects, the specific area may be included in a sub-picture or a block of a picture constituting the second bit stream.

[0074] According to the fourteenth aspect, since the specific area is included in the sub-picture or the block, the specific area can be appropriately decoded in the circuit.

[0075] In the image decoding device according to the fifteenth aspect of the present disclosure, in any one of the first to fourteenth aspects, the specific area may include a plurality of specific areas, and the second image may include the image data corresponding to the plurality of specific areas.

[0076] According to the fifteenth aspect, even when a plurality of objects such as people are close to or far away from each other in a captured image, a plurality of specific areas corresponding to the plurality of objects can be included in one second image.

[0077] In the image decoding device according to the sixteenth aspect of the present disclosure, in any one of the first to fifteenth aspects, the size of the picture of the first bit stream may be the same as the size of the picture of the second bit stream.

[0078] According to the sixteenth aspect, since the size of the picture of the first bit stream is the same as the size of the picture of the second bit stream, it is easy to associate the specific area in the first image with the second image.

[0079] In the image decoding device according to the seventeenth aspect of the present disclosure, in any one of the first to sixteenth aspects, the picture of the second bit stream may be an intra picture that performs intra prediction.

[0080] According to the seventeenth aspect, since the picture of the second bit stream is an intra-frame picture, there is no need to consider the movement of the object with the passage of time, and there is no need to store the previous frame or perform difference calculation with the previous frame.

[0081] The image decoding device involved in the 18th mode of the present disclosure may be any one of the 1st to 16th modes, wherein the picture of the second bit stream is an inter-frame picture that performs inter-screen prediction using a reference picture obtained by decoding the second bit stream or the first bit stream.

[0082] According to the eighteenth aspect, since the picture of the second bit stream is an inter-picture, the amount of code can be reduced.

[0083] The image decoding device involved in the 19th mode of the present disclosure may be, in any one of the 1st to 18th modes, that the first bit stream is transmitted from the image encoding device to the image decoding device using a base layer of a multi-layer method, and the second bit stream is transmitted from the image encoding device to the image decoding device using an enhancement layer of a multi-layer method.

[0084] According to the nineteenth aspect, the first bit stream and the second bit stream can be easily transmitted using the base layer and the enhancement layer of the multi-layer method standardized by VVC or the like.

[0085] In the image decoding device according to the 20th aspect of the present disclosure, in any one of the first to 19th aspects, the first bit stream and the second bit stream may be transmitted from the image encoding device to the image decoding device using different transmission paths.

[0086] According to the twentieth aspect, since the first bit stream is transmitted using a transmission path such as a public network, and the second bit stream is transmitted using a transmission path such as a private network, it is possible to achieve both reduction in transmission cost and protection of privacy information.

[0087] The image encoding device involved in the 21st embodiment of the present invention comprises: a circuit; and a memory connected to the circuit, wherein the circuit, when in operation, specifies a specific area for an input image, generates specifying information for the specific area, generates a first image obtained by processing the specific area in the input image, generates a second image including image data corresponding to the specific area, generates a first bit stream by encoding the first image and outputs it, and generates a second bit stream by encoding the specifying information and the second image and outputs it.

[0088] According to the 21st embodiment, since the first bit stream does not include the private information, the private information of an individual can be protected. In addition, the second image including the private information is transmitted from the image encoding device to the image decoding device as the second bit stream, and the image decoding device generates the image including the private information based on the first image and the second image, thereby enabling the image decoding device to perform a machine task or human vision using the private information.

[0089] In the image coding device according to the 22nd aspect of the present disclosure, in the 21st aspect, the specific area in the first image may include an image in which concealment processing has been performed on a privacy area including private information.

[0090] According to the twenty-second aspect, the privacy information included in the specific area in the first image can be appropriately protected by the concealment process.

[0091] In the image encoding device according to the 23rd aspect of the present disclosure, in the 22nd aspect, the second image may include an image in which the concealment process is not performed on the privacy area included in the specific area.

[0092] According to the 23rd aspect, since the second image and the third image include an image in which the privacy area included in the specific area is not concealed, a machine task or human vision using the privacy information can be appropriately performed on the image decoding device side.

[0093] In the image encoding device according to the 24th aspect of the present disclosure, in any one of the 21st to 23rd aspects, the designation information may include information indicating the position and size of the specific area in the first image.

[0094] According to the twenty-fourth aspect, the position and size of the specific area in the first image can be appropriately determined using the designation information.

[0095] In the image encoding device according to the 25th aspect of the present disclosure, in the 24th aspect, the designation information may further include information indicating a correspondence relationship between the specific area in the first image and the second image.

[0096] According to the twenty-fifth aspect, the correspondence relationship between the specific area in the first image and the second image can be appropriately determined using the designation information.

[0097] In the image coding device according to the 26th aspect of the present disclosure, in any one of the 21st to 25th aspects, the circuit stores the designation information in a header area of ​​the second bit stream.

[0098] According to the 26th aspect, since the designation information is stored in the header area of ​​the second bit stream, the image decoding apparatus can easily decode the designation information from the second bit stream.

[0099] The image encoding device involved in the 27th mode of the present disclosure may be, in any one of the 21st to 26th modes, wherein the circuit stores the information indicating the position and size of the specific area in the first image among the designated information in the header area of ​​the first bit stream.

[0100] According to the 27th aspect, it is possible to use information originally included in the header area of ​​the first bit stream as the designation information.

[0101] The image encoding device involved in the 28th mode of the present disclosure may be, in any one of the 21st to 27th modes, that the second image includes difference image data between the specific area in the input image and the specific area in the first image as the image data.

[0102] According to the 28th aspect, compared with the case where image data of a specific area that has not been subjected to concealment processing is transmitted, the amount of code transmitted from the image encoding device to the image decoding device can be reduced.

[0103] In the image encoding device according to the 29th aspect of the present disclosure, in any one of the 21st to 27th aspects, the second image may include image data of the specific area in the input image as the image data.

[0104] According to the 29th aspect, since the image decoding device only needs to replace the image data of the specific area in the first image with the image data of the second image, it is possible to reduce the processing load of the image decoding device.

[0105] The image encoding device involved in the 30th mode of the present disclosure may be, in any one of the 21st to 29th modes, that the specific area includes multiple specific areas, and the picture of the second bit stream includes the image data of a specific area among the multiple specific areas.

[0106] According to the 30th aspect, since the picture of the second bit stream includes only image data of one specific area, it is easy to associate each specific area in the first image with each second image.

[0107] In the image encoding device involved in the 31st mode of the present disclosure, in any one of the 21st to 29th modes, the specific area may include a plurality of specific areas, and the picture of the second bit stream includes the image data of the plurality of specific areas.

[0108] According to the 31st aspect, since the picture of the second bit stream includes image data of a plurality of specific regions, it is possible to reduce the number of pictures of the second bit stream to be transmitted from the image encoding device to the image decoding device.

[0109] In the image encoding device according to the 32nd aspect of the present disclosure, in any one of the 21st to 31st aspects, the specific area may be included in one or more coding unit blocks constituting a picture of the second bit stream.

[0110] According to the thirty-second aspect, since the specific area is included in one or more coding unit blocks, the specific area can be appropriately decoded in the image decoding device.

[0111] In the image encoding device according to the 33rd aspect of the present disclosure, in any one of the 21st to 31st aspects, the specific area may be included in a sub-picture or a block of a picture constituting the second bit stream.

[0112] According to the thirty-third embodiment, since the specific area is included in the sub-picture or the block, the specific area can be appropriately decoded in the image decoding device.

[0113] In the image encoding device involved in the 34th mode of the present disclosure, in any one of the 21st to 33rd modes, the specific area may include a plurality of specific areas, and the second image may include the image data corresponding to the plurality of specific areas.

[0114] According to the thirty-fourth aspect, even when a plurality of objects such as people are close to or far away from each other in a captured image, a plurality of specific areas corresponding to the plurality of objects can be included in one second image.

[0115] In the image encoding device according to the 35th aspect of the present disclosure, in any one of the 21st to 34th aspects, the size of the picture of the first bit stream may be the same as the size of the picture of the second bit stream.

[0116] According to the thirty-fifth aspect, since the size of the picture of the first bit stream is the same as the size of the picture of the second bit stream, it is easy to associate the specific area in the first image with the second image.

[0117] In the image encoding device according to the 36th aspect of the present disclosure, in any one of the 21st to 35th aspects, the picture of the second bit stream may be an intra picture that performs intra-frame prediction.

[0118] According to the 36th method, since the picture of the second bit stream is an intra-frame picture, there is no need to consider the movement of the object with the passage of time, and there is no need to maintain the previous frame or perform difference calculation with the previous frame.

[0119] The image encoding device involved in the 37th mode of the present disclosure may be, in any of the 21st to 35th modes, that the picture of the second bit stream is an inter-frame picture that performs inter-screen prediction using a reference picture obtained by decoding the second bit stream or the first bit stream.

[0120] According to the 37th method, since the picture of the second bit stream is an inter-frame picture, the amount of code can be reduced.

[0121] The image encoding device involved in the 38th mode of the present disclosure may be, in any one of the 21st to 37th modes, that the circuit uses a base layer of a multi-layer method to transmit the first bit stream to the image decoding device, and uses an enhancement layer of a multi-layer method to transmit the second bit stream to the image decoding device.

[0122] According to the thirty-eighth embodiment, the first bit stream and the second bit stream can be easily transmitted using the base layer and the enhancement layer of the multi-layer method standardized by VVC or the like.

[0123] In the image encoding device according to the 39th aspect of the present disclosure, in any one of the 21st to 38th aspects, the circuit may transmit the first bit stream and the second bit stream to the image decoding device using different transmission paths.

[0124] According to the 39th method, since the first bit stream is transmitted using a transmission path such as a public network, and the second bit stream is transmitted using a transmission path such as a private network, it is possible to achieve both reduction in transmission cost and protection of privacy information.

[0125] In the image decoding method involved in the 40th mode of the present disclosure, a first image is obtained by decoding a first bit stream, and a second image including image data of the specific area and designated information for designating a specific area within the first image is obtained by decoding a second bit stream, and a third image is generated based on the first image, the designated information and the second image.

[0126] According to the 40th embodiment, since the first bit stream does not include the private information, the private information of an individual can be protected. In addition, the second image including the private information is transmitted as the second bit stream, and the third image is generated based on the second image, thereby enabling a machine task or human vision using the private information to be performed on the image decoding device side.

[0127] In the image encoding method involved in the 41st mode of the present disclosure, a specific area is specified for an input image, specifying information of the specific area is generated, a first image is generated by processing the specific area in the input image, a second image including image data corresponding to the specific area is generated, a first bit stream is generated and output by encoding the first image, and a second bit stream is generated and output by encoding the specifying information and the second image.

[0128] According to the 41st embodiment, since the first bit stream does not include the private information, the private information of an individual can be protected. In addition, by transmitting the second image including the private information as the second bit stream from the image encoding device to the image decoding device, the image decoding device generates the image including the private information based on the first image and the second image, thereby enabling the image decoding device to perform a machine task or human vision using the private information.

[0129] (Embodiments of the present disclosure)

[0130] Hereinafter, the embodiments of the present disclosure will be described in detail using the drawings. In addition, elements given the same reference numerals in different drawings represent the same or corresponding elements.

[0131] In addition, the embodiments described below all represent a specific example of the present disclosure. The numerical values, shapes, structural elements, steps, the order of steps, etc. shown in the following embodiments are an example and are not intended to limit the present disclosure. In addition, for the structural elements in the following embodiments that are not recorded in the independent technical solutions representing the highest concept, they are described as arbitrary structural elements. In addition, in all embodiments, each content can also be combined.

[0132] Figure 1 FIG. 2 is a diagram schematically showing a configuration of an image processing system according to an embodiment of the present disclosure. The image processing system includes an image encoding device 1 , transmission paths NW1 and NW2 , and an image decoding device 2 .

[0133] The image encoding device 1 includes a region specifying unit 11 , a first image generating unit 12 , a second image generating unit 13 , a first encoding unit 14 , and a second encoding unit 15 .

[0134] The area designation unit 11 designates a specific area for the input image D1 and generates designation information D2 for the specific area. The first image generation unit 12 generates a first image D3 obtained by processing the specific area in the input image D1. The second image generation unit 13 generates a second image D4 including image data corresponding to the specific area. The first encoding unit 14 generates a first bit stream D5 by encoding the first image D3 and outputs it. The first bit stream D5 is sent to the image decoding device 2 via the transmission path NW1. The second encoding unit 15 generates a second bit stream D6 by encoding the designation information D2 and the second image D4 and outputs it. The second bit stream D6 is sent to the image decoding device 2 via the transmission path NW2 different from the transmission path NW1.

[0135] The transmission paths NW1 and NW2 are the Internet, WAN (Wide Area Network), LAN (Local Area Network) or any combination thereof. In addition, the transmission paths NW1 and NW2 are not necessarily limited to a two-way communication network, but may also be a one-way communication network that sends broadcast waves such as terrestrial digital broadcasting or satellite broadcasting. In addition, the transmission paths NW1 and NW2 may also be recording media such as DVD (Digital Versatile Disc) or BD (Blue-Ray Disc) that record the first bit stream D5 or the second bit stream D6. The transmission path NW1 is a public network, etc., and the transmission path NW2 is a dedicated network that ensures secure communication through access restrictions, etc. In addition, in the case where different access restrictions can be set for the first bit stream D5 and the second bit stream D6, the transmission paths NW1 and NW2 may also be physically the same communication network or recording medium.

[0136] The image decoding device 2 includes a first decoding unit 21 , a second decoding unit 22 , and an image generating unit 23 .

[0137] The first decoding unit 21 receives the first bit stream D5 transmitted from the first encoding unit 14 via the transmission path NW1. The first decoding unit 21 decodes the received first bit stream D5 to obtain a first image D7 corresponding to the first image D3.

[0138] The second decoding unit 22 receives the second bit stream D6 transmitted from the second encoding unit 15 via the transmission path NW2. The second decoding unit 22 decodes the received second bit stream D6 to obtain the designation information D8 corresponding to the designation information D2 and the second image D9 corresponding to the second image D4.

[0139] The image generation unit 23 generates a third image D10 corresponding to the input image D1 based on the first image D7, the designation information D8, and the second image D9. Specifically, for example, in a use case using an image that does not include private information (that is, a use case using an image obtained by hiding the private information), the image generation unit 23 generates the third image D10 using the first image D7 as it is. The use case using an image that does not include private information is, for example, a case where it is used for viewing and listening by a user who does not have a special access authority, or a machine task that does not require detailed information such as a face or a vehicle license plate. In addition, for example, in a use case using an image that includes private information (that is, a use case using an image that does not hide the private information), the image generation unit 23 generates the third image D10 based on the designation information D8 using the first image D7 and the second image D9 for the image data within the specific area, and generates the third image D10 using the first image D7 for the image data outside the specific area. The use case of using an image including private information is, for example, for a machine task based on the viewing and listening of a user with special access rights or requiring detailed information such as a face or a vehicle license plate. In this way, by switching the generation of the third image D10 according to the use of the image, it is possible to switch and execute a machine task or human vision using an image obtained by hiding the private information and a machine task or human vision using an image without hiding the private information on the image decoding device 2 side. The machine task includes object detection, object segmentation, object tracking, action recognition, or posture estimation.

[0140] Fig.12 This is a diagram showing a simplified implementation example of the image coding device 1. The image coding device 1 is composed of a processor 101 and a memory 102 connected to the processor 101. However, the memory 102 may be included in the processor 101. The processor 101 is a circuit that performs information processing. The processor 101 includes a CPU or a GPU, etc. The memory 102 includes a semiconductor memory such as a ROM or a RAM, a magnetic disk or an optical disk, etc. The memory 102 stores information required for the processor 101 to perform coding processing, etc. For example, the memory 102 stores an input image D1, designated information D2, a first image D3, and a second image D4. In addition, a program is stored in the memory 102. The processor 101 executes the program read from the memory 102, so that the processor 101 functions as the area designation unit 11, the first image generation unit 12, the second image generation unit 13, the first coding unit 14, and the second coding unit 15.

[0141] Fig.132 is a diagram showing a simplified implementation example of the image decoding device 2. The image decoding device 2 is composed of a processor 201 and a memory 202 connected to the processor 201. However, the memory 202 may also be included in the processor 201. The processor 201 is a circuit that performs information processing. The processor 201 includes a CPU or a GPU, etc. The memory 202 includes a semiconductor memory such as a ROM or a RAM, a magnetic disk or an optical disk, etc. The memory 202 stores information required for the processor 201 to perform decoding processing, etc. For example, the memory 202 stores a first bit stream D5, a second bit stream D6, a first image D7, designated information D8, and a second image D9. In addition, a program is stored in the memory 202. The processor 201 executes the program read from the memory 202, so that the processor 201 functions as a first decoding unit 21, a second decoding unit 22, and an image generation unit 23.

[0142] Figure 2 : is a flowchart showing the flow of processing performed by the image encoding device 1.

[0143] First, in step SP11 , the region specifying unit 11 specifies a specific region in the input image D1 , and generates specifying information D2 of the specific region.

[0144] Next, in step SP12 , the first image generating unit 12 generates a first image D3 obtained by processing a specific region in the input image D1 .

[0145] Next, in step SP13 , the second image generating unit 13 generates a second image D4 including image data corresponding to the specific region.

[0146] Next, in step SP14 , the first encoder 14 generates and outputs a first bit stream D5 by encoding the first image D3 .

[0147] Next, in step SP15 , the second encoding unit 15 generates and outputs a second bit stream D6 by encoding the designation information D2 and the second image D4 .

[0148] Figure 3 is a diagram showing an example of an input image D1. The input image D1 includes an image 31 of a person's face, an image 32 of the whole body including the person's face, and an image 33 of a background. The face of a person is an example of personal private information. In addition to the face of a person, private information also includes a vehicle number plate or a house number, etc.

[0149] Figure 44 is a diagram showing bounding boxes 41 and 42 as examples of specific areas. The area designation unit 11 sets the bounding box 41, which is a rectangle surrounding the privacy area including the image 31, as one of the specific areas. In addition, the area designation unit 11 sets the bounding box 42, which is a rectangle surrounding the privacy area including the image 32, as one of the specific areas. The image 33 does not include privacy information, so no bounding box is set.

[0150] The designation information D2 includes information indicating the positions and sizes of the bounding boxes 41 and 42 in the first image D3 .

[0151] Figure 5A to Figure 5F FIG. 4 is a diagram showing an example of the designated information D2. Figure 5A As shown, the designation information D2 includes the position coordinates (X1, Y1) of a specific vertex of the bounding box, the vertical size H of the bounding box, and the horizontal size W of the bounding box. Figure 5B As shown, the designated information D2 includes the position coordinates (X1, Y1) of a specific vertex of the bounding box and the position coordinates (X4, Y4) of the vertex located at the diagonal angle of the vertex. Figure 5C As shown, the designation information D2 includes the position coordinates (X0, Y0) of the center of the bounding box, the vertical size H of the bounding box, and the horizontal size W of the bounding box. Figure 5D As shown, the specified information D2 includes the position coordinates of the four vertices of the bounding box (X1, Y1) (X2, Y2) (X3, Y3) (X4, Y4). In this case, Figure 5E As shown in , for the tilted bounding box, its position and size can also be specified. Fig. 5F As shown, the designation information D2 includes mask information that indicates pixel positions within the specific area with "1" and indicates pixel positions outside the specific area with "0".

[0152] In addition, the designation information D2 includes information indicating the correspondence between the bounding boxes 41, 42 in the first image D3 and the second image D4. In addition, when the second image D4 includes a plurality of bounding boxes 41, 42, the designation information D2 includes information indicating the correspondence between the bounding boxes 41, 42 in the first image D3 and the bounding boxes 41, 42 in the second image D4.

[0153] Figure 6 4 is a diagram showing an example of the first image D3. The first image D3 includes an image in which a privacy area including privacy information is concealed. The first image generation unit 12 conceals the privacy area in the input image D1 based on the designation information D2 input from the area designation unit 11. The concealment process includes blurring, mosaic processing, or silhouette processing. Figure 6An example of silhouette processing is shown in FIG.

[0154] Figure 7A to Figure 7H It is a diagram showing an example of the second image D4.

[0155] like Fig. 7A As shown, the second image D4A has the same picture size as the input image D1. The rectangular area divided by the dotted line in the second image D4A is equivalent to the coding unit block. The coding unit block is a CU (cording unit) or a CTU (cording tree unit), for example, and has a size of 8×8 pixels. The image data of the area outside the bounding boxes 41 and 42 in the second image D4A can be the same as the input image D1, or can be filled with meaningless data (such as "0"). The position and size of the bounding boxes 41 and 42 in the second image D4A are the same as the position and size of the bounding boxes 41 and 42 in the input image D1. The size of the picture of the input image D1 is the same as the size of the picture of the second image D4A, so the first encoding unit 14 can use the base layer of the multi-layer method standardized by H.266 / VVC (Versatile Video Coding) and the like to transmit the first bit stream D5 to the image decoding device 2. In addition, the second encoding unit 15 can transmit the second bit stream D6 to the image decoding device 2 using the enhancement layer of the multi-layer method. In addition, when the multi-layer method is used, the enhancement layer can be encoded as difference information with the base layer, so it is also possible to encode the difference information of the first image D3 in which the private information is concealed and the input image D1 in which the private information is not concealed as the second bit stream D6 without setting a boundary box. In the image decoding device 2, by allowing any user to access the base layer, it is possible to perform machine tasks or human vision using images in which the private information is concealed. In addition, by allowing only users with special permissions to access the enhancement layer, it is possible to perform machine tasks or human vision using images in which the private information is not concealed.

[0156] The second image D4 includes an image in which concealment processing is not performed on the privacy area included in the boundary frames 41 and 42. The second image includes image data corresponding to the specific area as an image in which concealment processing is not performed. The image data corresponding to the specific area may be difference image data between the specific area in the input image D1 and the specific area in the first image D3, or may be the image data itself of the specific area in the input image D1. In the case where the specific area of ​​the second image D4 is difference image data, the first image D3 and the second image D4 are added for the specific area, thereby obtaining an image before concealment processing is performed on the privacy area. In the case where the specific area of ​​the second image D4 is the image data of the input image D1, the value obtained by multiplying the first image D3 by the weight value α and the value obtained by multiplying the second image D4 by the weight value β are added for the specific area, thereby obtaining an image before concealment processing is performed on the privacy area. The weight values ​​α and β are both greater than 0 and less than 1, and have the relationship of α+β=1. In addition, when α=0 and β=1, the specific area of ​​the second image D4 becomes the image data of the input image D1 itself. In this case, the image data of the specific area of ​​the first image D3 is not used at all, and the image before the privacy area is concealed is obtained. In addition, β can be set to 1-α and only one weight value α can be used for regulation.

[0157] In addition, the second image D4 may also be an intra-frame picture encoded by intra-frame prediction. That is, the second encoder 15 may also encode the second image D4 using intra-frame prediction. The picture of the second bit stream D6 is an intra-frame picture, so there is no need to consider the movement of the object with the passage of time, and thus there is no need to maintain the previous frame, perform a difference calculation with the previous frame, etc.

[0158] On the other hand, the second image D4 may be an inter-frame picture encoded by inter-frame prediction. That is, the second encoder 15 may encode the second image D4 by using inter-frame prediction. As a reference picture used for inter-frame prediction, a picture of the second image D4 obtained by decoding the second bit stream D6 (local decoding) or a picture of the first image D3 obtained by decoding the first bit stream D5 (local decoding) may be used. The picture of the second bit stream D6 is an inter-frame picture, so the amount of code can be reduced.

[0159] exist Fig. 7A The second image D4A shown includes a plurality of bounding boxes 41 and 42 , but the bounding box 41 and the bounding box 42 may be included in different second images D4B and D4C.

[0160] like Figure 7BAs shown, the second image D4B only includes the bounding box 41. The second image D4B has the same picture size as the input image D1. The image data of the area outside the bounding box 41 in the second image D4B can be the same as the input image D1, or can be filled with meaningless data. The position and size of the bounding box 41 in the second image D4B are the same as the position and size of the bounding box 41 in the input image D1.

[0161] like Figure 7C As shown, the second image D4C only includes the bounding box 42. The second image D4C has the same picture size as the input image D1. The image data of the area outside the bounding box 42 in the second image D4C can be the same as the input image D1, or can be filled with meaningless data. The position and size of the bounding box 42 in the second image D4C are the same as the position and size of the bounding box 42 in the input image D1.

[0162] like Fig.7D As shown, the second image D4D includes only the bounding box 41. Considering that the size of the bounding box 41 changes for each frame, the second image D4D is composed of the minimum number of coding unit blocks (4 in this example) including the bounding box 41, with the bounding box 41 placed close to the upper left corner. The image data of the area outside the bounding box 41 in the second image D4D is filled with meaningless data.

[0163] like Fig. 7E As shown, the second image D4E includes only the bounding box 42. Considering that the size of the bounding box 42 changes for each frame, the second image D4E is composed of the minimum number of coding unit blocks (15 in this example) including the bounding box 42, with the bounding box 42 placed close to the upper left corner. The image data of the area outside the bounding box 42 in the second image D4E is filled with meaningless data.

[0164] like Figure 7F As shown, the second image D4F includes the bounding boxes 41 and 42. The 15 coding unit blocks including the bounding box 42 are arranged to the right of the 4 coding unit blocks including the bounding box 41. In order to define the second image D4F as a quadrilateral region, the 6 coding unit blocks are arranged below the 4 coding unit blocks including the bounding box 41. The image data of the region outside the bounding boxes 41 and 42 in the second image D4F is filled with meaningless data.

[0165] like Figure 7GAs shown, the second image D4G includes bounding boxes 41 and 42. The picture of the second image D4G is divided into a plurality of sub-pictures standardized by VVC or the like. Bounding box 41 is included in the sub-picture at the upper left corner, and bounding box 42 is included in the sub-picture adjacent to the right thereof. The image data of the area outside the bounding boxes 41 and 42 in the second image D4G is filled with meaningless data.

[0166] like Figure 7H As shown, the second image D4H includes bounding boxes 41 and 42. The picture of the second image D4H is divided into a plurality of blocks standardized by VVC or the like. The bounding box 41 is included in the block at the upper left corner, and the bounding box 42 is included in the blocks adjacent to the right and lower right thereof. The image data of the area outside the bounding boxes 41 and 42 in the second image D4H is filled with meaningless data.

[0167] Figure 8 4 is a diagram showing a first example of a second image D4I in which a portion of a plurality of boundary boxes 41 and 42 overlap. Figure 8 As shown, the second image D4I includes bounding boxes 41 and 42. Due to the two people getting closer, the bounding boxes 41 and 42 partially overlap. The second image D4I is composed of a minimum number of coding unit blocks (24 in this example) including the bounding boxes 41 and 42, with the bounding box 41 close to the upper left corner. The image data of the area outside the bounding boxes 41 and 42 in the second image D4I is filled with meaningless data. When the bounding boxes 41 and 42 no longer overlap due to the two people moving away, the second image D4I is divided into Fig.7D The second image D4D and Fig. 7E Even when a portion of the plurality of bounding boxes 41 and 42 overlap, the plurality of bounding boxes 41 and 42 can be divided into two images including the overlapping portion according to the designation of each bounding box 41 and 42. Fig.7D The second image D4D and Fig. 7E The second image D4E is shown. In addition, the second image D4I may be defined by a sub-picture or a block instead of a plurality of coding unit blocks.

[0168] Fig.9A , 9B 4 is a diagram showing a second example of the second images D4J and D4K in which a portion of the plurality of boundary boxes 41 and 42 overlap. Fig.9AAs shown, the second image D4J includes the bounding boxes 41 and 42. Due to the proximity of the two characters, the bounding boxes 41 and 42 partially overlap. The second image D4I is constructed by a sub-picture including the bounding boxes 41 and 42 in a state where the bounding box 41 is close to the upper left corner. The number of coding unit blocks included in the sub-picture is set to a number (35 in this example) greater than the minimum number of coding unit blocks including the bounding boxes 41 and 42 (24 in this example). The image data of the area outside the bounding boxes 41 and 42 in the second image D4J is filled with meaningless data.

[0169] When the bounding boxes 41 and 42 become non-overlapping due to the two characters moving away from each other, as shown in FIG. Fig. 9B As shown, in the second image D4K, the overlap of the bounding boxes 41 and 42 is eliminated. The second image D4K includes the bounding boxes 41 and 42 that do not overlap each other. When the bounding box 42 exits the range of the second image D4K due to the further separation of the two characters, the second image D4K is as shown in FIG. Figure 7G The second images D4J and D4K may be divided into different sub-pictures as shown in the second image D4G. In addition, the second images D4J and D4K may be defined by a block or a plurality of coding unit blocks instead of a sub-picture.

[0170] Fig. 10A 1 is a diagram showing a first example of the data structure of the bit stream D6. The bit stream D6 has a header area H for storing management information and the like and a payload area P for storing image data. The second encoder 15 stores the encoded data of the second image D4 in the payload area P. In addition, the second encoder 15 stores the encoded data 70 of the designation information D2 in a predetermined position of the payload area P.

[0171] Fig. 10B 2 is a diagram showing a second example of the data structure of the bit stream D6. Fig. 10A Similarly, the bitstream D6 has a header area H and a payload area P. The second encoding unit 15 stores the encoded data of the second image D4 in the payload area P. In addition, the second encoding unit 15 stores the encoded data 70 of the designated information D2 in a given part of the header area H. The given part is, for example, a SEI (Supplemental Enhancement Information) area for storing additional information. The given part may also be a VPS, SPS, PPS, PH, SH, APS or a block header, etc. By storing the designated information D2 in the header area H of the bitstream D6, the second decoding unit 22 can easily obtain the designated information D8 through decoding. In addition, by storing the designated information D2 in the SEI area, the designated information D2 can be easily processed as additional information.

[0172] In addition, the first encoder 14 may store the information indicating the position and size of the bounding boxes 41 and 42 in the first image D3 in a given location in the header region H of the first bitstream D5, among the designated information D2. The given location may be, for example, an ARSEI (annotated region SEI: annotated region supplemental enhancement information) region for storing the bounding box information. Thus, the bounding box information originally included in the header region H of the first bitstream D5 can be used as part of the designated information D2.

[0173] Fig.11 It is a flowchart showing the flow of processing performed by the image decoding device 2.

[0174] First, in step SP21 , the first decoding unit 21 decodes the first bit stream D5 received from the image encoding device 1 , thereby acquiring the first image D7 corresponding to the first image D3 .

[0175] Next, in step SP22 , the second decoding unit 22 decodes the second bit stream D6 received from the image encoding device 1 , thereby acquiring the designation information D8 corresponding to the designation information D2 and the second image D9 corresponding to the second image D4 .

[0176] Next, in step SP23 , the image generating unit 23 generates a third image D10 corresponding to the input image D1 based on the first image D7 , the designation information D8 , and the second image D9 .

[0177] Similar to the first image D3 in the image encoding device 1, the areas within the bounding boxes 41 and 42 included in the first image D7 in the image decoding device 2 include an image obtained by performing concealment processing on the privacy area including the privacy information. The concealment processing includes blurring, mosaic processing, or silhouette processing.

[0178] Similar to the second image D4 and the input image D1 in the image encoding device 1, the second image D9 and the third image D10 in the image decoding device 2 include images in which the privacy area included in the bounding boxes 41 and 42 is not concealed. The second image D9 may include the difference image data between the bounding boxes 41 and 42 in the first image D7 and the bounding boxes 41 and 42 in the third image D10, or may include the image data of the bounding boxes 41 and 42 in the third image D10.

[0179] The second decoding unit 22 obtains the designated information D8 by decoding the payload area P or the header area H of the second bit stream D6. In addition, the information indicating the position and size of the bounding boxes 41 and 42 in the first image D7 in the designated information D8 may be stored in the header area H of the first bit stream D5. In this case, the first decoding unit 21 obtains the information by decoding the header area H of the first bit stream D5.

[0180] Similar to the designation information D2 in the image encoding device 1, the designation information D8 in the image decoding device 2 includes information indicating the position and size of the bounding boxes 41 and 42 in the first image D7. The designation information D8 may further include information indicating the correspondence between the bounding boxes 41 and 42 in the first image D7 and the second image D9.

[0181] Similar to the second image D4 in the image encoding device 1, the second image D9 in the image decoding device 2 may include image data of one of the plurality of bounding boxes 41 and 42, or may include image data of a plurality of bounding boxes 41 and 42. In addition, in the second image D9, the bounding boxes 41 and 42 may be included in one or more coding unit blocks of the picture constituting the second bit stream D6, or may be included in a sub-picture or block of the picture constituting the second bit stream D6. Furthermore, the second image D9 may include image data corresponding to a plurality of bounding boxes 41 and 42.

[0182] In addition, the size of the picture of the first bit stream D5 may be the same as the size of the picture of the second bit stream D6. Alternatively, the first bit stream D5 may be transmitted from the image encoding device 1 to the image decoding device 2 using a base layer of a multi-layer method, and the second bit stream D6 may be transmitted from the image encoding device 1 to the image decoding device 2 using an enhancement layer of a multi-layer method. The first bit stream D5 and the second bit stream D6 may be transmitted from the image encoding device 1 to the image decoding device 2 using different transmission paths NW1 and NW2. The picture of the second bit stream D6 may be either an intra-frame picture for intra-screen prediction or an inter-frame picture for inter-screen prediction. As a reference picture used for inter-screen prediction, either a picture of the second image D9 obtained by decoding the second bit stream D6 or a picture of the first image D7 obtained by decoding the first bit stream D5 may be used.

[0183] (Effects, etc.)

[0184] According to the image encoding device 1 and the image decoding device 2 of the present embodiment, since the first bit stream D5 does not include private information, personal private information can be protected. In addition, the second image D4 including private information is transmitted as the second bit stream D6, and the image decoding device 2 generates the third image D10 based on the second image D9, thereby enabling the image decoding device 2 to perform machine tasks or human vision using private information.

[0185] Industrial Applicability

[0186] The present disclosure is particularly useful when applied to an image processing system including an image encoding device that transmits an image and an image decoding device that receives an image.

Claims

1. An image decoding device, comprising: Circuits; and a memory connected to the circuit, The circuit performs the following processing in action: By decoding the first bit stream, a first image is obtained. By decoding the second bit stream, designation information designating a specific area in the first image and a second image including image data of the specific area are acquired; A third image is generated based on the first image, the designation information, and the second image.

2. The image decoding device according to claim 1, wherein: The specific area in the first image includes an image in which a privacy area including private information is concealed.

3. The image decoding device according to claim 2, wherein: The second image and the third image include an image in which the concealment process is not performed on the privacy area included in the specific area.

4. The image decoding device according to claim 1, wherein: The specific area in the first image includes an image in which a privacy area including private information is concealed. The second image includes an image in which the concealment process is not performed on the privacy area included in the specific area. In generating the third image, In the use of an image in which private information is concealed, the third image is generated using the first image, In the application of using an image whose private information is not concealed, the third image is generated using the first image and the second image.

5. The image decoding device according to claim 1, wherein: The designation information includes information indicating the position and size of the specific area in the first image.

6. The image decoding device according to claim 5, wherein: The designation information further includes information indicating a correspondence relationship between the specific area in the first image and the second image.

7. The image decoding device according to claim 1, wherein: The circuit acquires the specified information by decoding a header area of ​​the second bit stream.

8. The image decoding device according to claim 1, wherein: The information indicating the position and size of the specific area in the first picture among the designation information is stored in the header area of ​​the first bit stream. The circuit obtains the information by decoding the header area of ​​the first bit stream.

9. The image decoding device according to claim 1, wherein: The second image includes, as the image data, difference image data between the specific area in the first image and the specific area in the third image.

10. The image decoding device according to claim 1, wherein: The second image includes image data of the specific area in the third image as the image data.

11. The image decoding device according to claim 1, wherein: The specific area includes a plurality of specific areas. The picture of the second bitstream includes the image data of a specific area among the multiple specific areas.

12. The image decoding device according to claim 1, wherein: The specific area includes a plurality of specific areas. The picture of the second bit stream includes the image data of the multiple specific areas.

13. The image decoding device according to claim 1, wherein: The specific area is included in one or more coding unit blocks constituting a picture of the second bit stream.

14. The image decoding device according to claim 1, wherein: The specific area is included in a sub-picture or a block of a picture constituting the second bit stream.

15. The image decoding apparatus according to claim 1, wherein: The specific area includes a plurality of specific areas. The second image includes the image data corresponding to the plurality of specific areas.

16. The image decoding device according to claim 1, wherein: The size of the picture of the first bit stream is the same as the size of the picture of the second bit stream.

17. The image decoding apparatus according to claim 1, wherein: The picture of the second bit stream is an intra picture that performs intra-frame prediction.

18. The image decoding apparatus according to claim 1, wherein: The picture of the second bit stream is an inter-frame picture that performs inter-screen prediction using a reference picture obtained by decoding the second bit stream or the first bit stream.

19. The image decoding apparatus according to claim 1, wherein: The first bit stream is transmitted from the image encoding device to the image decoding device using a base layer of a multi-layer method, The second bit stream is transmitted from the image encoding device to the image decoding device using an enhancement layer of a multi-layer method.

20. The image decoding apparatus according to claim 1, wherein: The first bit stream and the second bit stream are transmitted from the image encoding device to the image decoding device using different transmission paths.

21. An image encoding device comprising: Circuits; and a memory connected to the circuit, The circuit performs the following processing in action: Specify a specific area of ​​the input image and generate specified information of the specific area. generating a first image obtained by processing the specific area in the input image, generating a second image including image data corresponding to the specific area, By encoding the first image, a first bit stream is generated and outputted, The designation information and the second image are encoded to generate a second bit stream and output it.

22. The image encoding device according to claim 21, wherein: The specific area in the first image includes an image in which a privacy area including private information is concealed.

23. The image encoding device according to claim 22, wherein: The second image includes an image in which the concealment process is not performed on the privacy area included in the specific area.

24. The image encoding device according to claim 21, wherein: The designation information includes information indicating the position and size of the specific area in the first image.

25. The image encoding device according to claim 24, wherein: The designation information further includes information indicating a correspondence relationship between the specific area in the first image and the second image.

26. The image encoding device according to claim 21, wherein: The circuit stores the specified information in a header area of ​​the second bit stream.

27. The image encoding device according to claim 21, wherein: The circuit stores information indicating the position and size of the specific area in the first image among the designation information in a header area of ​​the first bit stream.

28. The image encoding device according to claim 21, wherein: The second image includes, as the image data, differential image data between the specific area in the input image and the specific area in the first image.

29. The image encoding device according to claim 21, wherein: The second image includes, as the image data, image data of the specific area in the input image.

30. The image encoding device according to claim 21, wherein: The specific area includes a plurality of specific areas. The picture of the second bitstream includes the image data of a specific area among the multiple specific areas.

31. The image encoding device according to claim 21, wherein: The specific area includes a plurality of specific areas. The picture of the second bit stream includes the image data of the multiple specific areas.

32. The image encoding device according to claim 21, wherein: The specific area is included in one or more coding unit blocks constituting a picture of the second bit stream.

33. The image encoding device according to claim 21, wherein: The specific area is included in a sub-picture or a block of a picture constituting the second bit stream.

34. The image encoding device according to claim 21, wherein: The specific area includes a plurality of specific areas. The second image includes the image data corresponding to the plurality of specific areas.

35. The image encoding device according to claim 21, wherein: The size of the picture of the first bit stream is the same as the size of the picture of the second bit stream.

36. The image encoding device according to claim 21, wherein: The picture of the second bit stream is an intra picture that performs intra-frame prediction.

37. The image encoding device according to claim 21, wherein: The picture of the second bit stream is an inter-frame picture that performs inter-screen prediction using a reference picture obtained by decoding the second bit stream or the first bit stream.

38. The image encoding device according to claim 21, wherein: The circuit transmits the first bit stream to the image decoding device using a base layer of a multi-layer method, and transmits the second bit stream to the image decoding device using an enhancement layer of a multi-layer method.

39. The image encoding device according to claim 21, wherein: The circuit transmits the first bit stream and the second bit stream to the image decoding device using different transmission paths.

40. An image decoding method, By decoding the first bit stream, a first image is obtained. By decoding the second bit stream, designation information designating a specific area in the first image and a second image including image data of the specific area are acquired; A third image is generated based on the first image, the designation information, and the second image.

41. A method for encoding an image, Specify a specific area of ​​the input image and generate specified information of the specific area. generating a first image obtained by processing the specific area in the input image, generating a second image including image data corresponding to the specific area, By encoding the first image, a first bit stream is generated and outputted, The designation information and the second image are encoded to generate a second bit stream and output it.

Images