File processing device and file processing method

Abstract

The present technology relates to a file processing device and a file processing method for making it possible to limit the viewing of an image. A file control unit generates a file in which an encrypted image obtained by encrypting an image by means of a first encryption key and an encrypted encryption key obtained by encrypting the first encryption key by means of a second encryption key are associated with each other and stored. Further, a file control unit decrypts the encrypted encryption key in the file into the first encryption key, and decrypts the encrypted image into an image by means of the first encryption key obtained by the decrypting. The present technology is applicable, for example, to digital cameras and the like for capturing an image.

Description

[Technical field]

[0001] The present technology relates to a file processing device and a file processing method, and more particularly to a file processing device and a file processing method that enable, for example, restriction of viewing of images. [Background technology]

[0002] High Efficiency Image File Format (HEIF) is a file format for efficiently storing images (see Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] ISO / IEC 23008-12:2017, Information technology -- High efficiency coding and media delivery in heterogeneous environments -- Part 12: Image File Format Summary of the Invention [Problem to be solved by the invention]

[0004] For HEIF files and other files that store images, it would be useful to be able to restrict viewing of images stored in the files from the standpoint of copyright protection, portrait rights protection, and the like.

[0005] The present technology has been made in view of such circumstances, and makes it possible to restrict viewing of images stored in a file. [Means for solving the problem]

[0006] A first file processing device of the present technology is a file processing device that includes a file control unit that generates a file that stores an encrypted image obtained by encrypting an image with a first encryption key and an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key in association with each other.

[0007] A first file processing method of the present technology is a file processing method that includes generating a file that stores an encrypted image obtained by encrypting an image with a first encryption key and an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key in association with each other.

[0008] In the first file processing device and file processing method of the present technology, a file is generated that stores an encrypted image, obtained by encrypting an image with a first encryption key, and an encrypted encryption key, obtained by encrypting the first encryption key with a second encryption key, in association with each other.

[0009] A second file processing device of the present technology is a file processing device that has a file control unit that decrypts an encrypted image obtained by encrypting an image with a first encryption key and an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key, of a file stored in association with each other, into the first encryption key, and decrypts the encrypted image into the image using the first encryption key obtained by the decryption.

[0010] A second file processing method of the present technology is a file processing method that includes decrypting an encrypted image obtained by encrypting an image with a first encryption key and an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key, of a file stored in association with each other, into the first encryption key, and decrypting the encrypted image into the image using the first encryption key obtained by the decryption.

[0011] In a second file processing device and file processing method of the present technology, an encrypted image obtained by encrypting an image with a first encryption key and an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key are stored in association with each other in a file, and the encrypted encryption key is decrypted into the first encryption key, and the encrypted image is decrypted into the image using the first encryption key obtained by the decryption.

[0012] The first and second file processing devices may be independent devices, or may be internal blocks constituting a single device.

[0013] The first and second file processing devices can be realized by causing a computer to execute a program. The program that causes a computer to function as the first and second file processing devices can be provided by being recorded on a recording medium or transmitted via a transmission medium. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram showing an example of the configuration of an embodiment of a digital camera to which the present technology is applied. [Diagram 2] FIG. 1 is a diagram showing an example of a JPEG file format that complies with JPEG (Joint Photographic Experts Group). [Diagram 3] FIG. 1 is a diagram illustrating an example of an ISO base media file format. [Figure 4] FIG. 2 is a diagram showing an example of a format of a HEIF file that complies with HEIF. [Diagram 5] FIG. 1 is a diagram showing an example of the format of a HEIF file in image item format. [Figure 6] FIG. 13 is a diagram showing an example of an iprp box. [Figure 7] FIG. 1 is a diagram showing an example of the format of a HEIF file in image sequence format. [Figure 8] FIG. 1 is a diagram showing an example of a trak box. [Figure 9] FIG. 13 is a diagram showing an example of a normal collection file in which main images and thumbnail images are stored. [Figure 10] FIG. 13 is a diagram showing an example of a first association-type collection file. [Figure 11] FIG. 13 is a diagram showing an example of a second association-type collection file. [Figure 12] FIG. 13 is a diagram showing an example of a third association-type collection file. [Figure 13] FIG. 13 is a diagram showing an example of a normal sequence file in which a track of main images and a track of thumbnail images of the main images are stored. [Figure 14] FIG. 13 is a diagram showing an example of an associated sequence file. [Figure 15] 11 is a flowchart outlining an example of a generation process for generating an associated HEIF file. [Figure 16] 11 is a flowchart outlining an example of a playback process for playing an associated HEIF file. [Figure 17] 11 is a flowchart illustrating an example of a reproduction process for reproducing a collection file. [Figure 18] 13 is a flowchart illustrating an example of a process for reading an image to be reproduced in step S32. [Figure 19] 13 is a flowchart illustrating a first example of a process for obtaining a playback target item ID in step S31. [Figure 20] 13 is a flowchart illustrating a second example of the process of acquiring a playback target item ID in step S31. [Figure 21] 13 is a flowchart illustrating an example of a process for acquiring a uuid as identification information of a RAW file of a predetermined main image from a first associative collection file. [Figure 22] 13 is a flowchart illustrating an example of a process for acquiring a uuid as identification information of a RAW file of a predetermined main image from a second associative collection file. [Figure 23] 13 is a flowchart illustrating an example of a process for acquiring a uuid as identification information of a RAW file of a predetermined main image from a third associative collection file. [Figure 24] 13 is a flowchart illustrating an example of a process for acquiring a list of item IDs of main images from a collection file. [Diagram 25]11 is a flowchart illustrating an example of a process for reproducing a thumbnail image of a main image (a frame) corresponding to predetermined time information from a sequence file. [Figure 26] 13 is a flowchart illustrating an example of a process for acquiring a uuid as identification information of a RAW file of a predetermined main image (frame) from an associative sequence file. [Figure 27] FIG. 13 is a diagram showing an example of storing a uuid in a RAW file when a RAW file of a main image is used as external data and an associated collection file is generated. [Figure 28] FIG. 13 is a diagram showing an example of storing a uuid in a RAW file when a RAW file of a main image is used as external data and an associated sequence file is generated. [Figure 29] FIG. 13 is a diagram showing an example of storing a uuid in a WAV file when a WAV file of a main image is used as external data and an associated collection file is generated. [Diagram 30] 13 is a diagram showing an example of storing a uuid in a WAV file when a WAV file of a main image is used as external data and an associated sequence file is generated. FIG. [Diagram 31] 1 is a block diagram showing an example configuration of a first embodiment of an image processing system to which the present technology is applied. [Diagram 32] FIG. 1 shows an example of a first encrypted HEIF file. [Diagram 33] FIG. 13 is a diagram showing an example of a second encrypted HEIF file. [Diagram 34] FIG. 13 is a diagram showing an example of a third encrypted HEIF file. [Diagram 35] FIG. 13 is a diagram showing an example of a fourth encrypted HEIF file. [Diagram 36] A figure showing an example of a fifth encrypted HEIF file. [Figure 37] A figure showing an example of a sixth encrypted HEIF file. [Figure 38]11 is a flowchart illustrating an example of pre-processing performed when the first encrypted HEIF file is handled by the image processing system 100. [Figure 39] 11 is a flowchart illustrating an example of a generation process for generating a first encrypted HEIF file when the first encrypted HEIF file is handled by the image processing system 100. [Diagram 40] 11 is a flowchart illustrating an example of a playback process for playing back a first encrypted HEIF file when the first encrypted HEIF file is handled by the image processing system 100. [Diagram 41] 13 is a flowchart illustrating an example of pre-processing performed when the second encrypted HEIF file is handled by the image processing system 100. [Diagram 42] 13 is a flowchart illustrating an example of a generation process for generating a second encrypted HEIF file when the second encrypted HEIF file is handled by the image processing system 100. [Diagram 43] A flowchart illustrating an example of a restriction process that restricts viewing of the main image obtained by decrypting the encrypted image by anyone other than users who can obtain the private key corresponding to the encrypted image when a second encrypted HEIF file is handled by image processing system 100. [Diagram 44] 13 is a flowchart illustrating an example of a playback process for playing back a second encrypted HEIF file when the second encrypted HEIF file is handled by the image processing system 100. [Diagram 45] 13 is a flowchart illustrating an example of pre-processing performed when the image processing system 100 handles a third encrypted HEIF file. [Figure 46] 13 is a flowchart illustrating an example of a generation process for generating a third encrypted HEIF file when the third encrypted HEIF file is handled by the image processing system 100. [Figure 47] 13 is a flowchart illustrating an example of a playback process for playing back a third encrypted HEIF file when the third encrypted HEIF file is handled by the image processing system 100. [Figure 48] 13 is a flowchart illustrating an example of pre-processing performed when the fourth encrypted HEIF file is handled by the image processing system 100. [Figure 49] 13 is a flowchart illustrating an example of a generation process for generating a fourth encrypted HEIF file when the fourth encrypted HEIF file is handled by the image processing system 100. [Figure 50] 13 is a flowchart illustrating an example of a playback process for playing back the fourth encrypted HEIF file when the fourth encrypted HEIF file is handled by the image processing system 100. [Figure 51] 13 is a flowchart illustrating an example of pre-processing performed when the fifth or sixth encrypted HEIF file is handled by the image processing system 100. [Figure 52] 13 is a flowchart illustrating an example of a generation process for generating a fifth or sixth encrypted HEIF file when the fifth or sixth encrypted HEIF file is handled by the image processing system 100. [Figure 53] A flowchart illustrating an example of a restriction process that restricts viewing of the main image obtained by decrypting an encrypted image by anyone other than a user who can obtain the private key corresponding to the encrypted image when the fifth or sixth encrypted HEIF file is handled by image processing system 100. [Figure 54] 13 is a flowchart illustrating an example of a playback process for playing back the fifth or sixth encrypted HEIF file when the fifth or sixth encrypted HEIF file is handled by the image processing system 100. [Figure 55] FIG. 11 is a block diagram showing an example configuration of a second embodiment of an image processing system to which the present technology is applied. [Figure 56] FIG. 2 is a block diagram showing an example of the configuration of a digital camera 210. [Figure 57] FIG. 13 is a diagram showing an example of a grid file. [Figure 58] FIG. 2 is a diagram for explaining an overview of the processing of an encrypted grid file performed by a digital camera 210. [Figure 59]FIG. 13 is a diagram showing an example of an encrypted grid file. [Figure 60] 11 is a flowchart illustrating a first example of pre-processing performed when an encrypted grid file is handled by the image processing system 200. [Figure 61] 13 is a flowchart illustrating a first example of a generation process for generating an encrypted grid file when the encrypted grid file is handled by the image processing system 200. [Figure 62] 13 is a flowchart illustrating an example of a playback process for playing back an encrypted grid file when the encrypted grid file is handled by the image processing system 200. [Figure 63] 13 is a flowchart illustrating a second example of pre-processing performed when an encrypted grid file is handled by the image processing system 200. [Figure 64] 13 is a flowchart illustrating a second example of a generation process for generating an encrypted grid file when the encrypted grid file is handled by the image processing system 200. [Figure 65] 13 is a flowchart illustrating a third example of a generation process for generating an encrypted grid file when the encrypted grid file is handled by the image processing system 200. [Figure 66] 1 is a block diagram showing an example of the configuration of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] <One embodiment of a digital camera to which the present technology is applied>

[0016] FIG. 1 is a block diagram showing an example of the configuration of an embodiment of a digital camera to which the present technology is applied.

[0017] The digital camera 10 has an optical system 11, an image sensor 12, a signal processing unit 13, a medium 14, interfaces 15 and 16, buttons / keys 17, a touch panel 18, a liquid crystal panel 19, a viewfinder 20, and an interface 21, etc.

[0018] The optical system 11 collects light from a subject onto the image sensor 12 .

[0019] The image sensor 12 receives light from the optical system 11 and performs photoelectric conversion to capture an image, thereby generating image data as an electrical signal, and supplies the data to a signal processing unit 13 .

[0020] The signal processing unit 13 has an optical system / image sensor control unit 41, an encoding control unit 42, a file control unit 43, a media control unit 44, an operation control unit 45, a display control unit 46, and a UI control unit 47.

[0021] The optical system / image sensor control unit 41 controls the optical system 11 and the image sensor 12, and supplies the image (data) obtained by imaging performed under that control to the encoding control unit .

[0022] The encoding control unit 42 supplies the image from the optical system / image sensor control unit 41 to the display control unit 46, and also encodes the image as necessary and supplies it to the file control unit 43. The encoding control unit 42 also decodes the image supplied from the file control unit 43 as necessary and supplies it to the display control unit 46.

[0023] The file control unit 43 generates a file storing the image supplied from the encoding control unit 42, and supplies the file to the media control unit 44. The file control unit 43 also plays the file supplied from the media control unit 44, i.e., reads data such as images stored in the file. For example, an image read from a file is supplied from the file control unit 43 to the encoding control unit 42.

[0024] The media control unit 44 controls the exchange of files between the medium 14 and the interfaces 15 and 16. For example, the media control unit 44 causes a file from the file control unit 43 to be recorded on the medium 14 or transmitted from the interfaces 15 and 16. The media control unit 44 also reads a file from the medium 14 or causes the interfaces 15 and 16 to receive a file and supply it to the file control unit 43.

[0025] In response to a user's operation of the buttons / keys 17 or the touch panel 18, the operation control unit 45 supplies an operation signal corresponding to the operation to a necessary block.

[0026] The display control unit 46 performs display control and the like to supply the images and the like supplied from the encoding control unit 42 to the liquid crystal panel 19, the viewfinder 20, and the interface 21 for display.

[0027] The UI control unit 47 is in charge of UI (User Interface) control.

[0028] The medium 14 is, for example, a storage medium such as an SD card. The interface 15 is, for example, a LAN (Local Area Network) interface such as WiFi (registered trademark) or Ethernet (registered trademark). The interface 16 is, for example, a USB (Universal Serial Bus) interface. The buttons / keys 17 and the touch panel 18 are operated by a user when inputting commands and other information to the digital camera 10. The touch panel 18 can be configured integrally with a liquid crystal panel 19. The liquid crystal panel 19 and the viewfinder 20 display images and the like supplied from the display control unit 46. The interface 21 is an interface that transmits at least images, such as HDMI (registered trademark) (High-Definition Multimedia Interface) or DP (Display Port).

[0029] In the digital camera 10 configured as described above, the optical system / image sensor control unit 41 generates, from the RAW data image (hereinafter also referred to as the RAW image) obtained by imaging with the image sensor 12, for example, a YUV image having the same resolution (number of pixels) as the RAW image, as the master image of the HEIF file, and supplies this to the encoding control unit 42.

[0030] The encoding control unit 42 generates, from the YUV main image, a YUV image (hereinafter also referred to as a screen nail image) having a lower resolution than the main image as a first other image based on the main image for display on the liquid crystal panel 19 or an external display, and generates, for example, a YUV image (hereinafter also referred to as a thumbnail image) having a lower resolution than the screen nail image as a second other image based on the main image for index display (list display). The encoding control unit 42 supplies, for example, the screen nail image to the liquid crystal panel 19 via the display control unit 46, and displays it as a so-called through image. As the thumbnail image, for example, an image with a long side of 320 pixels or less can be used. The ratio of the size (number of pixels) between the main image and the screen nail image as the first other image based on the main image, or the thumbnail image as the second other image based on the main image can be, for example, 200 times or less. Similarly, the ratio of the size between the screen nail image as the first other image based on the main image and the thumbnail image as the second other image based on the main image can be 200 times or less. As the screen nail image, for example, an image with a resolution of 4K or more can be adopted. Furthermore, as the screen nail image, for example, an image with a resolution of 4K (QFHD) or FHD can be adopted depending on the user's selection. Furthermore, images with the same resolution can be adopted as the main image and the screen nail image. When images with the same resolution are adopted as the main image and the screen nail image, the HEIF file can store both the main image and the screen nail image, or the main image can be stored without storing the screen nail image. When the main image is stored without storing the screen nail image in the HEIF file, the main image can be resized and used as the screen nail image.

[0031] In addition, the encoding control unit 42 encodes the main image, screennail image, and thumbnail image corresponding to the RAW image (the main image, screennail image, and thumbnail image generated from the same RAW image) as necessary, and supplies them to the file control unit 43 together with the RAW image.

[0032] The file control unit 43 generates a RAW file in which a RAW image is stored, and also generates a HEIF file or a JPEG file in which a corresponding main image, screennail image, and thumbnail image (a main image, screennail image, and thumbnail image generated from the same RAW image) are stored, and supplies the files to the media control unit 44. A HEIF file is a file that conforms to HEIF (High Efficiency Image File Format), and a JPEG file is a file that conforms to JPEG (Joint Photographic Experts Group).

[0033] The media control unit 44 records the RAW file and the HEIF file or JPEG file from the file control unit 43 on the media 14 or transmits them from the interface 15 or 16.

[0034] In the file control unit 43, whether to generate a HEIF file or a JPEG file can be selected according to, for example, a user operation. Also, as described later, HEIF files come in an image item format and an image sequence format, and whether to adopt the image item format or the image sequence format can be selected according to, for example, a user operation. Furthermore, the file control unit 43 can perform mutual conversion between a HEIF file and a JPEG file according to a user operation.

[0035] Furthermore, in generating a HEIF file, the file control unit 43 can associate internal data (data stored in the HEIF file) in the HEIF file that is associated with external data outside the HEIF file (data not stored in the HEIF file) with specific information that identifies the external data and store it in the HEIF file. A HEIF file in which internal data and specific information of external data associated with the internal data are stored in association with each other is also called an associated HEIF file. In an associated HEIF file, for example, the internal data and specific information can be stored in association with each other by storing association information that associates the internal data and the specific information.

[0036] <JPEGファイル>

[0037] FIG. 2 is a diagram showing an example of a JPEG file format that complies with JPEG (Joint Photographic Experts Group).

[0038] A JPEG file is configured by storing, for example, Exif metadata, a thumbnail image, XMP (Extensible Metadata Platform) (registered trademark) metadata, an MPF ​​indicating the storage location (position) of the main image and the simplified display image, etc., a main image, and an image for simplified display. For example, a screen nail image can be used as the image for simplified display.

[0039] <ISOベースメディアファイルフォーマット>

[0040] FIG. 3 is a diagram showing an example of the ISO base media file format.

[0041] HEIF (ISO / IEC 23008-12) is a file format that conforms to the ISO Base Media File Format (ISO / IEC 14496-12), and therefore HEIF files conform to the ISO Base Media File Format.

[0042] The ISO base media file format is composed of units called boxes, which serve as containers for storing data, and has a structure called a box structure.

[0043] A box has a type (box type) and actual data (data). The type indicates the type of actual data in the box. Actual data can be playable media data such as images (still images, videos), audio, subtitles, etc., attribute names (field names) and attribute values ​​(field values) of the attribute names (variables represented by the attribute names), and various other data.

[0044] Furthermore, a box can be used as actual data, that is, a box can have another box as actual data, which allows a hierarchical structure to be created.

[0045] A base media file that complies with the ISO base media file format can have an ftyp box, a moov box (MovieBox), a meta box (MetaBox), an mdat box (MediaDataBox), and the like. The ftyp box stores identification information that identifies the file format. The moov box can store a trak box, and the like. The meta box can store an iinf box, an iprp box, an iref box, an iloc box, and the like. The mdat box can store media data (AV data) and any other data.

[0046] HEIF complies with the above ISO Base Media File Formats.

[0047] <HEIFファイル>

[0048] FIG. 4 is a diagram showing an example of a format of a HEIF file that complies with HEIF.

[0049] HEIF files are broadly divided into image item and image sequence formats, with the image item format further divided into single image formats that contain only one item, and image collection formats that contain multiple items, as described below.

[0050] A HEIF file in the image item format has an ftyp box, a meta box, and an mdat box.

[0051] A HEIF file in image sequence format has an ftyp box, a moov box, and an mdat box.

[0052] Note that a HEIF file can have both a meta box and a moov box, not just one of them.

[0053] The ftyp box stores identification information for identifying the file format, for example, that the file is a HEIF file in image item format or image sequence format.

[0054] The meta box and moov box store metadata necessary for playback and management of the media data stored in the mdat box, such as the storage location of the media data.

[0055] The mdat box stores media data (AV data) and the like.

[0056] In the digital camera 10, which HEIF file to generate, the image item format or the image sequence format, can be selected according to, for example, a user's operation. Also, when encoding and storing an image in the mdat box of the HEIF file, only intra-encoding is permitted for the image item format, and intra-encoding and inter-encoding are permitted for the image sequence format. Therefore, for example, when high-speed access to data stored in the HEIF file is prioritized, generation of a HEIF file in the image item format can be selected, and when reducing the size (amount of data) of the HEIF file is prioritized, generation of a HEIF file in the image sequence format can be selected.

[0057] FIG. 5 is a diagram showing an example of the format of a HEIF file in image item format.

[0058] In a HEIF file in the image item format, information indicating that it is a HEIF file in the image item format, such as mif1, is stored (as an attribute value) in the ftyp box.

[0059] The meta box stores an iinf box, an iref box, an iprp box, and an iloc box.

[0060] The iinf box stores the number of items (attribute names and attribute values ​​representing) that are media data (AV data) stored in the mdat box. An item is a piece of data stored in the mdat box of a HEIF file in image item format; for example, one image (screen) is an item. In this specification, one image, whether a still image or a video, is also called a frame. One frame is one item.

[0061] The iref box stores information indicating the relationship between items. For example, the mdat box can store the corresponding main image, screennail image, and thumbnail image as items. When the mdat box stores item I1 as the main image, item I2 as the screennail image, and item I3 as the thumbnail image, the iref box stores information indicating that item I2 is the screennail image of the main image as item I1, and information indicating that item I3 is the thumbnail image of the main image as item I1.

[0062] The iprp box stores information about the properties of an item.

[0063] The iloc box stores information about the storage location of the items stored in the mdat box.

[0064] In the mdat box of the image item format (HEIF file), for example, a frame of an image is stored as an item. One or more items can be stored in the mdat box. Also, a frame as an item can be encoded and stored in the mdat box. However, encoding of a frame as an item stored in the mdat box of the image item format is limited to intra encoding. As an encoding method (codec) for encoding a frame as an item, for example, HEVC or the like can be adopted.

[0065] FIG. 6 is a diagram showing an example of the iprp box of FIG.

[0066] The iprp box stores an ipco box and an ipma box related to item properties. The ipco box stores the properties of the item stored in the mdat box, such as codec information related to the codec of the image as the item and image size information related to the size. The ipma box stores an index (pointer) to the properties stored in the ipco box of the item stored in the mdat box.

[0067] FIG. 7 is a diagram showing an example of the format of a HEIF file in image sequence format.

[0068] In a HEIF file in the image sequence format, information indicating that it is a HEIF file in the image sequence format, such as msf1, is stored in the ftyp box.

[0069] The moov box stores a trak box, which stores information about the track stored in the mdat box.

[0070] A track consists of one independent piece of media data, such as images and audio, that is played according to a timeline. For example, a track consists of one or more frames of images that form an elementary stream. For tracks stored in an mdat box, multiple tracks, for example, image and audio tracks recorded at the same time, can be played simultaneously.

[0071] The media data of a track is composed of units called samples. A sample is the smallest unit (access unit) for accessing media data in a HEIF file. Therefore, it is not possible to access media data in a HEIF file in units smaller than a sample.

[0072] For image media data, one sample is, for example, one frame, etc. For audio media data, one sample is, for example, one audio frame or the like defined by the standard of the audio media data.

[0073] In the mdat box of the image sequence format (HEIF file), the media data of a track is arranged in units called chunks. A chunk is a set of one or more samples arranged at logically consecutive addresses.

[0074] When multiple tracks of media data are stored in the mdat box, the multiple tracks are arranged in an interleaved manner in chunk units.

[0075] As described above, an mdat box in the image sequence format stores one or more tracks made up of media data such as images and audio.

[0076] In the mdat box, the frames of the images that make up the track can be encoded and stored. To encode the frames that make up the track stored in the mdat box in the image sequence format, long GOP can be used as the GOP (Group of Picture), and both intra-encoding and inter-encoding can be used. As a codec for encoding the frames that make up the track, for example, HEVC can be used.

[0077] FIG. 8 is a diagram showing an example of a trak box.

[0078] The trak box can store a tkhd box and an mdia box. The tkhd box stores header information of the track managed by the trak box, such as the creation date and time of the track. The mdia box stores the minf box and the like. The minf box stores the stbl box. The stbl box stores the stsd box, stsc box, stsz box, and stco box, which store the track samples and, in turn, information for accessing the chunks. The stsd box stores codec information related to the codec of the track. The stsc box stores the chunk size (the number of samples in one chunk). The stsz box stores the sample size. The stco box stores the chunk offset, i.e., the offset of the placement position of each chunk of the track stored in the mdat box.

[0079] Here, a HEIF file in the image item format is also called a collection file, and a HEIF file in the image sequence format is also called a sequence file. Furthermore, an associated HEIF file in the image item format is also called an associated collection file, and an associated HEIF file in the image sequence format is also called an associated sequence file.

[0080] The digital camera 10 can generate a HEIF file (including an associated HEIF file) that stores the main image and also any required screennail and / or thumbnail images.

[0081] <Collection file>

[0082] FIG. 9 is a diagram showing an example of a normal collection file in which main images and thumbnail images are stored.

[0083] Here, a normal collection file refers to a collection file in which internal data within the collection file is not associated with specific information of external data.

[0084] Now, it is assumed that frames (items) are encoded in HEVC and stored in the mdat box of the collection file.

[0085] In the ftyp box, heic, which indicates that the file format is an image item format and that the codec is HEVC, is stored as identification information for identifying the file format.

[0086] The number of items (item count) stored in the mdat box is stored in the iinf box. In Fig. 9, a total of four items (frames) are stored in the mdat box: the main image (hereinafter also referred to as main image Item#1) identified by item ID#1, the main image Item#2, the thumbnail image (hereinafter also referred to as thumbnail image Item#101) identified by item ID#101, and the thumbnail image Item#102. Therefore, the number of items is four. Note that the thumbnail image Item#101 is the thumbnail image of the main image Item#1, and the thumbnail image Item#102 is the thumbnail image of the main image Item#2.

[0087] The iinf box further stores an infe box for each item stored in the mdat box. The infe box stores an item ID that identifies the item and an item type. In FIG. 9, there are infe boxes for the main images Item#1 and Item#2, and the thumbnail images Item#101 and Item#102.

[0088] In the iref box, for example, a thmb box is stored as information relating items stored in the mdat box. In the thmb box, a reference source and a reference destination are stored in association with each other as information relating a main image and a thumbnail image of the main image. In the thmb box, the reference source represents the item ID of the main image, and the reference destination represents the item ID of the thumbnail image of the main image specified by the item ID of the reference source. Therefore, according to the reference destination associated with the reference source, it is possible to recognize the item ID of the thumbnail image of the main image specified by the item ID represented by the reference source. Also, according to the reference source associated with the reference destination, it is possible to recognize the item ID of the main image of the thumbnail image specified by the item ID represented by the reference destination.

[0089] The iprp box stores an ipco box and an ipma box, as described in Fig. 6. The ipco box stores properties of a frame as an item stored in the mdat box, such as codec information related to the codec and image size information related to the size, as described in Fig. 6. The ipma box stores an index to the properties stored in the ipco box of the item stored in the mdat box, as described in Fig. 6.

[0090] The iloc box stores information about the storage location of an item in the mdat box, as described in Fig. 6. In Fig. 9, the iloc box stores that the number of items is 4. Furthermore, the iloc box stores the offsets to the storage locations and sizes of the main images Item#1 and Item#2 and the thumbnail images Item#101 and Item#102 stored in the mdat box, in association with the item IDs.

[0091] Hereinafter, an associated collection file in which internal data and specific information of external data are stored in association with each other in the normal collection file of FIG. 9 will be described.

[0092] FIG. 10 is a diagram showing an example of the first association-type collection file.

[0093] Hereinafter, the external data to be associated with a main image as internal data within a HEIF file will be, for example, a RAW image of the main image (or a RAW file in which the RAW image is stored).

[0094] In the first associative collection file, by storing association information that associates the main image as internal data with specific information of the RAW file in which the RAW image is stored (the RAW image stored in the RAW file), the main image and specific information of the RAW file in which the RAW image is stored are stored in association with each other. Furthermore, in the first associative collection file, the association information is stored in a meta box.

[0095] The identifying information for a RAW file in which a RAW image as external data is stored can include the file name of the RAW file, a universally unique identifier (uuid) issued to the RAW file, a uniform resource locator (URL), or any other information that can identify the RAW file (the RAW image stored in it).

[0096] For the first association type collection file, an association information storage box in which association information is stored is defined as a new box stored in the meta box, and is stored in the meta box. The association information storage box of the first association type collection file stores association information in which, for example, an item ID that identifies a main image and a uuid as identification information that identifies a RAW file (in which a RAW image is stored) associated with the main image are associated. Furthermore, the association information storage box stores the number of main images (the number of main images) associated with the RAW file (the RAW image stored in the RAW file). The number of main images stored in the association information storage box is the number of main images associated with the RAW file, and is therefore equal to or less than the number of main images stored in the mdat box.

[0097] In Fig. 10, the uuid of the RAW file of the main image Item#1 (the uuid of the RAW image associated with the main image Item#1) is UUID#1, and the uuid of the RAW file of the main image Item#2 is UUID#2. If a RAW file with a uuid of UUID#i is described as a RAW file UUID#i, then in Fig. 10, the item ID#1 of the main image Item#1 is associated with the uuid of the RAW file UUID#1, and the item ID#2 of the main image Item#2 is associated with the uuid of the RAW file UUID#2, and association information is stored in the association information storage box.

[0098] FIG. 11 is a diagram showing an example of the second association-type collection file.

[0099] In the second associative collection file, similarly to the first associative collection file, the main image as internal data is stored in association information that associates the main image with the specific information of the RAW file as external data, and the main image and the specific information of the RAW file are stored in association with each other. However, in the second associative collection file, the association information is stored in the mdat box.

[0100] For the second association collection file, for example, the same association information as that of the first association collection file is stored as an item in the mdat box. In Fig. 11, the association information is stored in the mdat box as an item of item ID #201.

[0101] As described above, in the second associative collection file, the association information for the item Item#201 is stored in the mdat box, and accordingly the information stored in the meta box differs from that in the normal collection file of Fig. 9. In the second associative collection file, metadata of the association information for the item Item#201 is stored in the meta box.

[0102] Specifically, in the second associative collection file, the number of items stored in the iinf box and the iloc box changes from 4 in the case of FIG. 9 to 5, which is 4 plus 1 for item Item#201. Furthermore, an infe box for item Item#201 is added to the iinf box, and an offset to the storage location and size of item Item#201 are added to the iloc box. The infe box for item Item#201 stores item ID#201 of item Item#201 and an item type IDIF (identifying data info) indicating that item Item#201 is associative information. IDIF is a newly defined attribute value (field value) indicating that the item is associative information.

[0103] FIG. 12 is a diagram showing an example of the third association-type collection file.

[0104] In the third associative collection file, the specific information of the RAW file as external data is stored as an item for each specific information in the mdat box, and the association information that associates the main image as internal data with the specific information of the RAW file as external data is stored in the meta box, so that the main image and the specific information of the RAW file are stored in association with each other. However, in the third associative collection file, the association information is information that associates the item ID of the main image as an item with the item ID of the specific information (of the RAW file) as an item, and is stored in the cdsc box stored in the iref box in the meta box.

[0105] In the cdsc box, a reference source and a reference destination can be stored in association with each other as information relating items as a main image and specific information of a RAW file of the main image. In the cdsc box, the reference source indicates the item ID of the main image, and the reference destination indicates the item ID of specific information as an item of the RAW file of the main image specified by the item ID of the reference source.

[0106] 12, UUID#1, which is the uuid as the identifying information of the RAW file of the main image Item#1, is stored in the mdat box as item Item#201, and UUID#2, which is the uuid as the identifying information of the RAW file of the main image Item#2, is stored in the mdat box as item Item#202. Furthermore, a cdsc box storing association information that associates the item ID#1 of the main image Item#1 with the item ID#201 of the identifying information UUID#1 as the reference source and the reference destination, respectively, is stored in the iref box, and a cdsc box storing association information that associates the item ID#2 of the main image Item#2 with the item ID#202 of the identifying information UUID#2 as the reference source and the reference destination, respectively, is stored in the iref box.

[0107] <Sequence file>

[0108] FIG. 13 is a diagram showing an example of a normal sequence file in which a track of a main image and a track of thumbnail images of the main image are stored.

[0109] Here, a normal sequence refers to a sequence file in which internal data within the sequence file is not associated with specific information of external data.

[0110] Now, let us assume that frames are encoded in HEVC and stored in the mdat box of the sequence file.

[0111] In the ftyp box, hevc, which indicates that the file format is an image sequence format and that the codec is HEVC, is stored as identification information for identifying the file format.

[0112] As described in Fig. 7, the moov box stores a trak box that manages each track stored in the mdat box. In Fig. 13, the track of the main image identified by track ID #1 (hereinafter also referred to as track #1) and track #2 of the thumbnail image of the main image in track #1 are stored in the mdat box. Therefore, the moov box stores a trak box that manages track #1 and a trak box that manages track #2. The (frame of) n-th thumbnail image (from the beginning) of track #2 is the thumbnail image of the n-th main image in track #1.

[0113] A sequence file is useful, for example, when continuous shooting is performed with the digital camera 10, in the case where a plurality of frames of main images and thumbnail images obtained by the continuous shooting are each recorded as one track.

[0114] The tkhd box of the trak box that manages track #1 of the main images stores track ID #1 that identifies track #1, the image size of the main images that make up track #1, rotation information that indicates the orientation of the digital camera 10 when the main images were captured, and the creation date and time of track #1. The tkhd box of the trak box that manages track #2 of the thumbnail images stores track ID #2 that identifies track #2 and the creation date and time of track #2.

[0115] In addition to the tkhd box and mdia box described in Fig. 7, the trak box can store a tref box. The tref box stores a track ID that identifies other tracks related to the track managed by the trak box in which the tref box is stored, and information that indicates the contents of the track. In Fig. 13, a tref box is provided in a trak box that manages track #2. The tref box stores information that indicates that the other track related to track #2 is track #1 (track_ID=1) and that the data that constitutes track #2 is a thumbnail image (track #2 is a thumbnail image track) (type=thmb).

[0116] In addition to the MINF box described in Fig. 8, the mdia box of the trak box can store an hdlr box. The hdlr box stores information indicating the type of data that constitutes the track managed by the trak box in which the hdlr box is stored. The hdlr box stored in the trak box that manages track #1 of the main image (in the mdia box in which it is stored) stores information (pict) indicating that the data that constitutes track #1 is a picture (frame), and the hdlr box stored in the trak box that manages track #2 of the thumbnail images stores information indicating that the data that constitutes track #2 is a picture.

[0117] The minf box is as explained in FIG.

[0118] Hereinafter, an associated sequence file in which internal data and specific information of external data are stored in association with each other in the normal sequence file of FIG. 13 will be described.

[0119] FIG. 14 is a diagram showing an example of an associated sequence file.

[0120] In an associated sequence file, track #3 of the (elementary) stream (Meta ES) of the uuid as specific information of the RAW file as external data is added to the mdat box, and the trak box that manages track #3 is added to the moov box.

[0121] Here, track #1 is a time series of one or more frames of the main image lined up on the timeline, and track #3 is a time series of the uuids of the RAW files of each frame of the main image lined up on the timeline.

[0122] The nth uuid (from the beginning) of track #3 is specific information of the RAW file of the nth frame of the main image of track #1. In addition, (data of) a plurality of tracks stored in the mdat box can be played back in synchronization according to time information on one timeline. Therefore, by storing track #1 of the main image and track #3 of the uuid (stream) of the RAW file of each frame of the main image constituting track #1 in the mdat box, the nth frame of the main image of track #1 and the uuid of the RAW file of the main image (frame) are stored in association with each other. In this case, it can be said that the frame of the main image of track #1 and the uuid of the RAW file of the main image (frame) are associated with each other by time information on the timeline.

[0123] In addition, the nth uuid (from the beginning) of track #3 is specific information for the RAW file of the nth frame of track #1, and the main images (frames) that make up track #1 and the uuids that make up track #3 can also be considered to be related by the order in which they are arranged on the tracks.

[0124] In an associative sequence file, when track #3 of the uuid of the RAW file is added to the mdat box, the trak box that manages that track #3 is added to the moov box.

[0125] The trak box that manages track #3 of the uuid of the RAW file stores a tkhd box, a tref box, an mdia box, and the like.

[0126] The tkhd box of the trak box that manages track #3 stores track ID #3 that identifies track #3 and the creation date and time of track #3.

[0127] The tref box of the trak box managing track #3 stores a track ID that identifies other tracks related to track #3 managed by the trak box in which the tref box is stored, and information that indicates the contents of track #3, etc. The uuid that constitutes track #3 is specific information of the RAW file of the main image that constitutes track #1, and since track #3 is related to track #1, the tref box of the trak box managing track #3 in Fig. 14 stores information that indicates that the other track related to track #3 is track #1 (track_ID=1) and that track #3 is a track of metadata (specific information here) (type=cdsc).

[0128] In the mdia box of the trak box that manages Track #3, the hdlr box and the minf box are stored. In the trak box that manages Track #3, the hdlr box stores information indicating that the data constituting Track #3 is metadata (of the main image), and the minf box stores the stsc box, the stsc box, the stsz box, and the stco box for Track #3.

[0129] <Generation and Playback of HEIF Files>

[0130] Figure 15 is a flowchart for explaining an overview of an example of a generation process for generating an associated HEIF file.

[0131] In the generation process, in step S11, the file control unit 43 generates a uuid as specific information of the RAW file (RAW image) of the frame of the main image, and the process proceeds to step S12.

[0132] In step S12, the file control unit 43 assigns the uuid generated in step S11 to the RAW file (RAW image) of the frame of the main image, and the process proceeds to step S13.

[0133] In step S13, the file control unit 43 generates an associated HEIF file in which the frame of the main image and the uuid of the RAW file of the frame are associated and stored in the HEIF file, and ends the generation process.

[0134] Figure 16 is a flowchart for explaining an overview of an example of a playback process for playing back an associated HEIF file.

[0135] In the playback process, in step S21, the file control unit 43 generates, for example, a handle list of handles for identifying each of the frames of the main image stored in the HEIF file stored in the medium 14, and the process proceeds to step S22.

[0136] Here, the handle of the main image frame includes the file name of the HEIF file in which the frame is stored. The handle of the main image frame (item) stored in the collection file further includes the item ID of the frame. The handle of the main image frame stored in the sequence file further includes the time information of the frame. The handle of the main image frame allows the frame for that handle to be uniquely identified (specified).

[0137] In addition, the handle of a main image frame stored in a sequence file can include the track ID of the track that contains the frame and the order of the frame in that track (the number of the frame) instead of the frame's time information.

[0138] Whether the sequence file contains one or multiple tracks made up of frames of the main image, the time information of each frame is unique. Therefore, even if multiple tracks are stored in the sequence file, the frame of the time information included in the handle can be uniquely identified from the frames that make up each of the multiple tracks, according to the frame time information. Therefore, if the handle of the main image frame contains the time information of that frame, the frame for the time information can be uniquely identified even if there is no track ID of the track in which the frame exists.

[0139] The handle list can be generated for all frames of the main image stored in the HEIF file stored on media 14, or it can be generated for only frames narrowed down by specific criteria, such as frames with a specific creation date and time.

[0140] After the file control unit 43 generates the handling list, the HEIF file is accessed by referring to the handling list as necessary.

[0141] In step S22, for example, upon waiting for the user to operate the digital camera 10 to display thumbnail images, the UI control unit 47 requests the file control unit 43 to display thumbnail images. In response to the request to display thumbnail images from the UI control unit 47, the file control unit 43 reads out from the HEIF file (a frame of) a thumbnail image of the main image frame identified by the handle in the handle list. The file control unit 43 then causes a list of thumbnail images read out from the HEIF file to be displayed, for example, on the liquid crystal panel 19 (FIG. 1), and the process proceeds from step S22 to step S23.

[0142] In step S23, for example, after the user selects a desired thumbnail (frame) from a list of thumbnail images, the UI control unit 47 requests the file control unit 43 for a main image corresponding to the thumbnail image selected by the user. In response to the request for the main image from the UI control unit 47, the file control unit 43 reads the main image from the HEIF file. The file control unit 43 can display the main image read from the HEIF file on the liquid crystal panel 19 as necessary.

[0143] Alternatively, the UI control unit 47 requests the uuid of the RAW file of the main image corresponding to the thumbnail image selected by the user from the file control unit 43. In response to the request for the uuid from the UI control unit 47, the file control unit 43 reads the uuid from the associated HEIF file. The file control unit 43 can access the RAW file identified by the uuid read from the associated HEIF file as necessary.

[0144] FIG. 17 is a flowchart illustrating an example of a playback process for playing back a collection file.

[0145] In step S31, the file control unit 43 acquires the item ID of the image to be reproduced (hereinafter, also referred to as the reproduction target item ID) which is the image (item) to be reproduced, and the process proceeds to step S32.

[0146] When obtaining the playback target item ID, for example, a main image identified by an arbitrary handle in the handle list, a thumbnail image of that main image, a thumbnail image selected by the user from a list of thumbnail images (hereinafter also referred to as a selected thumbnail image), or the main image of the selected thumbnail image is taken as the playback target image, and the item ID (playback target item ID) of that playback target image is obtained.

[0147] In step S32, the file control unit 43 reads out the image to be played back in accordance with the playback target item ID acquired in step S31.

[0148] In reading the image to be played back, the image to be played back specified by the playback target item ID is read out from the collection file.

[0149] FIG. 18 is a flowchart illustrating an example of the process of reading the image to be reproduced in step S32 of FIG.

[0150] In step S41, the file control unit 43 searches the iloc box of the collection file (FIGS. 9 to 12) for the playback target item ID, and the process proceeds to step S42.

[0151] In step S42, the file control unit 43 reads out the offset and size associated with the playback target item ID found in the iloc box in step S41, and the process proceeds to step S43.

[0152] In step S43, the file control unit 43 reads out the image to be played back stored in the mdat box of the collection file in accordance with the offset and size associated with the playback target item ID, and the process ends.

[0153] FIG. 19 is a flowchart illustrating a first example of the process of acquiring the playback target item ID in step S31 of FIG.

[0154] That is, FIG. 19 shows an example of acquiring the item ID of a thumbnail image which is an image to be played back.

[0155] In FIG. 19, it is assumed that the file control unit 43 recognizes the item ID of the main image of the thumbnail image as the image to be played back from the handle, for example.

[0156] In step S51, the file control unit 43 searches for a thmb box whose reference source matches the item ID of the main image from among the thmb boxes in the iref box of the collection file (FIGS. 9 to 12), and the process proceeds to step S52.

[0157] In step S52, file control unit 43 reads out the reference in the thmb box whose reference source matches the item ID of the main image found in step S51 as the item ID of the thumbnail image to be played back, and then the process ends.

[0158] FIG. 20 is a flowchart illustrating a second example of the process of acquiring the playback target item ID in step S31 of FIG.

[0159] That is, FIG. 20 shows an example of acquiring the item ID of the main image, which is the image to be reproduced, with the main image being the image to be reproduced.

[0160] In FIG. 20, for example, it is assumed that the user selects a thumbnail image (selected thumbnail image) from a list of thumbnail images, and file control unit 43 recognizes the item ID of the selected thumbnail image.

[0161] In step S61, the file control unit 43 searches the thmb boxes in the iref box of the collection file (FIGS. 9 to 12) for a thmb box whose reference matches the item ID of the selected thumbnail image, and the process proceeds to step S62.

[0162] In step S62, the file control unit 43 reads out the reference source in the thmb box whose reference destination matches the item ID of the selected thumbnail image found in step S61 as the item ID of the main image to be played back, and the process ends.

[0163] FIG. 21 is a flowchart illustrating an example of a process for acquiring the uuid as identifying information of a RAW file of a predetermined main image from the first associative collection file of FIG.

[0164] In FIG. 21, for example, it is assumed that the file control unit 43 recognizes the item ID of a predetermined main image by a handle list or the like.

[0165] In step S71, the file control unit 43 searches for the item ID of a predetermined main image from the association information in the association information storage box of the first association-type collection file (FIG. 10), and the process proceeds to step S72.

[0166] In step S72, the file control unit 43 reads out the uuid associated with the item ID of the predetermined main image searched for in step S71 from the association information, and the process ends.

[0167] The file control unit 43 can access the RAW file of a predetermined main image using the uuid read out as described above.

[0168] FIG. 22 is a flowchart illustrating an example of a process for acquiring the uuid as identifying information of a RAW file of a predetermined main image from the second associative collection file of FIG.

[0169] In FIG. 22, for example, it is assumed that the file control unit 43 recognizes the item ID of a predetermined main image by a handle list or the like.

[0170] In step S81, the file control unit 43 searches for an infe box of item type IDIF indicating that the item is association information from the infe box in the iinf box of the second associative collection file (FIG. 11), and the process proceeds to step S82.

[0171] In step S82, the file control unit 43 reads out the item ID of the association information as an item from the infe box of the item type IDIF searched for in step S81, and the process proceeds to step S83.

[0172] In step S83, the file control unit 43 searches the iloc box of the second associative collection file for the item ID of the association information read out in step S82, and the process proceeds to step S84.

[0173] In step S84, the file control unit 43 reads out the offset and size associated with the item ID of the association information searched for in step S83 in the iloc box, and the process proceeds to step S85.

[0174] In step S85, the file control unit 43 reads out the association information as an item stored in the mdat box of the second association type collection file according to the offset and size associated with the item ID of the association information read in step S84, and the process proceeds to step S86.

[0175] In step S86, the file control unit 43 searches for the item ID of a predetermined main image from the association information read out in step S85, and the process proceeds to step S87.

[0176] In step S87, the file control unit 43 reads out the uuid associated with the item ID of the predetermined main image searched for in step S86 from the association information, and the process ends.

[0177] The file control unit 43 can access the RAW file of a predetermined main image using the uuid read out as described above.

[0178] FIG. 23 is a flowchart illustrating an example of a process for acquiring the uuid as identifying information of a RAW file of a predetermined main image from the third associative collection file of FIG.

[0179] In FIG. 23, for example, it is assumed that the file control unit 43 recognizes the item ID of a predetermined main image by a handle list or the like.

[0180] In step S91, the file control unit 43 searches for a cdsc box whose reference source matches the item ID of a predetermined main image from the cdsc boxes in the iref box of the third associative collection file (FIG. 12), and the process proceeds to step S92.

[0181] In step S92, the file control unit 43 reads out the reference in the cdsc box searched for in step S91, whose reference source matches the item ID of the specified main image, as the item ID of the specific information of the RAW file of the specified main image as an item, and the processing proceeds to step S93.

[0182] In step S93, the file control unit 43 searches the iloc box of the third associative collection file for the item ID of the specific information as the item read out in step S92, and the process proceeds to step S94.

[0183] In step S94, the file control unit 43 reads out the offset and size associated with the item ID of the specific information searched for in step S93 in the iloc box, and the process proceeds to step S95.

[0184] In step S95, the file control unit 43 reads out the uuid as specific information of the RAW file of a specified main image stored in the mdat box of the third associative collection file, according to the offset and size associated with the item ID of the specific information read in step S94, and the processing ends.

[0185] The file control unit 43 can access the RAW file of a predetermined main image using the uuid read out as described above.

[0186] FIG. 24 is a flowchart illustrating an example of a process for acquiring a list of item IDs of main images from a collection file.

[0187] The process of acquiring the list of item IDs of main images from the collection file is performed, for example, when generating a handle list.

[0188] In step S101, the file control unit 43 reads out item IDs from all infe boxes in the iinf box of the collection file (Figures 9 to 12) and registers them in a list of item IDs of main images (hereinafter also referred to as the main image list), and the process proceeds to step S102.

[0189] In step S102, the file control unit 43 reads out the item IDs that are referenced from all boxes in the iref box of the collection file, and removes them from the main image list, and the process ends.

[0190] After the above processing, the item ID registered in the main image list becomes the item ID of the main image.

[0191] FIG. 25 is a flowchart illustrating an example of a process for reproducing a thumbnail image of a main image (a frame thereof) corresponding to predetermined time information from a sequence file.

[0192] In FIG. 25, for example, it is assumed that the file control unit 43 recognizes the time information (or order) of a predetermined main image by a handle list or the like.

[0193] In step S111, the file control unit 43 searches the trak boxes in the moov box of the sequence file (Figures 13 and 14) for a trak box in which information indicating that the data constituting the track is a thumbnail image is stored in the tref box, that is, a trak box in which the type in the tref box is thmb, as the trak box that manages the track of the thumbnail image of the main image for the specified time information, and the processing proceeds to step S112.

[0194] In step S112, the file control unit 43 reads out the track ID in the tkhd box in the trak box searched for in step S111 as the track ID of the track of the thumbnail image of the main image for the specified time information, and the process proceeds to step S113.

[0195] In step S113, the file control unit 43 plays the track with the track ID read out in step S112, and obtains from that track a thumbnail image (frame) for the specified time information (or sequence) as a thumbnail image of the main image for the specified time information, and then the processing ends.

[0196] The process of playing back image tracks stored in a sequence file is similar to the process of playing back video from an MP4 file.

[0197] FIG. 26 is a flowchart illustrating an example of a process for acquiring the uuid as identification information of the RAW file of a predetermined main image (frame) from an associative sequence file.

[0198] In FIG. 26, for example, it is assumed that the file control unit 43 recognizes the time information (or order) of a predetermined main image by a handle list or the like.

[0199] In step S121, the file control unit 43 searches the trak boxes in the moov box of the associative sequence file (Figure 14) for a trak box in which information indicating that the data constituting a track is specific information is stored in the tref box, that is, a trak box in which the type in the tref box is cdsc, as the trak box that manages the track of specific information, and the processing proceeds to step S122.

[0200] In step S122, the file control unit 43 reads out the track ID in the tkhd box in the trak box searched for in step S121 as the track ID of the specific information track, and the process proceeds to step S123.

[0201] In step S123, the file control unit 43 obtains the uuid as specific information for the time information (or order) of the specified main image from the track of the track ID read out in step S122, as the uuid of the RAW file of the specified main image, and the processing ends.

[0202] The file control unit 43 can access the RAW file of a predetermined main image using the uuid acquired as described above.

[0203] As described above, the file control unit 43 generates and plays an associated HEIF file in which a HEIF file conforming to HEIF is stored in association with a main image within the HEIF file and specific information identifying external data outside the HEIF file that is associated with the main image, so that the main image stored in the HEIF file can be associated with external data outside the HEIF file.

[0204] Furthermore, when a uuid is used as the identification information, even if the file name of the external data is changed, the uuid can maintain the association between the main image in the HEIF file and the external data after the file name is changed.

[0205] <Storage of specific information assigned to external data>

[0206] FIG. 27 is a diagram showing an example of storing uuid in a RAW file when a RAW file of a main image is used as external data (a file in which the external data is stored) and an associative collection file is generated.

[0207] In FIG. 27, the first associative collection file is used as the associative collection file.

[0208] A RAW file has an area called a MakerNote as part of an area for storing Exif attribute information as metadata.

[0209] The file control unit 43 can store the uuid assigned to a RAW file (RAW image) in, for example, a marker note of the RAW file.

[0210] 27, main images Item#1, Item#2, Item#3, and Item#4 are stored as four items in the association collection file, and RAW files #1, #2, #3, and #4 are generated in which RAW images of the main images Item#1, Item#2, Item#3, and Item#4 are stored. A UUID#i is assigned to the RAW file #i (RAW image), and as association information associating the main image Item#i with the UUID#i of the RAW file #i of the main image Item#i, association information in which an item ID#i identifying the main image Item#i is associated with the UUID#i of the RAW file #i associated with the main image Item#i is stored in an association information storage box.

[0211] FIG. 28 is a diagram showing an example of storing uuid in a RAW file when a RAW file of a main image is used as external data and an associated sequence file is generated.

[0212] When generating an associated sequence file, the file control unit 43 can store the uuid assigned to the RAW file in the marker note of the RAW file, just as when generating the associated collection file described in Figure 27.

[0213] 28, track #1 consisting of four frames of main images #1, #2, #3, #4 is stored in the associative sequence file, and RAW files #1, #2, #3, #4 are generated in which RAW images of main images #1, #2, #3, #4 are stored. UUID #i is assigned to RAW file #i, and track #3 is stored in the associative sequence file, in which UUID #i of RAW file #i is arranged to have the same time information as main image #i corresponding to RAW file #i (RAW image).

[0214] As described above, track #3 is constructed by arranging the UUID #i of RAW file #i so that it has the same time information as the main image #i corresponding to RAW file #i, and the i-th main image #i of track #1 and the i-th UUID #i of track #3, i.e., the UUID #i of the RAW file #i of main image #i, are associated and stored in an associated sequence file.

[0215] In the above, a RAW file (RAW image) of the main image is used as the external data, but other data can be used as the external data. For example, audio (sound) recorded together with capturing the main image can be used as the external data. For example, a WAV file in WAV format or an MP4 file in MP4 format can be used as the file storing the audio. In the following, for example, a WAV file will be used as the file storing the audio.

[0216] FIG. 29 is a diagram showing an example of storing uuid in a WAV file when a WAV file of a main image is adopted as external data (a file in which the external data is stored) and an associative collection file is generated.

[0217] In FIG. 29, the first associative collection file is used as the associative collection file.

[0218] A WAV file has an area called a List chunk as a part of the area in which metadata is written.

[0219] The file control unit 43 can store the uuid assigned to a WAV file (audio) in, for example, a List chunk of the WAV file.

[0220] 29, main images Item#1, Item#2, Item#3, and Item#4 are stored as four items in the associated collection file, and WAV files #1, #2, #3, and #4 for the main images Item#1, Item#2, Item#3, and Item#4 are generated. A UUID#i is assigned to the WAV file #i (audio), and as association information associating the main image Item#i with the UUID#i of the WAV file #i of the main image Item#i, association information in which the item ID#i identifying the main image Item#i corresponds to the UUID#i of the WAV file #i associated with the main image Item#i is stored in the association information storage box.

[0221] FIG. 30 is a diagram showing an example of storing uuid in a WAV file when a WAV file of a main image is used as external data and an associated sequence file is generated.

[0222] When generating an associative sequence file, the file control unit 43 can store the uuid assigned to a WAV file in the List chunk of the WAV file, just as when generating an associative collection file as described with reference to FIG. 29.

[0223] In Fig. 30, track #1 consisting of four frames of main images #1, #2, #3, #4 is stored in the associative sequence file, and WAV files #1, #2, #3, #4 of main images #1, #2, #3, #4 are generated. UUID #i is assigned to WAV file #i, and track #3 is stored in the associative sequence file, which is configured by arranging the UUID #i of WAV file #i so as to have the same time information as the main image #i corresponding to WAV file #i.

[0224] As described above, track #3 is constructed by arranging the UUID #i of WAV file #i so that it has the same time information as the main image #i corresponding to WAV file #i, and the i-th main image #i of track #1 and the i-th UUID #i of track #3, i.e., the UUID #i of WAV file #i of main image #i, are associated and stored in the associated sequence file.

[0225] In addition to HEIF files, this technology can also be applied to files other than HEIF files that have a box structure, such as ISO-based media files, MP4 files, and Miaf files.

[0226] Furthermore, the present technology can also be applied to, for example, a file that does not have a box structure and stores an image (main image) and another image that is the main image with a lowered resolution.

[0227] Furthermore, this technology can be applied to associating external data with the main image in a HEIF file, as well as with screennail and thumbnail images in a HEIF file.

[0228] The technology can also be applied in other applications, for example, when associating external data with internal data other than images, such as the main image in a HEIF file.

[0229] <First embodiment of image processing system to which the present technology is applied>

[0230] FIG. 31 is a block diagram showing an example configuration of a first embodiment of an image processing system to which the present technology is applied.

[0231] In Fig. 31, an image processing system 100 is composed of the digital camera 10 of Fig. 1 and an external computer 101 such as a cloud computer. The digital camera 10 and the external computer 101 can communicate with each other via a network such as the Internet.

[0232] Here, in order to protect the so-called copyright of an image captured by the digital camera 10 or to protect the portrait rights of people appearing in the image, it may be desirable to restrict viewing of the image.

[0233] Therefore, in the image processing system 100, by generating an encrypted HEIF file using an associated HEIF file in the digital camera 10, it is possible to restrict viewing of images captured by the digital camera 10.

[0234] In the following, for simplicity, the viewing of the main image stored in the HEIF file is restricted. However, this technology can restrict the viewing of any media data other than the main image stored in the HEIF file. This technology can also be applied to ISO-based media files (files conforming to ISO-based media files) other than HEIF files, and further, any files other than ISO-based media files.

[0235] In the digital camera 10 (Figure 1), the file control unit 43 can generate an encrypted HEIF file by storing an encrypted image, which is a main image stored in the HEIF file encrypted with a first encryption key, and an encrypted encryption key, which is an encryption key for encrypting the first encryption key with a second encryption key, in association with each other.

[0236] The file control unit 43 can use the association between the main image stored in the associated HEIF file and the identifying information of the external data to associate the encrypted image stored in the encrypted HEIF file with the encryption encryption key that encrypted the first encryption key used for encryption to obtain the encrypted image. That is, the file control unit 43 can associate the encrypted image stored in the encrypted HEIF file with the encryption encryption key in the same way as the association between the main image stored in the associated HEIF file and the identifying information of the external data.

[0237] In addition, the file control unit 43 decrypts the encryption key of the encrypted HEIF file into a first encryption key, and can use the first encryption key obtained by this decryption to decrypt the encrypted image into the original main image.

[0238] The first encryption key and the second encryption key can be requested from the external computer 101 by the digital camera 10. When the external computer 101 receives a request for the first encryption key and the second encryption key from the digital camera 10, it generates the first encryption key and the second encryption key and provides (transmits) them to the digital camera 10. Note that the first encryption key and the second encryption key can be generated by the digital camera 10. Also, one of the first encryption key and the second encryption key can be generated by the digital camera 10, and the other can be generated by the external computer 101.

[0239] The first encryption key and the second encryption key may be encryption keys of any encryption method. Also, the first encryption key and the second encryption key may be encryption keys of the same encryption method or encryption keys of different encryption methods.

[0240] In this embodiment, for example, a common key of a common key cryptosystem is adopted as the first encryption key, and a public key of a public key cryptosystem is adopted as the second encryption key.

[0241] The processing load for generating a common key is light compared to the processing load for generating a public key and a private key in a public key cryptosystem, and therefore can be easily performed in the digital camera 10. Furthermore, the common key allows encryption and decryption of images, which have a relatively large amount of data, to be performed in a short time compared to the public key and private key.

[0242] Furthermore, by employing a public key of a public key cryptosystem as the second encryption key, it is possible to improve the confidentiality of the common key encrypted with that public key.

[0243] In this embodiment, the common key is generated by the digital camera 10, and the public key and private key are generated and managed by the external computer 101.

[0244] As described above, in digital camera 10, by generating an encrypted HEIF file in which an encrypted image obtained by encrypting the main image stored in the HEIF file with a common key as a first encryption key and an encrypted encryption key obtained by encrypting the common key with a public key as a second encryption key are stored in association with each other, the viewing of the main image can be restricted so that only users who can obtain (acquire) the private key (or the common key before it is encrypted into the encryption key) corresponding to the public key used for encryption to obtain the encryption key can view the main image.

[0245] Among associated HEIF files, the following describes an encrypted HEIF file that uses, for example, the third associated collection file in Fig. 12 (association). However, as the encrypted HEIF file, it is possible to generate a HEIF file that uses an associated HEIF file other than the third associative collection file (the first associative collection file, the second associative collection file, or the associated sequence file).

[0246] In the following explanation of encrypted HEIF files, parts that are common to the HEIF files (including associated HEIF files) already explained will be omitted where appropriate.

[0247] <Encrypted HEIF file>

[0248] FIG. 32 is a diagram showing an example of the first encrypted HEIF file.

[0249] In FIG. 32, items Item#1, Item#2, Item#201, and Item#202 are stored in the mdat box. Item#1 is an encrypted image obtained by encrypting main image 1 with a common key (by common key encryption method), and item Item#2 is an encrypted image obtained by encrypting main image 2 with a common key. Different common keys can be used to encrypt different main images. For example, the common key used to encrypt the encrypted image as item Item#1 (hereinafter also referred to as encrypted image Item#1) is different from the common key used to encrypt the encrypted image Item#2. Note that the same common key can be used to encrypt different main images, as necessary.

[0250] Item#201 is an encryption key obtained by encrypting the common key used to encrypt main image 1 with a public key (by public key cryptography), and item#202 is an encryption key obtained by encrypting the common key used to encrypt main image 2 with a public key. A different public key (corresponding to a private key) can be used to encrypt different public keys. For example, the public key used to encrypt to obtain the encryption key as item#201 (hereinafter also referred to as encrypted encryption key Item#201) is different from the public key used to encrypt to obtain encrypted encryption key Item#202. Note that the same public key can be used to encrypt different common keys, if necessary.

[0251] In Fig. 32, a white padlock symbol indicates that the data has been encrypted with a common key, and a padlock symbol with a diagonal line indicates that the data has been encrypted with a public key. This is the same in the figures that will be described later.

[0252] In the first encrypted HEIF file, for example, association information that associates the main image stored in the mdat box with specific information is stored in the meta box, so that the encrypted image Item#i stored in the mdat box is associated with the encrypted encryption key Item#200+i, which is obtained by encrypting the common key used to obtain the encrypted image Item#i with the public key, in a similar manner to the third associated collection file (Figure 12) that associates the main image with specific information.

[0253] The association information associating the encrypted image Item#i with the encrypted encryption key Item#200+i is information that associates the item ID#i of the encrypted image Item#i with the item ID of the encrypted encryption key Item#200+i, similar to the third association collection file, and is stored in a cdsc box which is stored in the iref box in the meta box.

[0254] In Figure 32, the iref box stores a cdsc box that stores association information that associates item ID#1 of encrypted image Item#1 with item ID#201 of encrypted encryption key Item#201, which is obtained by encrypting the common key used in the encryption to obtain the encrypted image Item#1 with a public key, as the reference source and reference destination, respectively, and the iref box stores a cdsc box that stores association information that associates item ID#2 of encrypted image Item#2 with item ID#202 of encrypted encryption key Item#202, which is obtained by encrypting the common key used in the encryption to obtain the encrypted image Item#2 with a public key, as the reference source and reference destination, respectively.

[0255] For encrypted image Item#i stored in the first encrypted HEIF file, if a private key corresponding to that encrypted image Item#i, i.e., a private key corresponding to the public key used to encrypt the common key used to encrypt the encrypted image Item#i, can be obtained, the file control unit 43 can decrypt the encrypted encryption key Item#200+i associated with the encrypted image Item#i with the private key to a common key, and can decrypt the encrypted image Item#i to the original main image i with the common key obtained by the decryption. Therefore, viewing of the main image i can be restricted to users who can obtain the private key corresponding to the encrypted image Item#i.

[0256] In the first encrypted HEIF file, in the infe box of the encrypted encryption key Item#200+i, the item type of the encrypted encryption key Item#200+i is set to, for example, ECKI (Encryption key for Item), which indicates an encryption key (here, a common key). The attribute value of the item time, which indicates an encryption key, is not limited to ECKI. When decrypting the encrypted image Item#i, the file control unit 43 specifies the item ID of the reference destination of the cdsc box whose reference source is the item ID#i of the encrypted image Item#i as the encrypted encryption key Item#200+i corresponding to the encrypted image Item#i, that is, the item ID#200+i of the encrypted encryption key Item#200+i that encrypted the common key used to encrypt the encrypted image Item#i, and obtains (reads) the encrypted encryption key Item#200+i of the item ID#200+i from the mdat box according to the item ID#200+i, and decrypts it. Whether or not the item identified by the item ID referenced by the cdsc box is an encrypted encryption key can be recognized (determined) from the item type set in the infe box of the item identified by that item ID.

[0257] FIG. 33 is a diagram showing an example of a second encrypted HEIF file.

[0258] In Fig. 33, like Fig. 32, encrypted images Item#1 and Item#2, and encrypted encryption keys Item#201 and Item#202 are stored in the mdat box. Furthermore, in Fig. 33, plaintext common keys Item#301 and Item#302 are stored in the mdat box. Encrypted encryption key Item#200+i is obtained by encrypting plaintext common key Item#300+i as an item with a private key. Here, plaintext means data in an unencrypted state.

[0259] In the second encrypted HEIF file, the plaintext common key Item#300+i is stored in association with the encrypted image Item#i (the corresponding main image i), just like the encrypted encryption key Item#200+i corresponding to the common key Item#300+i, i.e., the encrypted encryption key Item#200+i obtained by encrypting the common key Item#300+i.

[0260] In Fig. 33, in addition to the cdsc box shown in Fig. 32, a cdsc box storing association information associating item ID#1 of encrypted image Item#1 with item ID#301 of plaintext common key Item#301 (common key Item#301 used in encryption to obtain encrypted image Item#1) corresponding to the encrypted image Item#1 as a reference source and a reference destination, respectively, is stored in the iref box. Furthermore, a cdsc box storing association information associating item ID#2 of encrypted image Item#2 with item ID#302 of plaintext common key Item#302 corresponding to the encrypted image Item#2 as a reference source and a reference destination, respectively, is stored in the iref box.

[0261] In the second encrypted HEIF file, the plaintext common key Item#300+i stored in the mdat box can be deleted in response to a user operation. For example, when restricting viewing of the main image i obtained by decrypting the encrypted image Item#i to specific users (users who can obtain the private key corresponding to the encrypted image Item#i), the file control unit 43 deletes the plaintext common key Item#300+i stored in the mdat box in response to a user operation. This makes it impossible to decrypt the encrypted image Item#i unless the private key corresponding to the encrypted image Item#i can be obtained.

[0262] That is, if the plaintext common key Item#300+i is not stored in the mdat box, the encrypted image Item#i cannot be decrypted unless the private key corresponding to the encrypted image Item#i can be obtained. If the private key corresponding to the encrypted image Item#i can be obtained, as with the first encrypted HEIF file, the file control unit 43 can decrypt the encrypted encryption key Item#200+i associated with the encrypted image Item#i stored in the second encrypted HEIF file with the corresponding private key into a common key, and can decrypt the encrypted image Item#i into the original main image i with the common key obtained by the decryption.

[0263] On the other hand, if a plaintext common key Item#300+i is stored in the mdat box, the file control unit 43 can decrypt the encrypted image Item#i into the original main image i using the plaintext common key Item#300+i associated with the encrypted image Item#i stored in the second encrypted HEIF file.

[0264] Therefore, in the second encrypted HEIF file, if the plaintext common key Item#300+i is stored in the mdat box, the encrypted image Item#i can be decrypted to main image i, regardless of obtaining the private key corresponding to the encrypted image Item#i, and any user can view main image i.

[0265] From the above, according to the second encrypted HEIF file, for a main image for which viewing restriction is not required, the user can allow any user to view it by leaving the plaintext common key associated with that main image (the encrypted image obtained by encrypting that main image) in the mdat box. Also, for a main image for which viewing restriction is desired, the user can restrict viewing by any user by deleting the plaintext common key associated with that main image from the mdat box.

[0266] In the second encrypted HEIF file, in the infe box of the plaintext common key Item#300+i, the item type of the plaintext common key Item#300+i is set to ECKI, which represents an encryption key, just like in the infe box of the encrypted encryption key Item#200+i.

[0267] FIG. 34 is a diagram showing an example of the third encrypted HEIF file.

[0268] In FIG. 34, like FIG. 32, encrypted images Item#1 and Item#2, and encrypted encryption keys Item#201 and Item#202 are stored in the mdat box.

[0269] However, in the third encrypted HEIF file, the encrypted encryption key Item#200+i is a specific-information-containing encryption key that is generated by encrypting, with a private key, a uuid as specific information that identifies the external data to be associated with the primary image i and the common key used to encrypt the primary image i, and is hereinafter also referred to as the specific-information-containing encryption key Item#200+i.

[0270] In the third encrypted HEIF file in Fig. 34, encrypted image Item#1 obtained by encrypting main image 1 with a common key, UUID#1 assigned to (RAW data stored in) RAW file ARW#1 associated with main image 1, and specific information-containing encryption key Item#201 obtained by encrypting the common key used to encrypt main image 1 with a private key are associated with each other by association information stored in the cdsc box of the iref box, and stored in the mdat box. Furthermore, encrypted image Item#2 obtained by encrypting main image 2 with a common key, UUID#2 assigned to WAV file WAV#2 associated with main image 2, and specific information-containing encryption key Item#202 obtained by encrypting the common key used to encrypt main image 2 with a private key are associated with each other by association information stored in the cdsc box of the iref box, and stored in the mdat box.

[0271] For the third encrypted HEIF file, as with the first encrypted HEIF file, if the private key corresponding to the encrypted image Item#i stored in the third encrypted HEIF file can be obtained, the file control unit 43 can use the private key to decrypt the specific information-containing encryption key Item#200+i associated with the encrypted image Item#i into a common key and UUID#i, and then use the common key obtained by the decryption to decrypt the encrypted image Item#i into the original main image i. Therefore, viewing of the main image i can be restricted to users who can obtain the private key corresponding to the encrypted image Item#i.

[0272] Furthermore, for the third encrypted HEIF file, if the private key corresponding to the encrypted image Item#i can be obtained, the external data identified by the UUID#i obtained by decrypting the specific information-containing encryption key Item#200+i using the private key, that is, the external data associated with the main image i, can be accessed. Therefore, access to the external data associated with the main image i can be restricted to users who can obtain the private key corresponding to the encrypted image Item#i. Furthermore, tampering with the uuid as the specific information of the external data associated with the main image i can be prevented.

[0273] FIG. 35 is a diagram showing an example of a fourth encrypted HEIF file.

[0274] In Figure 35, the fourth encrypted HEIF file is structured similarly to the third encrypted HEIF file (Figure 34).

[0275] However, for the fourth encrypted HEIF file, the external data associated with the primary image i is encrypted with the common key used to encrypt the primary image i. In Fig. 35, the RAW file ARW#1 (the RAW data stored in it) is encrypted with the common key used to encrypt the primary image 1 associated with the RAW file ARW#1. Furthermore, the WAV file WAV#2 (the audio stored in it) is encrypted with the common key used to encrypt the primary image 2 associated with the WAV file WAV#2.

[0276] For the fourth encrypted HEIF file, as with the third encrypted HEIF file, if the private key corresponding to the encrypted image Item#i stored in the fourth encrypted HEIF file can be obtained, the file control unit 43 can use the private key to decrypt the specific information-containing encryption key Item#200+i associated with the encrypted image Item#i into a common key and UUID#i, and can then use the common key obtained by this decryption to decrypt the encrypted image Item#i into the original main image i. Therefore, viewing of the main image i can be restricted to users who can obtain the private key corresponding to the encrypted image Item#i.

[0277] Furthermore, for the fourth encrypted HEIF file, if a private key corresponding to the encrypted image Item#i can be obtained, the encrypted external data identified by the UUID#i obtained by decrypting the specific information-containing encryption key Item#200+i using the private key, that is, the encrypted external data associated with the main image i, can be accessed, and further, the encrypted external data can be decrypted to the original external data with a common key obtained by decrypting the specific information-containing encryption key Item#200+i. Therefore, viewing of the external data associated with the main image i can be restricted to users who can obtain the private key corresponding to the encrypted image Item#i. Furthermore, tampering with the uuid as the specific information of the external data associated with the main image i can be prevented.

[0278] FIG. 36 is a diagram showing an example of a fifth encrypted HEIF file.

[0279] In Fig. 36, encrypted images Item#1 and Item#2, and specific information-containing encryption keys Item#201 and Item#202 are stored in the mdat box, similarly to Fig. 34. Furthermore, in Fig. 36, a set of a plaintext common key and UUID#1 as specific information (hereinafter also referred to as a common key set) Item#301, and a common key set of a plaintext common key and UUID#2 as specific information Item#302 are stored in the mdat box.

[0280] The common key constituting the common key set Item#300+i is the common key used to encrypt the main image i. The UUID#i constituting the common key set Item#300+i is a uuid serving as specific information that identifies external data associated with the main image i (encrypted image Item#i obtained by encrypting the main image i). The specific information-containing encryption key Item#200+i is the common key set Item#300+i encrypted with a private key.

[0281] In the fifth encrypted HEIF file, the plaintext common key and common key set Item#300+i of UUID#i are stored in association with the encrypted image Item#i, just like the specific-information-containing encryption key Item#200+i that encrypted the common key set Item#300+i.

[0282] 36, in addition to the cdsc box shown in Fig. 34, a cdsc box storing association information that associates item ID#1 of encrypted image Item#1 with item ID#301 of common key set Item#301 as a reference source and a reference destination, respectively, is stored in the iref box. Furthermore, a cdsc box storing association information that associates item ID#2 of encrypted image Item#2 with item ID#302 of common key set Item#302 as a reference source and a reference destination, respectively, is stored in the iref box.

[0283] In the fifth encrypted HEIF file, the (plaintext) common key set Item#300+i stored in the mdat box can be deleted in response to a user operation. For example, when restricting viewing of the main image i obtained by decrypting the encrypted image Item#i to a specific user, the file control unit 43 deletes the common key set Item#300+i stored in the mdat box in response to a user operation. This makes it impossible to decrypt the encrypted image Item#i unless the private key corresponding to the encrypted image Item#i can be obtained. Furthermore, it becomes impossible to access external data associated with the encrypted image Item#i (the main image i corresponding to the encrypted image Item#i) unless the private key corresponding to the encrypted image Item#i can be obtained.

[0284] That is, if the common key set Item#300+i is not stored in the mdat box, the encrypted image Item#i cannot be decrypted and external data associated with the encrypted image Item#i cannot be accessed unless the private key corresponding to the encrypted image Item#i can be obtained. If the private key corresponding to the encrypted image Item#i can be obtained, similarly to the third encrypted HEIF file (FIG. 34), the file control unit 43 can decrypt the specific information-containing encryption key Item#200+i associated with the encrypted image Item#i stored in the fifth encrypted HEIF file with the corresponding private key into a common key and UUID#i, and can decrypt the encrypted image Item#i into the original main image i with the common key obtained by the decryption. Furthermore, the external data associated with the encrypted image Item#i (here, the RAW file ARW#1 or the WAV file WAV#2) can be accessed according to the UUID#i obtained by decrypting the specific information-containing encryption key Item#200+i.

[0285] On the other hand, when the shared key set Item#300+i is stored in the mdat box, the file control unit 43 can decrypt the encrypted image Item#i to the original main image i using the shared key constituting the shared key set Item#300+i associated with the encrypted image Item#i stored in the fifth encrypted HEIF file. Furthermore, the file control unit 43 can access external data associated with the encrypted image Item#i according to the UUID#i constituting the shared key set Item#300+i associated with the encrypted image Item#i stored in the fifth encrypted HEIF file.

[0286] Therefore, in the fifth encrypted HEIF file, if the (plaintext) common key set Item#300+i is stored in the mdat box, any user can view main image i and access external data associated with main image i, regardless of obtaining the private key corresponding to encrypted image Item#i.

[0287] From the above, according to the fifth encrypted HEIF file, a user can allow any user to view a main image for which viewing restriction is not required by leaving the common key set associated with the main image in the mdat box. Furthermore, any user can be allowed to access the external data associated with such a main image.

[0288] In addition, for a main image for which viewing is to be restricted, the user can restrict viewing by any user by deleting the common key set associated with the main image from the mdat box. Furthermore, the user can restrict access by any user to external data associated with such a main image.

[0289] In the fifth encrypted HEIF file, in the infe box of the shared key set Item#300+i (infe box of item ID#300+i), the item type of the shared key set Item#300+i is set to ECKI, which represents the encryption key, just like the infe box of the specific information-containing encryption key Item#200+i.

[0290] FIG. 37 is a diagram showing an example of the sixth encrypted HEIF file.

[0291] In Figure 37, the sixth encrypted HEIF file is structured similarly to the fifth encrypted HEIF file (Figure 36).

[0292] However, for the sixth encrypted HEIF file, similar to the fourth encrypted HEIF file (Fig. 35), the external data associated with primary image i is encrypted with the common key used to encrypt primary image i. In Fig. 37, RAW file ARW#1 is encrypted with the common key used to encrypt primary image 1 associated with RAW file ARW#1, and WAV file WAV#2 is encrypted with the common key used to encrypt primary image 2 associated with WAV file WAV#2.

[0293] For the sixth encrypted HEIF file, similarly to the fifth encrypted HEIF file, if the common key set Item#300+i is not deleted and remains stored in the mdat box, the file control unit 43 can decrypt the encrypted image Item#i to the original main image i with the common key constituting the common key set Item#300+i associated with the encrypted image Item#i stored in the sixth encrypted HEIF file. Furthermore, the file control unit 43 can access the encrypted external data associated with the encrypted image Item#i according to the UUID#i constituting the common key set Item#300+i associated with the encrypted image Item#i stored in the sixth encrypted HEIF file, and can decrypt the encrypted external data to the original external data with the common key constituting the common key set Item#300+i.

[0294] Therefore, in the sixth encrypted HEIF file, when the (plaintext) common key set Item#300+i is stored in the mdat box, any user can view main image i and access external data associated with main image i to view it, regardless of whether they have obtained the private key corresponding to encrypted image Item#i.

[0295] On the other hand, a user can restrict viewing of a main image i for which viewing is to be restricted by deleting the common key set Item#300+i associated with the main image i from the mdat box, thereby restricting the viewing of the main image i by any user. Furthermore, access to and viewing of external data associated with such main image i by any user can be restricted. Furthermore, tampering with the uuid as identification information of the external data associated with the main image i can be prevented.

[0296] Furthermore, for the sixth encrypted HEIF file, even if the common key set Item#300+i associated with the main image i is deleted from the mdat box, when the private key corresponding to the encrypted image Item#i that encrypted the main image i is available, the file control unit 43 can decrypt the specific information-containing encryption key Item#200+i associated with the encrypted image Item#i with the private key into a common key and UUID#i, and can decrypt the encrypted image Item#i into the original main image i with the common key obtained by the decryption. Therefore, viewing of the main image i can be restricted to users who can obtain the private key corresponding to the encrypted image Item#i.

[0297] Furthermore, for the sixth encrypted HEIF file, when a private key corresponding to the encrypted image Item#i can be obtained, the encrypted external data identified by the UUID#i obtained by decrypting the specific information-containing encryption key Item#200+i using the private key, that is, the encrypted external data associated with the main image i, can be accessed, and further, the encrypted external data can be decrypted to the original external data with a common key obtained by decrypting the specific information-containing encryption key Item#200+i. Therefore, viewing of the external data associated with the main image i can be restricted to users who can obtain the private key corresponding to the encrypted image Item#i. Furthermore, tampering with the uuid as the specific information of the external data associated with the main image i can be prevented.

[0298] FIG. 38 is a flowchart illustrating an example of pre-processing performed when the first encrypted HEIF file is handled by the image processing system 100.

[0299] The pre-processing is performed, for example, immediately after the digital camera 10 is manufactured, or before an image is captured by the digital camera 10.

[0300] In the pre-processing, in step S211, the digital camera 10 of the image processing system 100 sends a request for a public key to the external computer 101, and the process proceeds to step S212.

[0301] In step S212, the external computer 101 receives the request for the public key from the digital camera 10, and generates a private key corresponding to the public key in response to the request for the public key, and the process proceeds to step S213.

[0302] In step S213, the external computer 101 transmits the public key to the digital camera 10, and the process proceeds to step S214.

[0303] In step S214, the digital camera 10 stores the public key from the external computer 101 in the medium 14 or the like, and the pre-processing ends.

[0304] Note that the digital camera 10 can transmit to the external computer 101 a request for public keys in a number equal to the number of main images that can be stored in the medium 14, for example, depending on the remaining capacity of the medium 14 that stores the first encrypted HEIF file. In this case, the external computer 101 generates public keys in a number corresponding to the public key requests and stores them in the digital camera 10. The digital camera 10 encrypts the common key used to encrypt each main image with a different public key.

[0305] FIG. 39 is a flowchart illustrating an example of a generation process for generating a first encrypted HEIF file when the first encrypted HEIF file is handled by the image processing system 100.

[0306] The generation process is started, for example, when the user operates the digital camera 10 to capture an image (main image).

[0307] In the generation process, in step S221, the digital camera 10 captures an image in response to, for example, a user operation, and the process proceeds to step S222.

[0308] In step S222, the file control unit 43 of the digital camera 10 generates a common key, and the process proceeds to step S223.

[0309] In step S223, file control unit 43 generates an encrypted image by encrypting the main image obtained by capturing the image in step S221 with the common key generated in step S222, and the process proceeds to step S224.

[0310] In step S224, the file control unit 43 selects one of the public keys stored in the previous process that has not yet been used to encrypt the common key as a target public key. Furthermore, the file control unit 43 generates an encryption key by encrypting the common key used to encrypt the main image with the target public key, and the process proceeds from step S224 to step S225.

[0311] In step S225, the file control unit 43 generates a first encrypted HEIF file in which the encrypted image generated in step S223 and the encryption key generated in step S224 are stored in association with each other, and the generation process ends.

[0312] FIG. 40 is a flowchart illustrating an example of a playback process for playing back the first encrypted HEIF file when the first encrypted HEIF file is handled by the image processing system 100.

[0313] The playback process is started, for example, when the user operates the digital camera 10 to play back an image (main image).

[0314] In the playback process, in step S231, the file control unit 43 of the digital camera 10 selects a first encrypted HEIF file to be played back, for example, in response to a user operation, and selects an encrypted image to be played back from the first encrypted HEIF file. Furthermore, the file control unit 43 obtains (reads) an encryption encryption key associated with the encrypted image to be played back (an encryption encryption key corresponding to the encrypted image) from the first encrypted HEIF file to be played back, and the process proceeds from step S231 to step S232.

[0315] In step S232, the file control unit 43 attempts to acquire a private key corresponding to the encrypted image to be played back, and the process proceeds to step S233.

[0316] For example, the file control unit 43 requests the external computer 101 for a private key corresponding to the encrypted image to be reproduced. The external computer 101 stores in advance, for example, a user ID and a password associated with the private key corresponding to the encrypted image. The external computer 101 prompts the user of the digital camera 10 to input the user ID and password, and performs authentication after the user ID and password are input. If the authentication is successful, for example, if the user ID and password stored in association with the private key corresponding to the encrypted image to be reproduced match the user ID and password input by the user, the external computer 101 transmits the private key corresponding to the encrypted image to be reproduced to the digital camera 10.

[0317] On the other hand, if the authentication fails, the external computer 101 transmits a message to that effect to the digital camera 10 .

[0318] In the digital camera 10, when a private key corresponding to an encrypted image to be reproduced is transmitted from the external computer 101, the file control unit 43 acquires the private key.

[0319] In step S233, file control unit 43 determines whether or not the private key has been obtained in the above manner.

[0320] If it is determined in step S233 that the private key could not be acquired, the process proceeds to step S234. In step S234, digital camera 10 performs error processing, for example, displays an error message on liquid crystal panel 19 indicating that image decryption has failed, and the playback process ends. Therefore, a user who cannot obtain the private key corresponding to the encrypted image to be played back cannot view the main image obtained by decrypting the encrypted image to be played back.

[0321] On the other hand, if it is determined in step S233 that the private key has been acquired, the process proceeds to step S235.

[0322] In step S235, the file control unit 43 decrypts the encrypted encryption key obtained in step S231 to a common key using the private key obtained in step S232, and the process proceeds to step S236.

[0323] In step S236, the file control unit 43 decrypts the encrypted image to be reproduced to a main image using the common key decrypted in step S236, and displays the main image on the liquid crystal panel 19, and the reproduction process ends.

[0324] FIG. 41 is a flowchart illustrating an example of pre-processing performed when the second encrypted HEIF file is handled by the image processing system 100.

[0325] In the pre-processing of FIG. 41, steps S251 to S254 are similar to steps S211 to S214 of the pre-processing of FIG. 38, respectively.

[0326] FIG. 42 is a flowchart illustrating an example of a generation process for generating a second encrypted HEIF file when the second encrypted HEIF file is handled by the image processing system 100.

[0327] In the generation process of FIG. 42, steps S261 to S264 are performed similarly to steps S221 to S224 of the generation process of FIG.

[0328] Then, in step S265, the file control unit 43 generates a second encrypted HEIF file in which the encrypted image generated in step S263 is associated with the common key generated in step S262 and the encrypted encryption key generated in step S264, and the generation process ends.

[0329] Figure 43 is a flowchart illustrating an example of a restriction process that restricts viewing of the main image obtained by decrypting the encrypted image by anyone other than users who can obtain the private key corresponding to the encrypted image when a second encrypted HEIF file is handled by the image processing system 100.

[0330] For example, when a user operates the digital camera 10 to perform restriction processing, in step S271, the file control unit 43 selects the second encrypted HEIF file to be played, for example, in response to the user's operation, and obtains (reads) from the second encrypted HEIF file the plaintext common key and the encrypted image to which the plaintext common key is associated, and processing proceeds to step S272.

[0331] In step S272, the file control unit 43 decrypts the encrypted image obtained in step S271 to a main image using the common key also obtained in step S271, and displays it on the liquid crystal panel 19, and the process proceeds to step S273.

[0332] In step S273, the file control unit 43 determines whether or not the user has performed a restricting operation on the digital camera 10 so as to restrict the viewing of the main image displayed in step S272.

[0333] If it is determined in step S273 that the restricted operation has not been performed, the restriction process ends.

[0334] Also, if it is determined in step S273 that a restriction operation has been performed, the process proceeds to step S274, in which the file control unit 43 deletes the plaintext common key acquired in step S271 from the second encrypted HEIF file, and the restriction process ends.

[0335] As described above, by deleting the plaintext common key associated with the encrypted image from the second encrypted HEIF file, it is possible to restrict viewing of the main image obtained by decrypting the encrypted image by anyone other than users who have access to the private key corresponding to the encrypted image.

[0336] FIG. 44 is a flowchart illustrating an example of a playback process for playing back the second encrypted HEIF file when the second encrypted HEIF file is handled by the image processing system 100.

[0337] 44, in step S281, the file control unit 43 selects the second encrypted HEIF file to be played back in response to, for example, a user operation, and selects an encrypted image to be played back from the second encrypted HEIF file.The file control unit 43 then determines whether a plaintext common key associated with the encrypted image to be played back is stored in the second encrypted HEIF file to be played back.

[0338] If it is determined in step S281 that a plaintext common key is stored, the process proceeds to step S287.

[0339] In step S287, the file control unit 43 decrypts the encrypted image to be played back using the plaintext common key associated with the encrypted image to generate the main image, and displays it on the liquid crystal panel 19, after which the playback process ends.

[0340] On the other hand, if it is determined in step S281 that the plaintext common key is not stored, that is, if the plaintext common key has been deleted in the restriction process of FIG. 43, the process proceeds to step S282.

[0341] In steps S282 through S287, the same processes as those in steps S231 through S236 of the playback process in FIG. 40 are carried out.

[0342] FIG. 45 is a flowchart illustrating an example of pre-processing performed when the image processing system 100 handles the third encrypted HEIF file.

[0343] In the pre-processing of FIG. 45, steps S311 to S314 are performed similarly to steps S211 to S214 of the pre-processing of FIG.

[0344] FIG. 46 is a flowchart illustrating an example of a generation process for generating a third encrypted HEIF file when the third encrypted HEIF file is handled by the image processing system 100.

[0345] In the generation process of FIG. 46, steps S321 to S323 are similar to steps S221 to S223 of the generation process of FIG.

[0346] Then, in step S324, the file control unit 43 selects one of the public keys stored in the previous process that has not yet been used to encrypt the common key as a target public key. The file control unit 43 also assigns a uuid as specific information to the external data to be associated with the main image (e.g., RAW data of the main image, etc.). The file control unit 43 then encrypts the common key used to encrypt the main image and the uuid of the external data to be associated with the main image with the target public key to generate a specific information-containing encryption key, and the process proceeds from step S324 to step S325.

[0347] In step S325, the file control unit 43 generates a third encrypted HEIF file that associates and stores the encrypted image generated in step S323 and the specific information-containing encryption key generated in step S324, and the generation process ends.

[0348] FIG. 47 is a flowchart illustrating an example of a playback process for playing back a third encrypted HEIF file when the third encrypted HEIF file is handled by the image processing system 100.

[0349] 47, in step S331, the file control unit 43 of the digital camera 10 selects the third encrypted HEIF file to be played back, for example, in response to a user operation, and selects the encrypted image to be played back from the third encrypted HEIF file. Furthermore, the file control unit 43 obtains the specific information-containing encryption key associated with the encrypted image to be played back from the third encrypted HEIF file to be played back, and the process proceeds from step S331 to step S332.

[0350] In step S332, the file control unit 43 attempts to acquire a private key corresponding to the encrypted image to be reproduced, for example, in the same manner as in step S233 of FIG. 40, and the process proceeds to step S333.

[0351] In step S333, file control unit 43 determines whether or not the private key has been acquired.

[0352] If it is determined in step S333 that the private key could not be acquired, the process proceeds to step S334. In step S334, the digital camera 10 performs error processing similar to step S234 in Fig. 40, and the playback process ends.

[0353] On the other hand, if it is determined in step S333 that the private key has been acquired, the process proceeds to step S335.

[0354] In step S335, the file control unit 43 decrypts the specific information-containing encryption key acquired in step S331 with the private key acquired in step S332 into the uuid as specific information and the common key, and the process proceeds to step S336.

[0355] In step S336, the file control unit 43 decrypts the encrypted image to be reproduced into a main image using the common key decrypted in step S335, and displays it on the liquid crystal panel 19, and the process proceeds to step S337.

[0356] In step S337, the file control unit 43 acquires the external data identified by the uuid decrypted in step S336, and the playback process ends. The external data is played back in response to, for example, a user operation.

[0357] FIG. 48 is a flowchart illustrating an example of pre-processing performed when the fourth encrypted HEIF file is handled by the image processing system 100.

[0358] In the pre-processing of FIG. 48, steps S351 to S354 are performed in the same manner as steps S211 to S214 of the pre-processing of FIG.

[0359] FIG. 49 is a flowchart illustrating an example of a generation process for generating a fourth encrypted HEIF file when the fourth encrypted HEIF file is handled by the image processing system 100.

[0360] In the generation process of FIG. 49, steps S361 to S363 are performed similarly to steps S321 to S323, respectively, of the generation process of FIG.

[0361] Then, in step S364, the file control unit 43 encrypts the external data associated with the main image (for example, the RAW data of the main image) with the common key used to encrypt the main image, and the process proceeds to step S365.

[0362] In step S365, the file control unit 43 selects one of the public keys stored in the previous process that has not yet been used to encrypt the common key as a target public key. The file control unit 43 also assigns a uuid as specific information to the external data to be associated with the main image (the external data encrypted in step S364). The file control unit 43 then encrypts the common key used to encrypt the main image and the uuid of the external data to be associated with the main image with the target public key to generate a specific information-containing encryption key, and the process proceeds from step S365 to step S366.

[0363] In step S366, the file control unit 43 generates a fourth encrypted HEIF file that associates and stores the encrypted image generated in step S363 and the specific information-containing encryption key generated in step S365, and the processing proceeds to step S367.

[0364] In step S367, file control unit 43 generates a file storing the external data encrypted in step S364, and stores the file in medium 14, for example, and the generation process ends.

[0365] FIG. 50 is a flowchart illustrating an example of a playback process for playing back the fourth encrypted HEIF file when the fourth encrypted HEIF file is handled by the image processing system 100.

[0366] In the playback process of FIG. 50, steps S371 to S377 are similar to steps S331 to S337 of FIG. 47, respectively.

[0367] Then, in step S378, file control unit 43 decrypts the external data acquired in step S377 with the common key decrypted in step S375, and the playback process ends.

[0368] That is, for the fourth encrypted HEIF file, the external data acquired in step S377 is encrypted with a common key, and therefore, in step S378, the encrypted external data is decrypted with the common key.

[0369] FIG. 51 is a flowchart illustrating an example of pre-processing performed when the fifth or sixth encrypted HEIF file is handled by the image processing system 100.

[0370] In the pre-processing of FIG. 41, steps S451 to S454 are similar to steps S211 to S214 of the pre-processing of FIG. 38, respectively.

[0371] FIG. 52 is a flowchart illustrating an example of a generation process for generating a fifth or sixth encrypted HEIF file when the fifth or sixth encrypted HEIF file is handled by the image processing system 100.

[0372] In the generation process of FIG. 52, steps S461 and S462 are performed similarly to steps S221 and S222, respectively, of the generation process of FIG.

[0373] Then, in step S463, file control unit 43 generates an encrypted image by encrypting the main image obtained by capturing the image in step S461 with the common key generated in step S462, and the process proceeds to step S464.

[0374] Note that for the sixth HEIF file, in step S463, the file control unit 43 further encrypts the external data associated with the main image with the common key used to encrypt the main image.

[0375] In step S464, the file control unit 43 selects one of the public keys stored in the previous process that has not yet been used to encrypt the common key as the target public key. The file control unit 43 also assigns a uuid as specific information to the external data to be associated with the main image. The file control unit 43 then encrypts a common key set consisting of the common key used to encrypt the main image and the uuid of the external data to be associated with the main image with the target public key to generate a specific information-containing encryption key, and the process proceeds from step S464 to step S465.

[0376] In step S465, the file control unit 43 generates a fifth or sixth encrypted HEIF file that stores the encrypted image generated in step S463 in association with the specific information-containing encryption key generated in step S464 and the (plaintext) common key set before being encrypted by the specific information-containing encryption key, and the generation process ends.

[0377] Note that, for the sixth HEIF file, in step S465, the file control unit 43 further generates a file storing the external data encrypted in step S463, and stores the file in the medium 14, for example.

[0378] Figure 53 is a flowchart illustrating an example of a restriction process that restricts viewing of the main image obtained by decrypting an encrypted image by anyone other than a user who can obtain the private key corresponding to the encrypted image when the fifth or sixth encrypted HEIF file is handled by the image processing system 100.

[0379] In the restriction processing of Figure 53, in step S471, the file control unit 43 selects the fifth or sixth encrypted HEIF file to be played, for example, in response to a user operation, and obtains from the fifth or sixth encrypted HEIF file a (plaintext) common key set and an encrypted image with which the common key set is associated, and the processing proceeds to step S472.

[0380] In step S472, the file control unit 43 decrypts the encrypted image obtained in step S471 into a main image using the common key constituting the common key set also obtained in step S471, displays it on the liquid crystal panel 19, and the processing proceeds to step S473.

[0381] In step S473, the file control unit 43 determines whether or not the user has performed a restricting operation on the digital camera 10 so as to restrict viewing of the main image displayed in step S472.

[0382] If it is determined in step S473 that the restricted operation has not been performed, the restriction process ends.

[0383] Also, if it is determined in step S473 that a restriction operation has been performed, the process proceeds to step S474, in which the file control unit 43 deletes the common key set acquired in step S471 from the fifth or sixth encrypted HEIF file, and the restriction process ends.

[0384] As described above, by deleting the plaintext common key associated with the encrypted image from the fifth or sixth encrypted HEIF file, it is possible to restrict viewing of the main image obtained by decrypting the encrypted image and access to external data associated with the main image by users other than those who have access to the private key corresponding to the encrypted image.

[0385] FIG. 54 is a flowchart illustrating an example of a playback process for playing back the fifth or sixth encrypted HEIF file when the fifth or sixth encrypted HEIF file is handled by the image processing system 100.

[0386] 54, in step S481, the file control unit 43 selects the fifth or sixth encrypted HEIF file to be played back in response to, for example, a user operation, and selects an encrypted image to be played back from the fifth or sixth encrypted HEIF file.The file control unit 43 then determines whether a plaintext common key set associated with the encrypted image to be played back is stored in the fifth or sixth encrypted HEIF file to be played back.

[0387] If it is determined in step S481 that a common key set is stored, the process proceeds to step S487.

[0388] In step S487, the file control unit 43 decrypts the encrypted image to be played back into a main image using a common key that constitutes the common key set associated with that encrypted image, and displays it on the liquid crystal panel 19, and the process proceeds to step S488.

[0389] In step S488, the file control unit 43 acquires the external data specified by the uuid constituting the common key set associated with the encrypted image to be reproduced, and the reproduction process ends.

[0390] In addition, since the external data for the sixth HEIF file is encrypted, in step S488, the file control unit 43 further acquires the encrypted external data identified by the uuid that constitutes the common key set, and then decrypts the encrypted external data with the common key that constitutes the common key set.

[0391] On the other hand, if it is determined in step S481 that the common key set is not stored, that is, if the common key set has been deleted in the restriction process of FIG. 53, the process proceeds to step S482.

[0392] In steps S482 through S488, the same processes as in steps S331 through S337 of the playback process in FIG. 47 are carried out, and then the playback process ends.

[0393] In addition, since the external data for the sixth HEIF file is encrypted, in step S488, the file control unit 43 further obtains the encrypted external data identified by the uuid obtained by decrypting the specific information-containing encryption key in step S486, and then decrypts the encrypted external data with the common key obtained by decrypting the specific information-containing encryption key in step S486.

[0394] <Second embodiment of image processing system to which the present technology is applied>

[0395] FIG. 55 is a block diagram showing an example configuration of a second embodiment of an image processing system to which the present technology is applied.

[0396] In the figure, parts corresponding to those in the image processing system 100 in FIG. 31 are given the same reference numerals, and descriptions thereof will be omitted below as appropriate.

[0397] In FIG. 55, an image processing system 200 is made up of a digital camera 210 and an external computer 101 .

[0398] Therefore, image processing system 200 is common to image processing system 100 in FIG. 31 in that it has external computer 101, but differs from image processing system 100 in that it has digital camera 210 instead of digital camera 10.

[0399] FIG. 56 is a block diagram showing an example of the configuration of the digital camera 210. As shown in FIG.

[0400] In the figure, parts corresponding to those of the digital camera 10 in FIG. 1 are given the same reference numerals, and the description thereof will be omitted below as appropriate.

[0401] The digital camera 210 includes an optical system 11 through an interface 21 , and the signal processing unit 13 includes an optical system / image sensor control unit 41 through a UI control unit 47 , and a recognition unit 211 .

[0402] Therefore, digital camera 210 has in common with digital camera 10 in FIG. 1 in that it has optical system 11 through interface 21, and signal processing section 13 has optical system / image sensor control section 41 through UI control section 47.

[0403] However, digital camera 210 differs from digital camera 10 in that signal processing unit 13 is newly provided with recognition unit 211 .

[0404] The recognition unit 211 receives (data of) the main image from the optical system / image sensor control unit 41.

[0405] The recognition unit 211 performs image recognition on the main image from the optical system / image sensor control unit 41. The recognition unit 211 recognizes a subject appearing in the main image as a recognition target through image recognition, and supplies the recognition result to the file control unit 43.

[0406] In the recognition unit 211, the recognition algorithm for recognizing the subject can be an algorithm that identifies (classifies) the type of subject as the recognition target, for example, a person, a car, or a trademark, for any subject appearing in the main image, or a recognition algorithm that identifies an individual person as the recognition target for a specific type of subject, that is, for example, a recognition algorithm that identifies an individual person.

[0407] Further, the digital camera 210 is provided with another sensor 212, and the recognition unit 211 can recognize the subject appearing in the main image by using the sensing results of the other sensor 212 in addition to the main image. As the other sensor 212, for example, a GPS (Global Positioning System) sensor, a direction sensor, a depth (distance measurement) sensor, etc. can be adopted. The recognition unit 211 can improve the recognition accuracy of the subject appearing in the main image by identifying the current location from the sensing results of the GPS sensor and the direction sensor, and limiting the recognition target to subjects that may exist at the current location. Furthermore, the recognition unit 211 can improve the recognition accuracy in low illumination by generating a distance image in which the pixel value is the distance to the subject from the sensing results of the depth sensor, and recognizing the subject appearing in the main image by using the distance image in addition to the main image.

[0408] Here, grid is an item type of HEIF. For a main image whose item type is grid, each of the multiple split images (tiles) obtained by dividing the main image into split images of the same size is stored as an item in the HEIF file. A main image whose item type is grid is reconstructed from the multiple split images as items. An item whose item type is grid (here, the main image), i.e., a main image divided into multiple split images (and a main image reconstructed from the multiple split images), is also called a grid item.

[0409] In the digital camera 210, the file control unit 43 can encrypt not the entire main image, but each divided image obtained by dividing the main image. For example, the file control unit 43 can encrypt each divided image showing a specific subject in the main image, according to the recognition result from the recognition unit 211.

[0410] Below, we will explain how to process encrypted HEIF files when encryption is performed on a split-image basis, but before that, we will explain HEIF files (hereinafter also referred to as grid files) that store main images with an item type of grid.

[0411] <gridファイル>

[0412] FIG. 57 is a diagram showing an example of a grid file.

[0413] In the grid file of Figure 57, the main image is divided into 3 x 3 divided images Item#1 to Item#9, and the divided images Item#1 to Item#9 are stored as items in the mdat box.

[0414] In Fig. 57, the items stored in the mdat box are nine items, the divided images Item#1 to Item#9, but the number of items in the iinf box and the iloc box is 10. This is because, in addition to the nine items, the divided images Item#1 to Item#9, the main image (reconstructed image) as a grid item reconstructed from the divided images Item#1 to Item#9 is counted as an item. In Fig. 57, the item ID of the main image as a grid item is 10, and for convenience of explanation, this main image will be described as main image Item#10.

[0415] For the main image #10 as a grid item, media data is not stored in the mdat box, and instead, an idat box is stored in the meta box. The idat box stores the number of grids horizontal, the number of grids vertical, the output horizontal size, and the output vertical size. The number of grids horizontal and the number of grids vertical represent the number of divided images #1 to #9 in the horizontal and vertical directions, respectively, that make up the main image #10 as a grid item. The output horizontal size and the output vertical size represent the sizes in the horizontal and vertical directions, respectively, of the main image #10 as a grid item reconstructed from the divided images #1 to #9.

[0416] Furthermore, for main image #10 as a grid item, information (offset and size) regarding the storage location of main image #10 is stored in the iloc box, and this information represents the storage location of the idat box of main image #10 as a grid item.

[0417] The grid file in Fig. 57 has 10 infe boxes since 10 items are stored: divided images Item#1 to Item#9 and a main image Item#10 reconstructed from the divided images Item#1 to Item#9. As described in Fig. 9, an item ID that identifies an item and an item type are stored (registered) in the infe box, and a grid that represents a grid item is stored as the item type of the main image Item#10 in the infe box of the main image Item#10 as a grid item.

[0418] In a grid file, a dimg box is stored in an iref box. The dimg box stores information that associates a grid item with a divided image that constitutes the grid item. For example, the dimg box stores the item ID of the divided image as a reference destination, and stores the item ID of the main image as a grid item reconstructed from the divided image as a reference source. In FIG. 57, item IDs #1 to #9 of divided images Item#1 to Item#9 are stored as reference destinations, and item ID #10 of main image Item#10 is stored as a reference source. In addition, a reference counter indicating the number of divided images is stored in the dimg box.

[0419] For the above grid file, the file control unit 43 can recognize that the main image Item#10 is a reconstructed image (grid item) reconstructed from a plurality of divided images from the item type grid stored in the infe box of the main image Item#10. Furthermore, the file control unit 43 can specify the item ID#10 of the main image Item#10 reconstructed from the divided images and the item IDs#1 to #9 of the divided images Item#1 to Item#9 used to reconstruct the main image Item#10 from the reference source and reference destination stored in the dimg box. Furthermore, the file control unit 43 can specify the number of divided images #1 to #9 in the horizontal and vertical directions constituting the main image #10 and the horizontal and vertical sizes of the main image #10 reconstructed from the divided images #1 to #9 from the grid number horizontal and grid number vertical stored in the idat box and the output horizontal and output vertical sizes.

[0420] The file control unit 43 can reconstruct a main image Item#10 of item ID#10 identified from the dimg box, of a size identified from the idat box, composed of split images Item#1 to Item#9 of item IDs #1 to #9 identified from the dimg box, with the number of split images in the horizontal and vertical directions identified from the idat box being the number identified from the idat box.

[0421] In the digital camera 210 (FIG. 56), the file control unit 43 can generate an encrypted grid file that stores encrypted split images obtained by encrypting split images obtained by dividing a main image with a common key (first encryption key) in association with an encrypted encryption key obtained by encrypting the common key with a public key (second encryption key). The encrypted grid file is a type of encrypted HEIF file.

[0422] The file control unit 43 can use the association between the main image stored in the associated HEIF file and the specific information of the external data to associate the encrypted split images stored in the encrypted grid file with the encryption encryption key that encrypted the common key used for encryption to obtain the encrypted split images. In other words, the file control unit 43 can associate the encrypted split images stored in the encrypted grid file with the encryption encryption key in the same way as the association between the main image stored in the associated HEIF file and the specific information of the external data.

[0423] The file control unit 43 can also decrypt the encryption key of the encrypted grid file into a common key, and can decrypt the encrypted split images into the original split images by using the common key obtained by the decryption. Furthermore, the file control unit 43 can reconstruct a reconstructed image as a main image by arranging the split images.

[0424] <Processing encrypted grid files>

[0425] FIG. 58 is a diagram for explaining an overview of the processing of the encrypted grid file performed by the digital camera 210. As shown in FIG.

[0426] When generating an encrypted grid file, the recognition unit 211 in the digital camera 210 (FIG. 56) recognizes, for example, a human face as the recognition target, a human face appearing in the main image, and supplies the recognition result to the file control unit 43.

[0427] Depending on the recognition result from the recognition unit 211, for example, among the split images obtained by dividing the main image, the file control unit 43 encrypts split images in which the recognition target has been recognized (split images in which a human face is shown as the recognition target) using a common key to create encrypted split images and stores them in an encrypted grid file. Split images other than the split image in which the recognition target has been recognized can be encrypted using a common key to create encrypted split images and stored in an encrypted grid file in the same way as the split image in which the recognition target has been recognized, or can be stored in the encrypted grid file without encryption (as plain text). Here, split images other than the split image in which the recognition target has been recognized are stored in the encrypted grid file without encryption.

[0428] As in the case of encrypted HEIF files, the file control unit 43 encrypts the common key used to encrypt the split images into an encryption key with a public key, associates the encryption key with the corresponding encrypted split images (split images encrypted with the public key that decrypted the encryption key) and stores it in the encrypted grid file.

[0429] When reproducing the encrypted grid file, in the digital camera 210, the file control unit 43 reconstructs a reconstructed image as a main image from the divided images stored in the encrypted grid file and the encrypted divided images.

[0430] When a private key corresponding to the public key can be obtained, in reconstructing the reconstructed image as the main image, the file control unit 43 uses the private key to decrypt the encrypted encryption key associated with the encrypted split image into a common key, and uses the common key to decrypt the encrypted split image into the original split image. Furthermore, the file control unit 43 arranges the (plain text) split image stored in the encrypted grid file and the split image obtained by decrypting the encrypted split image, to reconstruct the reconstructed image as the main image. Therefore, in this case, a reconstructed image can be obtained that shows the subject to be recognized that appears in the original main image, in this case, a human face.

[0431] On the other hand, if the private key corresponding to the public key cannot be obtained, in reconstructing the reconstructed image as the main image, the file control unit 43 arranges the (plain text) split images stored in the encrypted grid file and the encrypted split images (or, for example, a patternless, solid color image representing the encrypted split images) to reconstruct the reconstructed image as the main image. Therefore, in this case, a (concealed) reconstructed image is obtained in which the subject to be recognized that appears in the original main image, in this case, a human face, is not shown.

[0432] As described above, by using an encrypted grid file, viewing of a subject as a recognition target can be restricted to users who can obtain a private key corresponding to a public key.

[0433] In addition, in digital camera 210, the encrypted split images are encrypted with a different common key for each subject appearing in the corresponding split image, and the common key used to encrypt the split images can be encrypted with a different public key for each subject.

[0434] That is, in digital camera 210, the divided images can be encrypted with a different common key for each subject to be recognized that appears in the divided images. Furthermore, the common key that differs for each subject to be recognized that appears in the divided images can be encrypted with a different public key (hence, a different private key) for each subject to be recognized.

[0435] For example, when a human face is used as the recognition target and the face of person A and the face of person B in the main image are recognized as shown in Fig. 58, the divided image containing the face of person A and the divided image containing the face of person B can be encrypted with different common keys. Furthermore, the common key used to encrypt the divided image containing the face of person A and the common key used to encrypt the divided image containing the face of person B can be encrypted with different public keys KA and KB, respectively.

[0436] In this case, if the private key corresponding to the public key KA can be obtained, a reconstructed image showing the face of person A is reconstructed, and if the private key corresponding to the public key KB can be obtained, a reconstructed image showing the face of person B is reconstructed. Therefore, viewing of person A's face can be restricted to users who can obtain the private key corresponding to the public key KA, and viewing of person B's face can be restricted to users who can obtain the private key corresponding to the public key KB.

[0437] As described above, it is possible to restrict the users who can view each subject.

[0438] In addition, although a specific type of subject, a person, is used as the recognition target and an individual is identified here, multiple types of subjects can be used as the recognition target and the type of subject as the recognition target can be identified. For example, a person and a trademark can be identified as the recognition target. In this case, encryption can be performed using different common keys and public keys for each type of subject as the recognition target, i.e., for each person and trademark.

[0439] Furthermore, in the recognition unit 211, the subject to be recognized can be set according to a user's operation. For example, as described above, whether to identify an individual by using a specific type of subject, namely a person, as the recognition subject, or to identify a person and a trademark as the recognition subject, can be set according to a user's operation.

[0440] <Encrypted grid file>

[0441] FIG. 59 is a diagram showing an example of an encrypted grid file.

[0442] The encrypted grid file in Fig. 59 is an encrypted grid file corresponding to the grid file in Fig. 57. That is, the encrypted grid file in Fig. 59 is an encrypted grid file in which encrypted divided images Item#1 to Item#9 are stored as encrypted images obtained by encrypting the divided images Item#1 to Item#9 stored in the grid file in Fig. 57 in units of divided images.

[0443] In FIG. 59, the contents of the iloc box and the idat box are omitted, but are the same as those in FIG. 57.

[0444] The encrypted grid file in Fig. 59 differs from the grid file in Fig. 57 in that encrypted divided images Item#1 to Item#9 are stored in the mdat box as items instead of (plaintext) divided images Item#1 to Item#9. The encrypted grid file in Fig. 59 also differs from the grid file in Fig. 57 in that encrypted encryption keys Item#11 to Item#19, which are obtained by encrypting the common key used in the encryption to obtain the encrypted divided images Item#1 to Item#9 with a public key, are newly stored in the mdat box as items. The encrypted grid file in Fig. 59 also differs from the grid file in Fig. 57 in that a cdsc box in which association information is stored is newly stored in the iref box.

[0445] In addition, the encrypted grid file in Fig. 59 has an increased number of items by the encrypted encryption keys Item#11 to Item#19 compared to the grid file in Fig. 57. Therefore, in the encrypted grid file in Fig. 59, the number of items in the iinf box is 19, and in this respect the encrypted grid file in Fig. 59 differs from the grid file in Fig. 57 in which the number of items is 10. Furthermore, the encrypted grid file in Fig. 59 differs from the grid file in Fig. 57 in that an infe box is newly provided for the encrypted encryption keys Item#11 to Item#19, which have increased compared to the grid file in Fig. 57.

[0446] In Fig. 59, for example, divided images 1 to 9 before being encrypted into encrypted divided images Item#1 to Item#9 each show a different subject as a recognition target, and therefore divided images 1 to 9 are encrypted into encrypted divided images Item#1 to Item#9 with different common keys. Furthermore, the common keys used to encrypt divided images 1 to 9 showing subjects as different recognition targets are encrypted into encryption keys Item#11 to Item#19 with different public keys.

[0447] In the encrypted grid file, for example, similarly to the (first to sixth) encrypted HEIF files, the encrypted split image Item#i is associated with the encrypted encryption key Item#10+i by storing association information in the cdsc box that associates the encrypted split image Item#i with the encrypted encryption key Item#10+i, which is obtained by encrypting the common key used to obtain the encrypted split image Item#i with a public key.

[0448] For an encrypted split image Item#i stored in an encrypted grid file, if a private key (a private key corresponding to a public key used in encrypting the common key used in encrypting the encrypted split image Item#i) corresponding to the encrypted split image Item#i can be obtained, the file control unit 43 can decrypt the encrypted encryption key Item#10+i associated with the encrypted split image Item#i with the private key into a common key, and can decrypt the encrypted split image Item#i into the original split image i with the common key obtained by the decryption. Therefore, for each split image i obtained by dividing the main image, viewing of the split image i (the subject depicted in the split image) can be restricted to users who can obtain the private key corresponding to the encrypted split image Item#i.

[0449] When decrypting the encrypted split image Item#i, the file control unit 43 identifies the item ID of the referenced item of the cdsc box whose reference source is the item ID #i of the encrypted split image Item#i as the encrypted encryption key Item#10+i corresponding to the encrypted split image Item#i, i.e., the item ID#10+i of the encrypted encryption key Item#10+i which encrypted the common key used to encrypt the encrypted image Item#i, and obtains and decrypts the encrypted encryption key Item#10+i from the mdat box according to the item ID#10+i.

[0450] In the encrypted grid file in Fig. 59, all of the nine divided images 1 to 9 constituting the main image are encrypted into encrypted divided images Item#1 to Item#9, but it is not necessary for all divided images 1 to 9 constituting the main image to be encrypted. In other words, for divided images 1 to 9 constituting the main image, only divided images that include a subject to be recognized can be encrypted.

[0451] FIG. 60 is a flowchart illustrating a first example of pre-processing performed when an encrypted grid file is handled by the image processing system 200.

[0452] In a first example of pre-processing, in step S511, the digital camera 210 of the image processing system 200 sends a request for a predetermined number of public keys to the external computer 101, and the process proceeds to step S512.

[0453] For example, in the case where the recognition unit 211 is set to identify individuals by using people as the recognition target, face recognition is performed using an image captured for displaying a so-called through image of the digital camera 210, thereby identifying the number of people (individuals) as the recognition target, and a public key equal to the number of people as the recognition target is requested.

[0454] Also, for example, in the digital camera 210, the user is asked to input information about people whose faces he wishes to conceal, and according to that information, public keys are requested for the number of people whose faces he wishes to conceal.

[0455] In steps S512 to S514, the same processes as in steps S212 to S214 in Fig. 38 are carried out. As a result, digital camera 210 stores a necessary number of public keys, for example, the number of public keys corresponding to the number of people whose faces wish to be kept secret.

[0456] FIG. 61 is a flowchart illustrating a first example of a generation process for generating an encrypted grid file when the encrypted grid file is handled by the image processing system 200. As shown in FIG.

[0457] In the first example of the generation process, in step S521, the digital camera 210 captures an image in response to, for example, a user operation, and the process proceeds to step S522.

[0458] In step S522, the recognition unit 211 performs image recognition using the image captured in step S521, and the process proceeds to step S523.

[0459] In step S523, the file control unit 43 controls the encoding control unit 42 to divide the main image obtained by capturing the image in step S521 according to the recognition target recognized by the image recognition of the recognition unit 211. The encoding control unit 42 divides the main image into a plurality of divided images according to the control of the file control unit 43, for example, so that different recognition targets appear in different divided images (so that one or less recognition targets appear in one divided image), and supplies the divided images to the file control unit 43, and the process proceeds from step S523 to step S524.

[0460] In step S524, file control unit 43 generates a different common key for each recognition target recognized by image recognition by recognition unit 211, and the process proceeds to step S525.

[0461] In step S525, the file control unit 43 generates encrypted split images by encrypting the split images, among those divided from the main image, that contain a recognition target recognized by the image recognition of the recognition unit 211, with a common key for that recognition target, and the process proceeds to step S526.

[0462] In step S526, the file control unit 43 generates an encryption key for each recognition target by encrypting the common key for each recognition target recognized by the image recognition of the recognition unit 211 with the public key stored in the first example of preprocessing, and the process proceeds to step S527. Note that the common keys for different recognition targets are encrypted with different public keys.

[0463] In step S527, the file control unit 43 stores the split images obtained by dividing the main image that do not include the recognition target, and generates an encrypted grid file that stores the encrypted split images generated in step S525 in association with the encryption encryption key corresponding to the encrypted split images generated in step S526 (the encryption encryption key obtained by encrypting the common key used to encrypt the encrypted split images), and then the generation process ends.

[0464] In HEIF, the main image is divided into multiple split images of the same size. Therefore, when the main image is divided into split images each containing one or less recognition targets, one recognition target may appear across multiple split images. When one recognition target appears across multiple split images, the multiple split images can be encrypted with the same common key.

[0465] Also, here, a common key is generated for each recognition target recognized by image recognition, and the split images in which the recognition target appears are encrypted with the common key for that recognition target. However, the common key can also be generated in other ways, for example, for each split image in which the recognition target recognized by image recognition appears, and the split image in which the recognition target appears can be encrypted with the common key for that split image.

[0466] FIG. 62 is a flowchart illustrating an example of a playback process for playing back an encrypted grid file when the encrypted grid file is handled by the image processing system 200. As shown in FIG.

[0467] In the playback process, in step S531, the file control unit 43 of the digital camera 210 selects an encrypted grid file to be played back in response to, for example, a user operation, and selects a main image to be played back from the encrypted grid file. Furthermore, the file control unit 43 reconstructs the main image using the divided images and the encrypted divided images that constitute the main image to be played back, and displays the main image on the liquid crystal panel 19.

[0468] In the main image reconstructed in step S531, the area of ​​the encrypted divided image is in a state in which the subject appearing in that area is not visible, and after such a main image is displayed in step S531, processing proceeds to step S532.

[0469] In step S532, for example, when the user designates an encrypted split image on the main image displayed in step S531, the file control unit 43 selects the encrypted split image designated by the user as the target encrypted split image. Furthermore, the file control unit 43 obtains the encryption encryption key associated with the target encrypted split image (the encryption encryption key corresponding to the encrypted split image) from the encrypted grid file to be played back, and the process proceeds from step S532 to step S533.

[0470] In step S533, the file control unit 43 attempts to obtain a private key corresponding to the target encrypted split image (a private key corresponding to the public key used for encryption to obtain the target encrypted split image), similar to, for example, step S232 in FIG. 40, and processing proceeds to step S534.

[0471] In step S534, the file control unit 43 determines whether or not the private key corresponding to the target encrypted divided image has been acquired.

[0472] If it is determined in step S534 that the private key could not be acquired, the process proceeds to step S535, where the same error processing as in step S234 in Fig. 40 is performed, and the playback process ends. Therefore, a user who cannot obtain the private key corresponding to the target encrypted split image cannot view the main image that appears in the split images obtained by decrypting the target encrypted split image.

[0473] On the other hand, if it is determined in step S534 that the private key has been acquired, the process proceeds to step S536.

[0474] In step S536, the file control unit 43 decrypts the encrypted encryption key obtained in step S532 to a common key using the private key obtained in step S533, and the process proceeds to step S537.

[0475] In step S537, the file control unit 43 decrypts the target encrypted divided image into divided images by using the common key decrypted in step S536, and the process proceeds to step S538.

[0476] In step S538, the file control unit 43 reconstructs the main image using the split images obtained by decrypting the targeted encrypted split image instead of the targeted encrypted split image, and displays it on the liquid crystal panel 19, and the playback process ends.

[0477] FIG. 63 is a flowchart illustrating a second example of pre-processing performed when an encrypted grid file is handled by the image processing system 200.

[0478] In the second example of pre-processing, in step S611, the digital camera 210 of the image processing system 200 sends a request for a predetermined number of public keys to the external computer 101, and the process proceeds to step S612.

[0479] For example, if the method of dividing the main image into divided images of the same size, i.e., the number of horizontal and vertical divided images that make up the main image, is set in advance, public keys are required for the number of divided images that make up the main image.

[0480] In steps S512 to S514, the same processes as in steps S212 to S214 in Fig. 38 are carried out. As a result, in the digital camera 210, the maximum number of public keys that may be required, that is, the number of public keys corresponding to the divided images that make up the main image, are obtained and stored as public keys for each of the divided images that make up the main image.

[0481] FIG. 64 is a flowchart illustrating a second example of a generation process for generating an encrypted grid file when the encrypted grid file is handled by the image processing system 200. As shown in FIG.

[0482] In the second example of the generation process, in step S621, the digital camera 210 captures an image in response to, for example, a user operation, and the process proceeds to step S622.

[0483] In step S622, the file control unit 43 controls the encoding control unit 42 to divide the main image obtained by capturing the image in step S621. In response to the control of the file control unit 43, the encoding control unit 42 divides the main image into a plurality of divided images of the same size, for example, in a division manner set in advance, and supplies the divided images to the file control unit 43, and the process proceeds from step S622 to step S623.

[0484] In step S623, the recognition unit 211 performs image recognition using the image captured in step S621, and the process proceeds to step S624.

[0485] In step S624, file control unit 43 generates a common key for each of the divided images that form the main image and that contain the recognition target recognized by image recognition unit 211, and the process proceeds to step S625.

[0486] In step S625, the file control unit 43 generates an encrypted split image by encrypting, among the split images obtained by dividing the main image, a split image that contains a recognition target recognized by the image recognition of the recognition unit 211, with a common key for that split image, and the process proceeds to step S626.

[0487] In step S626, the file control unit 43 generates an encryption key for each split image containing the recognition target recognized by the image recognition of the recognition unit 211 by encrypting the common key for the split image containing the recognition target using the public key for that split image stored in the second example of preprocessing, and the process proceeds from step S626 to step S627.

[0488] In step S627, the file control unit 43 stores the split images obtained by dividing the main image that do not include the recognition target, and generates an encrypted grid file that stores the encrypted split images generated in step S625 in association with the encryption encryption key corresponding to the encrypted split images generated in step S626, and then the generation process ends.

[0489] The playback process for playing back the encrypted grid file generated in the second example of the generation process is similar to that shown in FIG. 62, for example, and therefore will not be described.

[0490] FIG. 65 is a flowchart illustrating a third example of a generation process for generating an encrypted grid file when the encrypted grid file is handled by the image processing system 200. As shown in FIG.

[0491] In the first and second examples of the generation process, the common key is encrypted with a public key that was previously stored in digital camera 210 by the first and second examples of preprocessing, respectively. However, in the third example of the generation process, the necessary public key is obtained in the generation process without performing preprocessing.

[0492] In the third example of the generation process, in steps S721 and S722, the same processes as in steps S521 and S522 in FIG. 61 are performed, respectively, and the process proceeds to step S723.

[0493] In step S723, digital camera 210 obtains and stores a public key for a new recognition target among the recognition targets recognized by image recognition by recognition unit 211 in step S722, and the process proceeds to step S724.

[0494] That is, the digital camera 210 stores the recognition target recognized by the image recognition of the recognition unit 211, and when a new recognition target is recognized, it acquires and stores a public key for the new recognition target. Specifically, the digital camera 210 transmits a request for public keys to the external computer 101 for the number of new recognition targets. In response to the public key request from the digital camera 210, the external computer 101 generates a public key and a corresponding private key for the new recognition target and transmits the public key to the digital camera 210. The digital camera 210 receives and stores the public key for the new recognition target from the external computer 101.

[0495] In step S724, the file control unit 43 controls the encoding control unit 42 to divide the main image obtained by capturing the image in step S721 according to the recognition target recognized by the image recognition of the recognition unit 211. The encoding control unit 42 divides the main image into a plurality of divided images in accordance with the control of the file control unit 43, for example, similar to step S523 in Fig. 61, and supplies them to the file control unit 43, and the process proceeds from step S724 to step S725.

[0496] In step S725, file control unit 43 generates a common key for each recognition target recognized by image recognition by recognition unit 211, and the process proceeds to step S726.

[0497] In step S726, the file control unit 43 generates encrypted split images by encrypting the split images, among those divided from the main image, that contain a recognition target recognized by the image recognition of the recognition unit 211, with a common key for that recognition target, and the process proceeds to step S727.

[0498] In step S727, the file control unit 43 generates an encryption key for each recognition target by encrypting the common key for each recognition target recognized by the image recognition of the recognition unit 211 with the public key for that recognition target, and the process proceeds to step S728.

[0499] In step S728, the file control unit 43 stores the split images obtained by dividing the main image that do not include the recognition target, and generates an encrypted grid file that stores the encrypted split images generated in step S726 in association with the encryption encryption key corresponding to the encrypted split images generated in step S727, and then the generation process ends.

[0500] In this embodiment, a common key is generated for each recognition target recognized by image recognition, and the divided images in which the recognition target appears are encrypted with the common key for the recognition target. However, the common key can also be generated in other ways, for example, for each divided image in which the recognition target appears by image recognition, and the divided images in which the recognition target appears can be encrypted with the common key for the divided images.

[0501] <Description of the computer to which this technology is applied>

[0502] Next, the series of processes of the above-mentioned file control unit 43 and other blocks constituting the signal processing unit 13 (FIG. 1) can be performed by hardware or software. When the series of processes are performed by software, a program constituting the software is installed in a computer or the like.

[0503] FIG. 66 is a block diagram showing an example of the configuration of an embodiment of a computer in which a program for executing the series of processes described above is installed.

[0504] The program can be recorded in advance on the hard disk 905 or ROM 903 as a recording medium built into the computer.

[0505] Alternatively, the program can be stored (recorded) in a removable recording medium 911 driven by the drive 909. Such a removable recording medium 911 can be provided as a so-called package software. Here, examples of the removable recording medium 911 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

[0506] The program can be installed in the computer from the removable recording medium 911 as described above, or can be downloaded to the computer via a communication network or a broadcasting network and installed in the built-in hard disk 905. That is, the program can be transferred to the computer wirelessly from a download site via an artificial satellite for digital satellite broadcasting, or transferred to the computer by wire via a network such as a LAN (Local Area Network) or the Internet.

[0507] The computer includes a CPU (Central Processing Unit) 902 , to which an input / output interface 910 is connected via a bus 901 .

[0508] When a user inputs a command by operating an input unit 907 via an input / output interface 910, the CPU 902 executes a program stored in a read only memory (ROM) 903 in accordance with the command. Alternatively, the CPU 902 loads a program stored in a hard disk 905 into a random access memory (RAM) 904 and executes the program.

[0509] As a result, the CPU 902 performs processing according to the above-mentioned flowchart or processing performed by the configuration of the above-mentioned block diagram. Then, the CPU 902 outputs the processing result from the output unit 906 via the input / output interface 910, or transmits it from the communication unit 908, or further records it on the hard disk 905, as necessary.

[0510] The input unit 907 is composed of a keyboard, a mouse, a microphone, etc. The output unit 906 is composed of an LCD (Liquid Crystal Display), a speaker, etc.

[0511] Here, in this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order according to the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing executed in parallel or individually (for example, parallel processing or processing by objects).

[0512] The program may be processed by one computer (processor), or may be distributed among multiple computers. Furthermore, the program may be transferred to a remote computer for execution.

[0513] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device in which multiple modules are housed in a single housing, are both systems.

[0514] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0515] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0516] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0517] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices.

[0518] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0519] The present technology can have the following configurations.

[0520] <1> an encrypted image obtained by encrypting an image with a first encryption key; an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key; A file control unit that generates a file that associates and stores the File processing device. <2> The file further stores the first encryption key in plaintext. <1> The file processing device according to claim 1 . <3> The encryption key is a specific information-containing encryption key obtained by encrypting specific information for identifying external data outside the file to be associated with the image and the first encryption key with the second encryption key. <1> The file processing device according to claim 1 . <4> The external data is encrypted with the first encryption key. <3> The file processing device according to claim 1 . <5> The specific information in plain text and the first encryption key are further stored in the file. <3> or <4> The file processing device according to claim 1 . <6> The encrypted image is an encrypted divided image obtained by dividing the image and encrypting the divided images with the first encryption key. <1> or <2> The file processing device according to claim 1 . <7> the encrypted divided images are encrypted with the first encryption key, which differs for each subject appearing in the corresponding divided image; The first encryption key used for encrypting the divided images is encrypted with the second encryption key, which is different for each of the subjects. <6> The file processing device according to claim 1 . <8> The image is divided into the divided images according to the object appearing in the image. <6> or <7> The file processing device according to claim 1 . <9> the first encryption key is a common key of a common key cryptosystem; the second encryption key is a public key of a public key cryptosystem; The file is a High Efficiency Image File Format (HEIF) file. <1> Or <8> 2. A file processing device according to claim 1 , <10> an encrypted image obtained by encrypting an image with a first encryption key; an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key; Generate a file that associates and stores 23. A file processing method comprising: <11> an encrypted image obtained by encrypting an image with a first encryption key; an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key; a file control unit that decrypts the encrypted encryption key of the file stored in association with the first encryption key, and decrypts the encrypted image to the image by using the first encryption key obtained by the decryption. File processing device. <12> The file control unit If the file further contains the first encryption key in plaintext, the encrypted image is decrypted to the image using the first encryption key in plaintext; If the first encryption key is not stored in the file in plaintext, the encrypted encryption key is decrypted to the first encryption key, and the encrypted image is decrypted to the image by the first encryption key obtained by the decryption. <11> The file processing device according to claim 1 . <13> the encryption key is a specific-information-containing encryption key obtained by encrypting specific information for identifying external data outside the file to be associated with the image and the first encryption key with the second encryption key, The file control unit decrypts the specific information-containing encryption key into the specific information and the first encryption key, and acquires the external data identified by the specific information obtained by the decryption. <11> The file processing device according to claim 1 . <14> The file control unit, when the external data is encrypted with the first encryption key, decrypts the encrypted external data with the first encryption key. <13> The file processing device according to claim 1 . <15> The file control unit When the plaintext identifying information and the first encryption key are further stored in the file, the external data identified by the plaintext identifying information is acquired; If the specific information and the first encryption key are not stored in the file in plain text, the specific information-containing encryption key is decrypted into the specific information and the first encryption key, and the external data identified by the specific information obtained by the decryption is acquired. <13> or <14> The file processing device according to claim 1 . <16> the encrypted image is an encrypted divided image obtained by dividing the image and encrypting the divided image with the first encryption key, The file control unit decrypts the encrypted encryption key into the first encryption key, and decrypts the encrypted split images into the split images by using the first encryption key obtained by the decryption. <11> or <12> The file processing device according to claim 1 . <17> the encrypted divided images are encrypted with the first encryption key, which differs for each subject appearing in the corresponding divided image; the first encryption key used for encrypting the divided images is encrypted with the second encryption key, which is different for each of the subjects; The file control unit decrypts the encrypted encryption key obtained by encrypting the first encryption key for the target subject into the first encryption key, and decrypts the encrypted split images including the target subject into the split images using the first encryption key for the target subject obtained by the decryption. <16> The file processing device according to claim 1 . <18> The image is divided into the divided images according to the object appearing in the image. <16> or <17> The file processing device according to claim 1 . <19> the first encryption key is a common key of a common key cryptosystem; the second encryption key is a public key of a public key cryptosystem; The file is a High Efficiency Image File Format (HEIF) file. <11> Or <18> 2. A file processing device according to claim 1 , <20> an encrypted image obtained by encrypting an image with a first encryption key; an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key; The encrypted encryption key of the file stored in association with the first encryption key is decrypted into the first encryption key, and the encrypted image is decrypted into the image by the first encryption key obtained by the decryption. 23. A file processing method comprising: [Explanation of symbols]

[0521] 10 digital camera, 11 optical system, 13 signal processing section, 14 media, 15,16 interface, 17 button / key, 18 touch panel, 19 liquid crystal panel, 20 viewfinder, 21 interface, 41 optical system / image sensor control section, 42 encoding control section, 43 file control section, 44 media control section, 45 operation control section, 46 display control section, 47 UI control section, 901 bus, 902 CPU, 903 ROM, 904 RAM, 905 hard disk, 906 output section, 907 input section, 908 communication section, 909 drive, 910 input / output interface, 911 removable recording medium

Claims

1. an encrypted image obtained by encrypting an image with a first encryption key; an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key; a file control unit that generates a file in which the above-mentioned items are associated with each other and stored; The file further stores the first encryption key in plain text. File processing device.

2. The encryption key is a specific information-containing encryption key obtained by encrypting specific information for identifying external data outside the file to be associated with the image and the first encryption key with the second encryption key. The file processing device according to claim 1 .

3. The external data is encrypted with the first encryption key. The file processing device according to claim 2 .

4. The specific information is further stored in the file in plain text. The file processing device according to claim 2 .

5. The encrypted image is an encrypted divided image obtained by dividing the image and encrypting the divided images with the first encryption key. The file processing device according to claim 1 .

6. the encrypted divided images are encrypted with the first encryption key, which differs for each subject appearing in the corresponding divided image; The first encryption key used for encrypting the divided images is encrypted with the second encryption key, which is different for each of the subjects. The file processing device according to claim 5 .

7. The image is divided into the divided images according to the object appearing in the image. The file processing device according to claim 5 .

8. the first encryption key is a common key of a common key cryptosystem; the second encryption key is a public key of a public key cryptosystem; The file is a High Efficiency Image File Format (HEIF) file. The file processing device according to claim 1 .

9. A file processing device comprising: an encrypted image obtained by encrypting an image with a first encryption key; an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key; and store them together. The first encryption key in plaintext Generate a file that contains 23. A file processing method comprising:

10. an encrypted image obtained by encrypting an image with a first encryption key; an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key; a file control unit that decrypts the encrypted encryption key of the file stored in association with the first encryption key, and decrypts the encrypted image into the image by using the first encryption key obtained by the decryption; The file control unit If the file further contains the first encryption key in plain text, the encrypted image is decrypted to the image using the first encryption key in plain text; If the first encryption key is not stored in the file in plain text, the encrypted encryption key is decrypted to the first encryption key, and the encrypted image is decrypted to the image by the first encryption key obtained by the decryption. File processing device.

11. the encryption cryptographic key is a specific-information-containing cryptographic key obtained by encrypting specific information for identifying external data outside the file to be associated with the image and the first cryptographic key with the second cryptographic key, The file control unit decrypts the specific information-containing encryption key into the specific information and the first encryption key, and acquires the external data identified by the specific information obtained by the decryption. The file processing device according to claim 10.

12. The file control unit, when the external data is encrypted with the first encryption key, decrypts the encrypted external data with the first encryption key. The file processing device according to claim 11.

13. The file control unit When the file further stores the identifying information in plain text and the first encryption key, the external data identified by the identifying information in plain text is acquired; If the specific information and the first encryption key are not stored in the file in plain text, the specific information-containing encryption key is decrypted into the specific information and the first encryption key, and the external data identified by the specific information obtained by the decryption is acquired. The file processing device according to claim 11.

14. the encrypted image is an encrypted divided image obtained by dividing the image and encrypting the divided images with the first encryption key; When the first encryption key in plaintext is not stored in the file, the file control unit decrypts the encrypted encryption key into the first encryption key, and decrypts the encrypted split images into the split images with the first encryption key obtained by the decryption. The file processing device according to claim 10.

15. the encrypted divided images are encrypted with the first encryption key, which differs for each subject appearing in the corresponding divided image; the first encryption key used for encrypting the divided images is encrypted with the second encryption key, which is different for each of the subjects; When the first encryption key in plain text is not stored in the file, the file control unit decrypts the encrypted encryption key obtained by encrypting the first encryption key for the target subject of interest into the first encryption key, and decrypts the encrypted split images in which the target subject is captured into the split images using the first encryption key for the target subject obtained by the decryption. The file processing device according to claim 14.

16. The image is divided into the divided images according to the object appearing in the image. The file processing device according to claim 14.

17. the first encryption key is a common key of a common key cryptosystem; the second encryption key is a public key of a public key cryptosystem; The file is a High Efficiency Image File Format (HEIF) file. The file processing device according to claim 10.

18. A file processing device comprising: an encrypted image obtained by encrypting an image with a first encryption key; an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key; The file stored with the associated If the first encryption key in plaintext is further stored, decrypting the encrypted image to the image using the first encryption key in plaintext; If the first encryption key in plaintext is not stored, the encrypted encryption key is decrypted to the first encryption key, and the encrypted image is decrypted to the image by the first encryption key obtained by the decryption.

23. A file processing method comprising:

19. An encrypted image obtained by encrypting an image with a first encryption key; an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key; a file control unit that generates a file in which the above-mentioned items are associated with each other and stored; The encryption key is a specific information-containing encryption key obtained by encrypting specific information for identifying external data outside the file to be associated with the image and the first encryption key with the second encryption key. File processing device.

20. An encrypted image obtained by encrypting an image with a first encryption key; an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key; a file control unit that decrypts the encrypted encryption key of the file stored in association with the first encryption key, and decrypts the encrypted image to the image by the first encryption key obtained by the decryption; the encryption cryptographic key is a specific-information-containing cryptographic key obtained by encrypting specific information for identifying external data outside the file to be associated with the image and the first cryptographic key with the second cryptographic key, The file control unit decrypts the specific information-containing encryption key into the specific information and the first encryption key, and acquires the external data identified by the specific information obtained by the decryption. File processing device.

Images