File processing apparatus and file processing method

Abstract

The present technology relates to a file processing apparatus and a file processing method that enable limiting viewing of an image. A file control unit generates a file in which an encrypted image obtained by encrypting an image with a first encryption key and an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key are associated with each other and stored. Further, the file control unit also decrypts the encrypted encryption key in the file into the first encryption key, and decrypts the encrypted image into an image using the first encryption key obtained by the decryption. The present technology can be applied to, for example, a digital camera or the like that captures an image.

Description

Technical Field

[0001] The present technology relates to a document processing device and a document processing method, and more particularly, for example, to a document processing device and a document processing method that enable limiting viewing of an image. Background Art

[0002] As a file format for efficiently storing images, there is the High Efficiency Image File Format (HEIF) (see Non-Patent Document 1).

[0003] Citation List

[0004] Non-patent literature

[0005] Non-patent document 1: ISO / IEC 23008-12:2017, Information technology--Highefficiency coding and media delivery in heterogeneous environments--Part 12:Image File Format Summary of the Invention

[0006] Problems to be solved by the present invention

[0007] For HEIF files or some other files storing images, it would be user-friendly if viewing of the images stored in the file could be restricted to protect so-called copyright, portrait rights, etc.

[0008] The present technology has been made in view of such circumstances, and aims to enable limiting viewing of images stored in files.

[0009] Solution to the problem

[0010] A first file processing device of the present technology is a file processing device including a file control unit that generates a file storing an encrypted image obtained by encrypting the image with a first encryption key and an encrypted encryption key obtained by encrypting the first encryption with a second encryption key, the encrypted image and the encrypted encryption key being associated with each other in the file.

[0011] A first file processing method of the present technology is a file processing method including generating a file storing an encrypted image obtained by encrypting the image with a first encryption key and an encrypted encryption key obtained by encrypting the first encryption with a second encryption key, the encrypted image and the encrypted encryption key being associated with each other in the file.

[0012] In the first file processing device and file processing method of the present technology, a file is generated that stores an encrypted image obtained by encrypting the image with a first encryption key and an encrypted encryption key obtained by encrypting the first encryption with a second encryption key, and the encrypted image and the encrypted encryption key are associated with each other in the file.

[0013] A second file processing device of the present technology is a file processing device including a file control unit, which decrypts an encrypted image into an image using a first encryption key, the first encryption key being obtained by decrypting an encryption key encrypted in a file, the file storing an encrypted image obtained by encrypting the image using the first encryption key and an encrypted encryption key obtained by encrypting the first encryption using a second encryption key, the encrypted image and the encrypted encryption key being associated with each other in the file.

[0014] A second file processing method of the present technology is a file processing method including decrypting an encrypted image into an image using a first encryption key, the first encryption key being obtained by decrypting an encryption key encrypted in a file, the file storing an encrypted image obtained by encrypting the image using the first encryption key and an encrypted encryption key obtained by encrypting the first encryption using a second encryption key, the encrypted image and the encrypted encryption key being associated with each other in the file.

[0015] In a second file processing device and a file processing method of the present technology, an encrypted image is decrypted into an image using a first encryption key, the first encryption key being obtained by decrypting an encryption key encrypted in a file storing an encrypted image obtained by encrypting the image using the first encryption key and an encrypted encryption key obtained by encrypting the first encryption using a second encryption key, the encrypted image and the encrypted encryption key being associated with each other in the file.

[0016] Note that the first and second file processing devices may be independent devices, or may be internal blocks in a single device.

[0017] Alternatively, the first and second file processing devices may be formed by causing a computer to execute a program. The program for causing a computer to function as the first and second file processing devices may be recorded on a recording medium and provided, or transmitted and provided via a transmission medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a block diagram showing an example configuration of an embodiment of a digital camera to which the present technology is applied.

[0019] Figure 2is a diagram showing an example format of a JPEG file conforming to the Joint Photographic Experts Group (JPEG).

[0020] Figure 3 is a diagram showing an example of an ISO-based media file format.

[0021] Figure 4 is a diagram illustrating an example format of a HEIF file compliant with HEIF.

[0022] Figure 5 is a diagram showing an example format of a HEIF file in an image item format.

[0023] Figure 6 is a diagram showing an example of an iprp box.

[0024] Figure 7 is a diagram showing an example format of a HEIF file in image sequence format.

[0025] Figure 8 is a diagram showing an example of a trak box.

[0026] Figure 9 is a diagram showing an example of a conventional collection file storing a main image and thumbnail images.

[0027] Figure 10 is a diagram showing an example of a first association set file.

[0028] Figure 11 is a diagram showing an example of a second association set file.

[0029] Figure 12 is a diagram showing an example of a third association set file.

[0030] Figure 13 is a diagram showing an example of a conventional sequence file in which a track of a main image and a track of thumbnail images of the main image are stored.

[0031] Figure 14 is a diagram showing an example of an associated sequence file.

[0032] Figure 15 is a flowchart for explaining an overview of an example of a generation process for generating an associated HEIF file.

[0033] Figure 16 is a flowchart for explaining an outline of an example of a reproduction process for reproducing an associated HEIF file.

[0034] Figure 17 is a flowchart for explaining an example of a reproduction process for reproducing a set file.

[0035] Figure 18: is a flowchart for explaining an example of a process of reading a reproduction target image in step S32 .

[0036] Figure 19 : is a flowchart for explaining a first example of a process of acquiring a reproduction target item ID in step S31.

[0037] Figure 20 4 is a flowchart for explaining a second example of the process of acquiring the reproduction target item ID in step S31.

[0038] Figure 21 is a flowchart for explaining an example of a process of acquiring a uuid as specific information about a RAW file of a predetermined main image from a first association set file.

[0039] Figure 22 is a flowchart for explaining an example of a process of acquiring uuid, which is specific information about a RAW file of a predetermined main image, from a second association set file.

[0040] Figure 23 is a flowchart for explaining an example of a process of acquiring uuid, which is specific information about a RAW file of a predetermined main image, from a third association set file.

[0041] Figure 24 is a flowchart for explaining an example of a process of acquiring a list of item IDs of main images from a collection file.

[0042] Figure 25 is a flowchart for explaining an example of a process of reproducing thumbnail images of (frames of) a main image corresponding to predetermined time information from a sequence file.

[0043] Figure 26 is a flowchart for explaining an example of a process of acquiring a uuid, which is specific information about a RAW file of (a frame of) a predetermined main image, from an associated sequence file.

[0044] Figure 27 : is a diagram showing an example of storing uuid in a RAW file when a RAW file of a main image is used as external data and an associated set file is generated.

[0045] Figure 28 : is a diagram showing an example of storing uuid in a RAW file when a RAW file of a main image is adopted as external data and a related sequence file is generated.

[0046] Figure 29 : is a diagram showing an example of storing uuid in a WAV file when a WAV file of a main image is used as external data and an associated set file is generated.

[0047] Figure 30 : is a diagram showing an example of storing uuid in a WAV file in the case of adopting a WAV file of a main image as external data and generating an associated sequence file.

[0048] Figure 31 is a block diagram showing an example configuration of a first embodiment of an image processing system to which the present technology is applied.

[0049] Figure 32 is a diagram illustrating an example of a first encrypted HEIF file.

[0050] Figure 33 is a diagram illustrating an example of a second encrypted HEIF file.

[0051] Figure 34 is a diagram showing an example of a third encrypted HEIF file.

[0052] Figure 35 is a diagram illustrating an example of a fourth encrypted HEIF file.

[0053] Figure 36 is a diagram showing an example of a fifth encrypted HEIF file.

[0054] Figure 37 is a diagram showing an example of the sixth encrypted HEIF file.

[0055] Figure 38 is a flowchart for explaining an example of pre-processing to be performed in the case where a first encrypted HEIF file is handled by the image processing system 100 .

[0056] Figure 39 1 is a flowchart for explaining an example of a generation process of generating a first encrypted HEIF file in a case where the first encrypted HEIF file is handled by the image processing system 100 .

[0057] Figure 40 1 is a flowchart for explaining an example of a reproduction process of reproducing a first encrypted HEIF file in a case where the first encrypted HEIF file is handled by the image processing system 100 .

[0058] Figure 41 is a flowchart for explaining an example of pre-processing to be performed in the case where the second encrypted HEIF file is handled by the image processing system 100.

[0059] Figure 42 1 is a flowchart for explaining an example of a generation process of generating a second encrypted HEIF file in a case where the second encrypted HEIF file is handled by the image processing system 100 .

[0060] Figure 43 This is a flowchart for explaining an example of a restriction process of restricting viewing of a main image obtained by decrypting an encrypted image to only users who can obtain a private key corresponding to the encrypted image when a second encrypted HEIF file is handled by the image processing system 100.

[0061] Figure 44 1 is a flowchart for explaining an example of a reproduction process of reproducing a second encrypted HEIF file in a case where the second encrypted HEIF file is handled by the image processing system 100 .

[0062] Figure 45 is a flowchart for explaining an example of pre-processing to be performed in the case where the third encrypted HEIF file is handled by the image processing system 100 .

[0063] Figure 46 1 is a flowchart for explaining an example of a generation process of generating a third encrypted HEIF file in a case where the third encrypted HEIF file is handled by the image processing system 100 .

[0064] Figure 47 1 is a flowchart for explaining an example of a reproduction process of reproducing the third encrypted HEIF file in a case where the third encrypted HEIF file is handled by the image processing system 100 .

[0065] Figure 48 is a flowchart for explaining an example of pre-processing to be performed in the case where the fourth encrypted HEIF file is handled by the image processing system 100.

[0066] Figure 49 1 is a flowchart for explaining an example of a generation process of generating a fourth encrypted HEIF file in a case where the fourth encrypted HEIF file is handled by the image processing system 100 .

[0067] Figure 50 1 is a flowchart for explaining an example of a reproduction process of reproducing the fourth encrypted HEIF file in a case where the fourth encrypted HEIF file is handled by the image processing system 100 .

[0068] Figure 51 is a flowchart for explaining an example of pre-processing to be performed in the case where the fifth or sixth encrypted HEIF file is handled by the image processing system 100 .

[0069] Figure 52 1 is a flowchart for explaining an example of a generation process of generating the fifth or sixth encrypted HEIF file in a case where the fifth or sixth encrypted HEIF file is handled by the image processing system 100 .

[0070] Figure 53This is a flowchart for explaining an example of a restriction process of limiting viewing of a main image obtained by decrypting an encrypted image to only users who can obtain a private key corresponding to the encrypted image when the fifth or sixth encrypted HEIF file is handled by the image processing system 100.

[0071] Figure 54 1 is a flowchart for explaining an example of a reproduction process of reproducing the fifth or sixth encrypted HEIF file in a case where the fifth or sixth encrypted HEIF file is handled by the image processing system 100 .

[0072] Figure 55 is a block diagram illustrating an example configuration of a second embodiment of an image processing system to which the present technology is applied.

[0073] Figure 56 is a block diagram illustrating an example configuration of the digital camera 210 .

[0074] Figure 57 is a diagram showing an example of a mesh file.

[0075] Figure 58 is a diagram for explaining an outline of encrypted mesh file processing to be executed by the digital camera 210 .

[0076] Figure 59 is a diagram showing an example of an encrypted mesh file.

[0077] Figure 60 is a flowchart for explaining a first example of pre-processing to be performed in the case where an encrypted mesh file is handled by the image processing system 200 .

[0078] Figure 61 1 is a flowchart for explaining a first example of a generation process of generating an encrypted mesh file in a case where the encrypted mesh file is handled by the image processing system 200 .

[0079] Figure 62 is a flowchart for explaining an example of a reproduction process of reproducing an encrypted mesh file in a case where the encrypted mesh file is handled by the image processing system 200 .

[0080] Figure 63 is a flowchart for explaining a second example of pre-processing to be performed in the case where an encrypted mesh file is handled by the image processing system 200 .

[0081] Figure 64 1 is a flowchart for explaining a second example of a generation process of generating an encrypted mesh file in a case where the encrypted mesh file is handled by the image processing system 200 .

[0082] Figure 651 is a flowchart for explaining a third example of a generation process of generating an encrypted mesh file in a case where the encrypted mesh file is handled by the image processing system 200 .

[0083] Figure 66 is a block diagram illustrating an example configuration of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION

[0084] <Embodiment of a Digital Camera to Which the Present Technology is Applied>

[0085] Figure 1 is a block diagram showing an example configuration of an embodiment of a digital camera to which the present technology is applied.

[0086] The digital camera 10 includes an optical system 11 , an image sensor 12 , a signal processing unit 13 , a medium 14 , interfaces 15 and 16 , a button / key 17 , a touch panel 18 , a liquid crystal panel 19 , a viewfinder 20 , an interface 21 , and the like.

[0087] The optical system 11 condenses light from a subject onto the image sensor 12 .

[0088] The image sensor 12 generates image data as an electric signal by receiving light from the optical system 11 and performing imaging involving photoelectric conversion, and supplies the image data to the signal processing unit 13 .

[0089] The signal processing unit 13 includes an optical system / image sensor control unit 41 , a decoding control unit 42 , a file control unit 43 , a media control unit 44 , an operation control unit 45 , a display control unit 46 , and a UI control unit 47 .

[0090] The optical system / image sensor control unit 41 controls the optical system 11 and the image sensor 12 , and supplies (data of) an image obtained by imaging performed according to the control to the decoding control unit 42 .

[0091] The decoding control unit 42 supplies the image from the optical system / image sensor control unit 41 to the display control unit 46, and encodes the image when necessary and supplies the encoded image to the file control unit 43. The decoding control unit 42 also decodes the image supplied from the file control unit 43 when necessary and supplies the decoded image to the display control unit 46.

[0092] File control unit 43 generates a file storing the image supplied from decode control unit 42 and supplies the file to media control unit 44. File control unit 43 also reproduces the file supplied from media control unit 44. In other words, file control unit 43 reads data such as the image stored in the file. For example, the image read from the file is supplied from file control unit 43 to decode control unit 42.

[0093] The media control unit 44 controls file exchange between the medium 14 and the interfaces 15 and 16. For example, the media control unit 44 causes the medium 14 to record a file supplied from the file control unit 43, or causes the interfaces 15 and 16 to transfer a file supplied from the file control unit 43. The media control unit 44 also reads a file from the medium 14 or causes the interfaces 15 and 16 to receive a file, and supplies the file to the file control unit 43.

[0094] In accordance with an operation performed by the user on the button / key 17 or the touch panel 18 , the operation control unit 45 supplies an operation signal corresponding to the operation to a necessary block.

[0095] The display control unit 46 performs display control and the like to supply the image and the like supplied from the decoding control unit 42 to the liquid crystal panel 19 , the viewfinder 20 , and the interface 21 , and displays the image and the like.

[0096] The UI control unit 47 manages user interface (UI) control.

[0097] The medium 14 is, for example, a storage medium such as an SD card. The interface 15 is, for example, an interface of a local area network (LAN) such as WiFi (registered trademark) or Ethernet (registered trademark). The interface 16 is, for example, an interface of a universal serial bus (USB). The button / key 17 and the touch panel 18 are operated by the user to input commands or other information into the digital camera 10. The touch panel 18 can be formed integrally with the liquid crystal panel 19. The liquid crystal panel 19 and the viewfinder 20 display images supplied from the display control unit 46, etc. The interface 21 is an interface for transmitting at least an image, such as a high-definition multimedia interface (HDMI (registered trademark)) or a display port (DP).

[0098] In the digital camera 10 designed as described above, the optical system / image sensor control unit 41 generates a YUV image as a main image of, for example, a HEIF file, and supplies the YUV image to the decoding control unit 42. The YUV image is generated from an image of RAW data obtained by imaging performed by the image sensor 12 (this image will also be referred to as a RAW image hereinafter), and the YUV image has the same resolution (same number of pixels) as the RAW image.

[0099] The decoding control unit 42 generates, for example, a YUV image with a lower resolution than the main image (this YUV image will also be referred to as the screen nail image) from the main image as a first other image based on the main image, for display on the liquid crystal panel 19 or an external display. The decoding control unit 42 also generates a YUV image with a lower resolution than the screen nail image (this YUV image will also be referred to as a thumbnail image) as a second other image based on the main image, for use in an index display (list display). For example, the decoding control unit 42 supplies the screen nail image to the liquid crystal panel 19 via the display control unit 46 so that the screen nail image is displayed as a so-called through-the-lens image. For example, an image with a long side size of 320 pixels or less can be used as a thumbnail image. The size (pixel count) ratio between the main image and the thumbnail image serving as the first other image based on the main image or the second other image based on the main image can be, for example, 200 or less. Similarly, the size ratio between the screen nail image serving as the first other image based on the main image and the thumbnail image serving as the second other image based on the main image can be 200 or less. As the screen nail image, for example, an image with a resolution of 4K or higher can be used. Moreover, as the screen nail image, for example, a 4K (QFHD) or FHD image can be used according to the user's selection. In addition, an image with the same resolution can be used as the main image and the screen nail image. In the case of using images with the same resolution as the main image and the screen nail image, both the main image and the screen nail image can be stored in a HEIF file, or the main image can be stored in a HEIF file, but the screen nail image is not. In the case where the main image is stored in a HEIF file and the screen nail image is not, the main image can be resized and used as the screen nail image.

[0100] The decoding control unit 42 also encodes the main image, screen nail image and thumbnail image corresponding to the RAW image (the main image, screen nail image and thumbnail image generated from the same RAW image) when necessary, and supplies the encoded image and the RAW image to the file control unit 43.

[0101] The file control unit 43 generates a RAW file in which a RAW image is stored, generates a HEIF file, a JPEG file, etc. in which corresponding main images, screen nail images, and thumbnail images (main images, screen nail images, and thumbnail images generated from the same RAW image) are stored, and supplies the files to the media control unit 44. The HEIF file is a file that conforms to the High Efficiency Image File Format (HEIF), and the JPEG file is a file that conforms to the Joint Photographic Experts Group (JPEG).

[0102] The media control unit 44 records the RAW file and the HEIF file or JPEG file supplied from the file control unit 43 on the medium 14, or transmits the RAW file and the HEIF file or JPEG file from the interface 15 or 16.

[0103] For example, the file control unit 43 can select a HEIF file or a JPEG file as the file to be generated according to the user's operation. In addition, as will be described later, the HEIF file is in the image item format and the image sequence format. For example, the image item format or the image sequence format can be adopted according to the user's operation. In addition, the file control unit 43 can perform mutual conversion between the HEIF file and the JPEG file according to the user's operation.

[0104] In addition, when generating a HEIF file, the file control unit 43 can associate the internal data (data stored in the HEIF file) in the HEIF file with the external data outside the HEIF file (data not stored in the HEIF file) and the specific information for specifying the external data, and store the internal data and the specific information in the HEIF file. The HEIF file in which the internal data and the specific information about the external data associated with the internal data are associated with each other and stored is also called an associated HEIF file. For example, the associated HEIF file can store the associated internal data and specific information by storing the association information for associating the internal data with the specific information, etc.

[0105] <JPEG file>

[0106] Figure 2 is a diagram showing an example format of a JPEG file conforming to the Joint Photographic Experts Group (JPEG).

[0107] The JPEG file is designed to store, for example, Exif metadata, thumbnail images, Extensible Metadata Platform (XMP, registered trademark) metadata, MPF indicating the storage location (position) of the main image and the image for simplified display, etc., the main image, and the image for simplified display. As the image for simplified display, for example, a screen thumbnail image can be adopted.

[0108] <ISO Base Media File Format>

[0109] Figure 3 is a diagram showing an example of the ISO Base Media File Format.

[0110] HEIF (ISO / IEC 23008-12) is a file format conforming to the ISO Base Media File Format (ISO / IEC 14496-12), and accordingly, the HEIF file conforms to the ISO Base Media File Format.

[0111] The ISO base media file format includes units called boxes as containers for storing data and has a structure called the box structure.

[0112] A box includes a type (box type), actual data (data), etc. The type indicates the type of the actual data in the box. The actual data can be reproducible media data such as images (still images or video images), audio, and subtitles, attribute names ((field names and attribute values (field values)) of variables (represented by) the attribute name, and various other data.

[0113] In addition, a box can be adopted as the actual data. That is, a box can have a box as the actual data, so it can have a hierarchical structure.

[0114] A basic media file conforming to the ISO base media file format can include an ftyp box, a moov box (MovieBox), a meta box (MetaBox), a mdat box (MediaDataBox), etc. The ftyp box stores identification information for identifying the file format. The moov box can store trak boxes, etc. The Meta box can store iinf boxes, iprp boxes, iref boxes, iloc boxes, etc. The mdat box can store media data (AV data) and other desired data.

[0115] HEIF conforms to the above ISO base media file format.

[0116] <HEIF file>

[0117] Figure 4 is a diagram showing an example format of a HEIF file conforming to HEIF.

[0118] HEIF files are roughly classified into files in the image item format and files in the image sequence format. In addition, the image item format includes a single image format that contains only one item as described later, and an image set format that contains multiple items.

[0119] A HEIF file in the image item format includes an ftyp box, a meta box, and a mdat box.

[0120] A HEIF file in the image sequence format includes an ftyp box, a moov box, and a mdat box.

[0121] Note that a HEIF file can contain not only a meta box or a moov box, but also both boxes.

[0122] The ftyp box stores identification information for identifying the file format, such as information indicating that the file is a HEIF file in the image item format or the image sequence format.

[0123] The meta box and the moov box store metadata required for reproducing and managing the media data stored in the mdat box, such as metadata indicating the storage location of the media data.

[0124] The mdat box stores media data (AV data) and the like.

[0125] In the digital camera 10, for example, it is possible to select to generate a HEIF file in an image item format or a HEIF file in an image sequence format according to a user operation. In addition, in the case where an image is encoded and stored in an mdat box of a HEIF file, only intra-frame coding is allowed for the image item format, and intra-frame coding and inter-frame coding are allowed for the image sequence format. Accordingly, in the case where high-speed access to data stored in a HEIF file is prioritized, for example, it is possible to select to generate a HEIF file in an image item format. In the case where reducing the size (data amount) of a HEIF file is prioritized, it is possible to select to generate a HEIF file in an image sequence format.

[0126] Figure 5 is a diagram showing an example format of a HEIF file in an image item format.

[0127] In a HEIF file in the image item format, information indicating that the HEIF file is in the image item format (such as, for example, mif1) is stored in the ftyp box (as an attribute value).

[0128] The meta box stores the iinf box, iref box, iprp box, and iloc box.

[0129] The iinf box stores (indicates) the number of items (attribute names and attribute values) of media data (AV data) stored in the mdat box. An item is a piece of data stored in the mdat box of a HEIF file in the image item format, and for example, one image (screen) is one item. In this specification, one image, which is a still image or video image, is also referred to as a frame. One frame is one item.

[0130] The iref box stores information indicating the relationship between items. For example, in the mdat box, corresponding main images, screen nail images, and thumbnail images can be stored as items. When item I1 is the main image, item I2 is the screen nail image, and item I3 is the thumbnail image stored in the mdat box, information indicating that item I2 is the screen nail image that is the main image of item I1, and information indicating that item I3 is the thumbnail image that is the main image of item I1 is stored in the iref box.

[0131] The iprp box stores information about the properties of an item.

[0132] The iloc box stores information about the storage location of the item stored in the mdat box.

[0133] For example, a frame of an image as an item is stored in an mdat box (of a HEIF file) in an image item format. One or more items can be stored in an mdat box. Moreover, a frame encoded as an item can be stored in an mdat box. However, encoding of a frame as an item to be stored in an mdat box in an image item format is limited to intra-frame encoding. As an encoding method (codec) for encoding a frame as an item, HEVC or the like can be used, for example.

[0134] Figure 6 It shows Figure 5 Figure 2 shows an example of an iprp box shown in FIG.

[0135] The iprp box stores the ipco box and ipma box regarding the characteristics of the item. The ipco box stores the characteristics of the item stored in the mdat box, such as codec information regarding the codec of the image as the item and image size information regarding the size. The ipma box stores indexes (pointers) to the characteristics stored in the ipco box, which are related to the item stored in the mdat box.

[0136] Figure 7 is a diagram showing an example format of a HEIF file in image sequence format.

[0137] In a HEIF file in an image sequence format, information indicating that the HEIF file is in an image sequence format (such as msf1) is stored in, for example, an ftyp box.

[0138] The moov box stores the trak box. The trak box stores information about the track stored in the mdat box.

[0139] A track is formed by an independent piece of media data (such as an image or sound) to be reproduced according to a timeline. For example, a track is formed by one or more frames of an image as an elementary stream. For tracks stored in an mdat box, for example, multiple tracks can be reproduced simultaneously, such as corresponding tracks of images and sound recorded simultaneously.

[0140] The media data of a track is composed of units called samples. A sample is the smallest unit (access unit) for accessing media data in a HEIF file. Therefore, media data in a HEIF file cannot be accessed in units smaller than a sample.

[0141] In media data of images, for example, one frame is one sample. Also, in media data of sounds, for example, one audio frame defined in the standard of the media data of sounds is one sample.

[0142] In the mdat box of the image sequence format (HEIF file), the media data of a track is placed in units called chunks. A chunk is a collection of one or more samples placed at logically consecutive addresses.

[0143] In the case where a plurality of tracks as media data are stored in the mdat box, the plurality of tracks are interleaved and placed in blocks.

[0144] As described above, in the mdat box of the picture sequence format, one or more tracks formed of media data such as images and sounds are stored.

[0145] The mdat box can store encoded frames of images that constitute a track. When encoding the frames constituting the track stored in the mdat box in an image sequence format, a long GOP can be used as a group of pictures (GOP), and intra-frame coding and inter-frame coding can be used. As a codec for encoding the frames constituting the track, for example, HEVC can be used.

[0146] Figure 8 is a diagram showing an example of a trak box.

[0147] The trak box can store a tkhd box and an mdia box. The tkhd box stores header information about the track, such as the creation date and time of the track managed by the trak box. The mdia box stores a minf box, etc. The minf box stores an stbl box. The stbl box stores an stsd box, stsc box, stsz box, and stco box that store track samples, which are information used to access chunks. The stsd box stores codec information about the codec of the track. The stsc box stores the chunk size (the number of samples in one chunk). The stsz box stores the sample size. The stco box stores the chunk offset, which is the offset of the placement position of each chunk of the track stored in the mdat box.

[0148] Here, HEIF files in the image item format are also referred to as collection files, and HEIF files in the image sequence format are also referred to as sequence files. In addition, associated HEIF files in the image item format are also referred to as associated collection files, and associated HEIF files in the image sequence format are also referred to as associated sequence files.

[0149] In the digital camera 10 , it is possible to generate a HEIF file (including associated HEIF files) in which a main image and a desired screen nail image and / or thumbnail image are stored.

[0150] <Collection File>

[0151] Figure 9 is a diagram showing an example of a conventional collection file storing a main image and thumbnail images.

[0152] Here, a regular collection file refers to a collection file in which internal data in the collection file is not associated with specific information about external data.

[0153] Here, frames (items) encoded by HEVC are stored in the mdat box of the collection file.

[0154] In the ftyp box, heic indicating that the format is the image item format and the codec is HEVC is stored as identification information for identifying the file format.

[0155] The iinf box stores the number of items stored in the mdat box (number of items). Figure 9 In the mdat box, a total of four items (frames) are stored: the main image specified by item ID #1 (this main image will also be written as main image Item #1 below), main image Item #2, the thumbnail image specified by item ID #101 (this thumbnail image will also be written as thumbnail image Item #101 below), and thumbnail image Item #102. Accordingly, the number of items is four. Note that thumbnail image Item #101 is a thumbnail image of main image Item #1, and thumbnail image Item #102 is a thumbnail image of main image Item #2.

[0156] For example, the iinf box also stores an infe box for each item stored in the mdat box. In the infe box, the item ID and item type for the specified item are registered. Figure 9 In the image, there are corresponding info boxes for the main images Item#1 and Item#2 and the thumbnail images Item#101 and Item#102.

[0157] For example, the iref box stores the thmb box as information for associating the items stored in the mdat box with each other. In the thmb box, a reference source and a reference destination associated with each other are stored as information for associating a main image with a thumbnail image of the main image. In the thmb box, the reference source indicates the item ID of the main image, and the reference destination indicates the item ID of the thumbnail image of the main image specified by the item ID of the reference source. Accordingly, through the reference destination associated with the reference source, the item ID of the thumbnail image of the main image specified by the item ID indicated by the reference source can be identified. Moreover, through the reference source associated with the reference destination, the item ID of the main image of the thumbnail image specified by the item ID indicated by the reference destination can be identified.

[0158] As referenced above Figure 6 As mentioned above, the iprp box stores the ipco box and the ipma box. Figure 6As described above, for example, the ipco box stores characteristics of a frame as items stored in the mdat box, such as codec information about a codec and image size information about a size. Figure 6 As described above, the ipma box stores indexes stored in the ipco box regarding characteristics of items stored in the mdat box.

[0159] As referenced above Figure 6 As described above, the iloc box stores information about the storage location of items in the mdat box. Figure 9 , the information stored in the iloc box indicates that the number of items is 4. In addition, in the iloc box, the offsets and sizes of the respective storage locations of the main images Item#1 and Item#2 and the thumbnail images Item#101 and Item#102 stored in the mdat box are associated with the item ID and stored.

[0160] In the following description, the Figure 9 An associative collection file that stores internal data related to each other and specific information about external data in a regular collection file.

[0161] Figure 10 is a diagram showing an example of a first association set file.

[0162] Here, for example, a RAW image (RAW file in which) the main image is used (stored) as external data associated with the main image which is internal data in the HEIF file.

[0163] The first association set file stores association information for associating a main image (internal data) with specific information about a RAW file (a RAW image stored therein) storing the RAW image (external data). This information is stored in a meta box.

[0164] As specific information about a RAW file storing a RAW image as external data, any information from which it is possible to specify (the RAW image stored in) a RAW file may be employed, such as a file name of the RAW file, a universally unique identifier (UUID) issued to the RAW file, or a uniform resource locator (URL).

[0165] In the first association set file, an association information storage box storing association information is defined as a new box to be stored in the meta box and stored in the meta box. For example, the association information storage box of the first association set file stores association information that associates the item ID and uuid for specifying a main image, as specific information for specifying (storing) the RAW file (the RAW image stored in) associated with the main image. Furthermore, the association information storage box stores the number of main images (the number of main images) associated with the RAW file (the RAW image stored in the RAW file). As the number of main images associated with the RAW file, the number of main images stored in the association information storage box is equal to or less than the number of main images stored in the mdat box.

[0166] exist Figure 10 In the example, the uuid of the RAW file of the main image Item#1 (the uuid of the RAW image associated with the main image Item#1) is UUID#1, and the uuid of the RAW file of the main image Item#2 is UUID#2. In the case where the RAW file whose uuid is UUID#i is written into the RAW file UUID#i, the association information in which the item ID#1 of the main image Item#1 is associated with the uuid of the RAW file UUID#1 and the item ID#2 of the main image Item#2 is associated with the uuid of the RAW file UUID#2 is stored in Figure 10 The associated information is stored in the box.

[0167] Figure 11 is a diagram showing an example of a second association set file.

[0168] In the second association set file, association information is used to associate the main image (internal data) with the specific information about the RAW file (external data). Thus, the main image and the specific information about the RAW file are associated and stored, as in the first association set file. However, in the second association set file, the association information is stored in the mdat box.

[0169] In the second association set file, association information similar to that in the case of the first association set file is stored as an entry in the mdat box, for example. Figure 11 In , the associated information is stored in the mdat box as an item of item ID#201.

[0170] As described above, in the second associated set file, the information stored in the meta box is Figure 9 The situation of the conventional collection file in the first embodiment is different because the associated information is stored in the mdat box as the item Item# 201. In the second associated collection file, metadata as the associated information of the item Item# 201 is stored in the meta box.

[0171] Specifically, in the second association set file, the number of items stored in the iinf box and the iloc box is five. Figure 9 In the case where four are added, Item #201 is added as one of the four. Furthermore, the infe box for Item #201 is added to the iinf box, and the offset and size of the storage location of Item #201 are added to the iloc box. The infe box for Item #201 stores the item ID #201 of Item #201 and the item type IDIF (Identification Data Information) indicating that Item #201 is related information. IDIF is a newly defined attribute value (field value) indicating that the item is related information.

[0172] Figure 12 is a diagram showing an example of a third association set file.

[0173] In the third association set file, specific information about the RAW file as external data is stored as an item for each piece of specific information in the mdat box, and association information for associating the main image as internal data with the specific information about the RAW file as external data is stored in the meta box, thereby associating the main image and the specific information about the RAW file and storing them. However, in the third association set file, the association information is information in which the item ID of the main image as an item is associated with the item ID of the specific information (about the RAW file) as an item, and is stored in the cdsc box in the iref box stored in the meta box.

[0174] In the CDSC box, a reference source and a reference destination associated with each other may be stored as information for associating items, the items being a main image and specific information about the RAW file of the main image. In the CDSC box, the reference source indicates the item ID of the main image, while the reference destination indicates the item ID of the specific information of the RAW file of the main image specified by the item ID of the reference source.

[0175] exist Figure 12In the example, UUID#1, which is the UUID of the specific information about the RAW file of main image Item#1, is stored as item Item#201 in the mdat box, and UUID#2, which is the UUID of the specific information about the RAW file of main image Item#2, is stored as item Item#202 in the mdat box. In addition, the cdsc box storing the association information of item ID#1 of main image Item#1 and item ID#201 of specific information UUID#1 as a reference source and a reference destination is stored in the iref box, and the cdsc box storing the association information of item ID#2 of main image Item#2 and item ID#202 of specific information UUID#2 as a reference source and a reference destination is stored in the iref box.

[0176] <sequence file>

[0177] Figure 13 is a diagram showing an example of a conventional sequence file in which a track of a main image and a track of thumbnail images of the main image are stored.

[0178] Here, the regular sequence refers to a sequence file in which internal data in the sequence file is not associated with specific information about external data.

[0179] Here, frames encoded by HEVC are stored in the mdat box of the sequence file.

[0180] In the ftyp box, hevc indicating that the format is the image sequence format and the codec is HEVC is stored as identification information for identifying the file format.

[0181] The moov box stores and manages the trak box of the corresponding track stored in the mdat box, as referenced above. Figure 7 As stated. Figure 13 In the [1], the track of the main image specified by track ID #1 (this track will also be written as track #1 below) and track #2 of the thumbnail images of the main image of track #1 are stored in the mdat box. Accordingly, the moov box stores the trak box that manages track #1 and the trak box that manages track #2. The nth thumbnail image (from the top) of track #2 is the thumbnail image of the nth main image of track #1.

[0182] For example, a sequence file is useful in cases such as when continuous shooting is performed by the digital camera 10 and main images of a plurality of frames obtained by the continuous shooting are recorded as one track and thumbnail images of the main images are also recorded as one track.

[0183] The tkhd box of the trak box for managing track #1 for main images stores track ID #1 for specifying track #1, the image size of the main image constituting track #1, rotation information indicating the orientation of the digital camera 10 when the main image was captured, and the creation date and time of track #1. The tkhd box of the trak box for managing track #2 for thumbnail images stores track ID #2 for specifying track #2, and the creation date and time of track #2.

[0184] In the trak box, in addition to the above reference Figure 7 In addition to the tkhd box and mdia box described above, a tref box may be stored. The tref box stores a track ID for specifying another track associated with the track managed by the trak box storing the tref box, information indicating the track content, and the like. Figure 13 In the example, the tref box is provided in the trak box that manages track #2. In addition, the tref box stores information (type=thmb) indicating that another track associated with track #2 is track #1 (track_ID=1) and that data constituting track #2 is a thumbnail image (track #2 is a track of thumbnail images).

[0185] In the mdia box in the trak box, in addition to the above reference Figure 8 In addition to the minf box described above, an hdlr box can also be stored. The hdlr box stores information indicating the type of data constituting the track managed by the trak box storing the hdlr box. Information (pict) indicating that the data constituting track #1 is a picture (frame) is stored in the hdlr box stored in the trak box (stored in the mdia box) of track #1 that manages the main image, and information indicating that the data constituting track #2 is a picture is stored in the hdlr box stored in the trak box of track #2 that manages the thumbnail image.

[0186] minf box as mentioned above Figure 8 As stated.

[0187] In the following description, the Figure 13 An associated sequence file that stores internal data associated with each other and specific information about external data in a regular sequence file.

[0188] Figure 14 is a diagram showing an example of an associated sequence file.

[0189] In the associated sequence file, track #3 of the (elementary) stream (Meta ES) regarding the uuid of specific information of the RAW file as external data is added to the mdat box, and a trak box managing track #3 is added to the moov box.

[0190] Here, track #1 is a time sequence of one or more frames of main images arranged on the timeline, and track #3 is a time sequence of UUIDs of RAW files of corresponding frames of the main images arranged on the timeline.

[0191] The nth uuid (starting from the top) of track #3 is specific information about the RAW file of the frame of the nth main image in track #1. In addition, multiple tracks (data) stored in the mdat box can be synchronously reproduced according to the time information on a timeline. Therefore, track #1 of the main image and track #3 of the uuid (stream) of the RAW file of the corresponding frame of the main image constituting track #1 are stored in the mdat box, so that the frame of the nth main image of track #1 and the uuid of the RAW file of the main image (frame) are associated with each other and stored. In this case, the frame of the main image of track #1 and the uuid of the RAW file of the main image (frame) can be associated with each other through the time information on the timeline.

[0192] Note that the nth UUID of track #3 (starting from the top) is specific information of the RAW file of the nth frame of track #1, and it can be understood that the main image (frame) constituting track #1 and the UUIDs constituting track #3 are associated with each other through the arrangement sequence in the track.

[0193] In the associated sequence file, the trak box managing track #3 is added to the moov box because the track #3 of the RAW file's uuid is added to the mdat box.

[0194] The trak box of track #3 that manages the UUID of the RAW file stores a tkhd box, a tref box, an mdia box, and the like.

[0195] The tkhd box of the trak box managing the track #3 stores the track ID #3 for specifying the track #3, and the creation date and time of the track #3.

[0196] The tref box of the trak box that manages track #3 stores a track ID for specifying another track associated with track #3 managed by the trak box storing the tref box, information indicating the contents of track #3, etc. Since the uuid constituting track #3 is specific information about the RAW file of the main image constituting track #1, and track #3 is associated with track #1, information indicating that the other track associated with track #3 is track #1 (track_ID=1) and that track #3 is a metadata track (specific information in this case) (type=cdsc) is stored in Figure 14 The tref box of the trak box that manages track #3.

[0197] The mdia box that manages the trak box of track #3 stores the hdlr box and the minf box. In the trak box of track #3, information indicating that the data constituting track #3 is metadata (of the main image) is stored in the hdlr box, and the stsc box, stsc box, stsz box, and stco box for track #3 are stored in the minf box.

[0198] <Generation and Reproduction of HEIF Files>

[0199] Figure 15 It is a flowchart outlining an example of the generation process for generating an associated HEIF file.

[0200] In the generation process, in step S11, the file control unit 43 generates a uuid as specific information for the RAW file (RAW image) of the frame of the main image, and then the process moves to step S12.

[0201] In step S12, the file control unit 43 assigns the uuid generated in step S11 to the RAW file (RAW image) of the frame of the main image, and then the process moves to step S13.

[0202] In step S13, the file control unit 43 generates an associated HEIF file in which the uuid of the frame of the main image and the RAW file of the frame that are associated with each other are stored in the HEIF file, and then the generation process ends.

[0203] Figure 16 It is a flowchart outlining an example of the reproduction process for reproducing an associated HEIF file.

[0204] In the reproduction process, in step S21, the file control unit 43, for example, generates a handle list for identifying each frame of the main image stored in the HEIF file stored in the medium 14, and then the process moves to step S22.

[0205] Here, the handle of the frame of the main image includes the file name of the HEIF file in which the frame is stored. The handle of the frame (item) of the main image stored in the collection file also includes the item ID of the frame. The handle of the frame of the main image stored in the sequence file also includes the time information about the frame. Using the handle of the frame of the main image, the frame corresponding to the handle can be uniquely identified (specified).

[0206] Note that the handle of the frame of the main image stored in the sequence file may include the track ID of the track containing the frame and the order of the frame in the track (what is the order of the frame), rather than the time information about the frame.

[0207] The time information associated with each frame is unique, regardless of whether the number of tracks formed by the frames of the main image stored in the sequence file is one or more. Accordingly, using the time information associated with each frame, even if multiple tracks are stored in the sequence file, the frame whose time information is included in the handle can be uniquely specified from the frames constituting each track. Therefore, if the handle of a main image frame contains time information associated with a frame, even if the track ID for the track containing that frame does not exist, the frame corresponding to that time information can be uniquely specified.

[0208] The handle list may be generated for all frames of the main image in the HEIF file stored in the medium 14, or may be generated only for frames that are downscaled under certain conditions, such as frames of a certain creation date and time.

[0209] After generating the handle list, the file control unit 43 accesses the HEIF file by referring to the handle list when necessary.

[0210] In step S22, after waiting for the user to operate the digital camera 10 to display a thumbnail image, for example, the UI control unit 47 requests the file control unit 43 to display the thumbnail image. In response to the thumbnail image display request from the UI control unit 47, the file control unit 43 reads the thumbnail (frame) of the frame of the main image identified by the handle in the handle list from the HEIF file. The file control unit 43 then causes the liquid crystal panel 19 ( Figure 1 ) For example, a list of thumbnail images read from the HEIF file is displayed, and the process moves from step S22 to step S23.

[0211] In step S23, after waiting for the user to select a desired thumbnail image (frame) from a list of thumbnail images, etc., the UI control unit 47 requests the file control unit 43 to display the main image corresponding to the thumbnail image selected by the user. In response to the request for the main image from the UI control unit 47, the file control unit 43 reads the main image from the HEIF file. The file control unit 43 can cause the liquid crystal panel 19 to display the main image read from the HEIF file, if necessary.

[0212] Alternatively, the UI control unit 47 requests the file control unit 43 to supply the UUID of the RAW file of the main image corresponding to the thumbnail image selected by the user. In response to the UUID request from the UI control unit 47, the file control unit 43 reads the UUID from the associated HEIF file. The file control unit 43 can access the RAW file specified by the UUID read from the associated HEIF file when necessary.

[0213] Figure 17 is a flowchart for explaining an example of a reproduction process of reproducing a set file.

[0214] In step S31 , the file control unit 43 acquires the item ID of the reproduction target image which is the image (item) to be reproduced (this ID will hereinafter also be referred to as the reproduction target item ID), and the process moves to step S32 .

[0215] In acquiring the reproduction target item ID, the item ID of the reproduction target image (reproduction target item ID) is acquired, and the reproduction target image is, for example, a main image identified by a handle in the handle list, a thumbnail image of the main image, a thumbnail image selected by the user from the list of thumbnail images (this thumbnail image will also be referred to as the selected thumbnail image hereinafter), a main image of the selected thumbnail image, etc.

[0216] In step S32 , the file control unit 43 reads the reproduction target image based on the reproduction target item ID acquired in step S31 .

[0217] In reading of the reproduction target image, the reproduction target image specified by the reproduction target item ID is read from the set file.

[0218] Figure 18 Is used to explain Figure 17 Flowchart of an example of a process of reading a reproduction target image in step S32.

[0219] In step S41, the file control unit 43 selects the collection file ( Figures 9 to 12 )'s iloc box, and then the process moves to step S42.

[0220] In step S42, the file control unit 43 reads the offset and size associated with the reproduction target item ID retrieved in step S41 in the iloc box, and then the process moves to step S43.

[0221] In step S43, the file control unit 43 reads the reproduction target image stored in the mdat box of the set file according to the offset and size corresponding to the reproduction target item ID, and the process ends.

[0222] Figure 19 Is used to explain Figure 17 Flowchart of a first example of the process of acquiring the reproduction target item ID in step S31.

[0223] Right now, Figure 19 An example is shown in which the item ID of a thumbnail image as a reproduction target image is acquired in a case where the reproduction target image is a thumbnail image.

[0224] Note that, in Figure 19In the example, the file control unit 43 recognizes the item ID of the main image which is the thumbnail image of the reproduction target image based on its handle.

[0225] In step S51, the file control unit 43 selects the collection file ( Figures 9 to 12 ) retrieves the thmb box whose reference source matches the item ID of the main image from the thmb box in the iref box, and then the process moves to step S52.

[0226] In step S52, the file control unit 43 reads the reference destination in the thmb box that is retrieved in step S51 and has the reference source that matches the item ID of the main image, as the item ID of the thumbnail image that is the reproduction target image, and the process ends.

[0227] Figure 20 Is used to explain Figure 17 Flowchart of a second example of the process of acquiring the reproduction target item ID in step S31 in .

[0228] Right now, Figure 20 An example is shown in which the item ID of the main image, which is the reproduction target image, is acquired in a case where the reproduction target image is the main image.

[0229] Note that, in Figure 20 In the example, the user selects a thumbnail image (selected thumbnail image) from the list of thumbnail images, and the file control unit 43 recognizes, for example, the item ID of the selected thumbnail image.

[0230] In step S61, the file control unit 43 selects the collection file ( Figures 9 to 12 ) retrieves a thmb box whose reference destination matches the item ID of the selected thumbnail image from the thmb boxes in the iref box, and then the process moves to step S62.

[0231] In step S62, the file control unit 43 reads the reference source in the thmb box retrieved in step S61 and having the reference destination matching the item ID of the selected thumbnail image as the item ID of the main image as the reproduction target image, and the process ends.

[0232] Figure 21 Is used to explain Figure 10 Flowchart of an example of a process of acquiring the uuid of specific information of a RAW file as a predetermined main image in the first associated set file shown in .

[0233] Note that, in Figure 21 In the example, the file control unit 43 identifies the item ID of the predetermined main image from the handle list or the like.

[0234] In step S71, the file control unit 43 selects the first associated set file ( Figure 10 ) is retrieved from the associated information in the associated information storage box of , and then the process moves to step S72.

[0235] In step S72 , the file control unit 43 reads the uuid associated with the item ID of the predetermined main image retrieved in step S71 in the association information, and the process ends.

[0236] Through the uuid read in the above-described manner, the file control unit 43 can access the RAW file of the predetermined master image.

[0237] Figure 22 Is used to explain Figure 11 Flowchart of an example of a process of acquiring the uuid of specific information of a RAW file as a predetermined main image in the second associated set file shown in .

[0238] Note that, in Figure 22 In the example, the file control unit 43 identifies the item ID of the predetermined main image from the handle list or the like.

[0239] In step S81, the file control unit 43 selects the second association set file ( Figure 11 ) retrieves the infe box of the item type IDIF indicating that the item is related information from the infe box in the iinf box of the item, and then the process moves to step S82.

[0240] In step S82, the file control unit 43 reads the item ID as the associated information of the item from the infe box of the item type IDIF retrieved in step S81, and the process moves to step S83.

[0241] In step S83, the file control unit 43 retrieves the item ID of the association information read in step S82 from the iloc box in the second association set file, and the process moves to step S84.

[0242] In step S84 , the file control unit 43 reads the offset and size associated with the item ID of the association information retrieved in step S83 in the iloc box, and then the process moves to step S85 .

[0243] In step S85, the file control unit 43 reads the association information as an item stored in the mdat box in the second association set file based on the offset and size associated with the item ID of the association information read in step S84, and the process moves to step S86.

[0244] In step S86 , the file control unit 43 retrieves the item ID of the predetermined main image from the association information read in step S85 , and the process moves to step S87 .

[0245] In step S87 , the file control unit 43 reads the uuid associated with the item ID of the predetermined main image retrieved in step S86 in the association information, and the process ends.

[0246] Through the uuid read in the above-described manner, the file control unit 43 can access the RAW file of the predetermined master image.

[0247] Figure 23 Is used to explain Figure 12 Flowchart of an example of a process of acquiring a uuid as specific information about a RAW file of a predetermined main image in a third association set file shown in .

[0248] Note that, in Figure 23 In the example, the file control unit 43 identifies the item ID of the predetermined main image from the handle list or the like.

[0249] In step S91, the file control unit 43 selects the third associated set file ( Figure 12 ) retrieves the cdsc box having a reference source matching the item ID of the predetermined main image from the cdsc box in the iref box of , and then the process moves to step S92.

[0250] In step S92, the file control unit 43 reads the reference destination within the cdsc box of the reference source retrieved in step S91 and having the item ID matching the item ID of the predetermined main image as the item ID of the specific information about the RAW file of the predetermined main image as the item, and then the process moves to step S93.

[0251] In step S93, the file control unit 43 retrieves the item ID as the specific information of the item read in step S92 from the iloc box in the third association set file, and the process moves to step S94.

[0252] In step S94 , the file control unit 43 reads the offset and size associated with the item ID of the specific information retrieved in step S93 in the iloc box, and then the process moves to step S95 .

[0253] In step S95, the file control unit 43 reads the uuid of the specific information of the RAW file regarding the predetermined main image stored in the mdat box in the third associated set file according to the offset and size associated with the item ID of the specific information read in step S94, and the process ends.

[0254] Through the uuid read in the above-described manner, the file control unit 43 can access the RAW file of the predetermined master image.

[0255] Figure 24 is a flowchart for explaining an example of a process of acquiring a list of item IDs of main images from a collection file.

[0256] For example, in the case where a handle list or the like is to be generated, a process of acquiring a list of item IDs of main images from a collection file is performed.

[0257] In step S101, the file control unit 43 selects the collection file ( Figures 9 to 12 ) reads the item ID from all infe boxes within the iinf box of the main image, and registers the item ID in a list of item IDs of the main image (this list is also referred to as the main image list hereinafter). The process then moves to step S102.

[0258] In step S102, the file control unit 43 reads the item IDs serving as reference destinations from all boxes within the iref box of the collection file, and eliminates the read item IDs from the main image list. The process then ends.

[0259] After the above process, the item ID registered in the main image list is the item ID of the main image.

[0260] Figure 25 is a flowchart for explaining an example of a process of reproducing thumbnail images of (frames of) a main image corresponding to predetermined time information from a sequence file.

[0261] Note that, in Figure 25 In the example, the file control unit 43 identifies time information on predetermined main images (or their order) from a handle list or the like.

[0262] In step S111, from the sequence file ( Figure 13 and 14 ), the file control unit 43 retrieves a trak box in which information indicating that data constituting the track is a thumbnail image is stored in a tref box, or a trak box in which the type in the tref box is "thmb", as the trak box for managing the track of the thumbnail image of the main image corresponding to the predetermined time information. The process then moves to step S112.

[0263] In step S112, the file control unit 43 reads the track ID in the tkhd box in the trak box retrieved in step S111 as the track ID of the track of the thumbnail image of the main image corresponding to the predetermined time information. The process then moves to step S113.

[0264] In step S113, the file control unit 43 reproduces the track having the track ID read in step S112, and acquires (the frame of) the thumbnail image corresponding to the predetermined time information (or order) from the track as the thumbnail image of the main image corresponding to the predetermined time information. The process then ends.

[0265] Note that the process of reproducing a track of images stored in a sequence file is similar to the process of reproducing video images in an MP4 file.

[0266] Figure 26 is a flowchart for explaining an example of a process of acquiring a uuid as specific information about a RAW file of (a frame of) a predetermined main image from an associated sequence file.

[0267] Note that, in Figure 26 In the example, the file control unit 43 identifies time information on predetermined main images (or their order) from a handle list or the like.

[0268] In step S121, from the associated sequence file ( Figure 14 ), the file control unit 43 retrieves a trak box in which information indicating that data constituting a track is specific information is stored in a tref box, or a trak box whose type in the tref box is "cdsc", as a trak box for managing a track of specific information. The process then moves to step S122.

[0269] In step S122, the file control unit 43 reads the track ID in the tkhd box in the trak box retrieved in step S121 as the track ID of the track of the specific information. The process then moves to step S123.

[0270] In step S123, the file control unit 43 obtains the uuid, which is specific information corresponding to the time information (or the order thereof) of the predetermined main image, from the tracks having the track ID read in step S122 as the uuid of the RAW file of the predetermined main image. The process then ends.

[0271] Through the uuid acquired in the above-described manner, the file control unit 43 can access the RAW file of the predetermined master image.

[0272] As described above, the file control unit 43 generates and reproduces an associated HEIF file in which the main image in the HEIF file and specific information for specifying external data outside the HEIF file are associated with each other and stored in the HEIF file in accordance with HEIF. Therefore, the main image stored in the HEIF file can be associated with external data outside the HEIF file.

[0273] In addition, when UUIDs are used as specific information, even if the file name of the external data is changed, the association between the main image in the HEIF file and the external data after the file name is changed can be maintained through the UUID.

[0274] <Storage of specific information assigned to external data>

[0275] Figure 27 : is a diagram showing an example of storing uuid in a RAW file when a RAW file of a main image is used as (a file storing) external data and an associated set file is generated.

[0276] Note that, in Figure 27 In the example, the first associated set file is used as the associated set file.

[0277] RAW files have an area called a marker comment (MakerNote) as part of an area that stores information accompanying Exif as metadata.

[0278] For example, the file control unit 43 may store the uuid assigned to the RAW file (RAW image) in the mark comment of the RAW file.

[0279] exist Figure 27 In the example, main images Item#1, Item#2, Item#3, and Item#4 are stored as four items in the association set file, and RAW files #1, #2, #3, and #4 are generated in which the RAW images of main images Item#1, Item#2, Item#3, and Item#4 are stored. In addition, UUID#i is assigned to RAW file #i (RAW image), and as association information for associating main image Item#i with UUID#i of RAW file #i of main image Item#i, association information for associating Item ID#i for specifying main image Item#i with UUID#i of RAW file #i associated with main image Item#i is stored in the association information storage box.

[0280] Figure 28 : is a diagram showing an example of storing uuid in a RAW file when a RAW file of a main image is adopted as external data and a related sequence file is generated.

[0281] In the case of generating an associated sequence file, the file control unit 43 may also store the uuid assigned to the RAW file in the tag annotation of the RAW file, as described above with reference to Figure 27 The same is true for the case of generating the associated collection file.

[0282] exist Figure 28In the example, track #1 formed of main images #1, #2, #3, and #4 as four frames is stored in the associated sequence file, and RAW files #1, #2, #3, and #4 are generated in which the RAW images of main images #1, #2, #3, and #4 are stored. In addition, UUID #i is assigned to RAW file #i, and track #3 in which UUID #i of RAW file #i is placed so as to have the same time information as that of main image #i corresponding to RAW file #i (RAW image) is stored in the associated sequence file.

[0283] As described above, the UUID #i of RAW file #i is placed with the same time information as the main image #i corresponding to RAW file #i, forming track #3. Therefore, the i-th main image #i in track #1 and the i-th UUID #i in track #3, which is the UUID #i of RAW file #i of main image #i, are associated with each other and stored in the associated sequence file.

[0284] While the above example uses the main image's RAW file (RAW image) as external data, other data may also be used. For example, sound (voice) recorded along with the main image may be used as external data. Files storing sound, such as WAV files and MP4 files, may be used. In the following description, a WAV file is used as an example of a file storing sound.

[0285] Figure 29 : is a diagram showing an example of storing uuid in a WAV file when a WAV file of a main image is used as (a file storing) external data and an associated set file is generated.

[0286] Note that, in Figure 29 In the example, the first associated set file is used as the associated set file.

[0287] A WAV file has an area called a list chunk as part of an area in which metadata is written.

[0288] For example, the file control unit 43 may store the uuid assigned to the WAV file (audio) in the list block of the WAV file.

[0289] exist Figure 29In the example, main image Item#1, Item#2, Item#3, and Item#4 are stored as four items in the association set file, and WAV files #1, #2, #3, and #4 are generated for the main image Item#1, Item#2, Item#3, and Item#4. Furthermore, UUID#i is assigned to WAV file #i (sound), and as association information for associating main image Item#i with UUID#i of WAV file #i of main image Item#i, association information for associating Item ID#i specifying main image Item#i with UUID#i of WAV file #i associated with main image Item#i is stored in the association information storage box.

[0290] Figure 30 : is a diagram showing an example of storing uuid in a WAV file in the case of adopting a WAV file of a main image as external data and generating an associated sequence file.

[0291] In the case of generating an associated sequence file, the file control unit 43 may also store the uuid assigned to the WAV file in the list block of the WAV file, as described above with respect to Figure 29 The same is true for the case of generating the associated collection file.

[0292] exist Figure 30 In the example, track #1 formed of four frames of main images #1, #2, #3, and #4 is stored in the associated sequence file, and WAV files #1, #2, #3, and #4 of the main images #1, #2, #3, and #4 are generated. Furthermore, UUID #i is assigned to WAV file #i, and track #3, in which UUID #i of WAV file #i is placed so as to have the same time information as that of main image #i corresponding to WAV file #i, is stored in the associated sequence file.

[0293] As described above, the UUID #i of WAV file #i is placed with the same time information as the main image #i corresponding to WAV file #i, forming track #3. Therefore, the i-th main image #i in track #1 and the i-th UUID #i of WAV file #i as the main image #i in track #3 are associated with each other and stored in the associated sequence file.

[0294] Note that the present technology can be applied not only to HEIF files but also to, for example, ISO base media files, MP4 files, Miaf files, etc., which have a box structure different from that of HEIF files.

[0295] In addition to this, for example, the present technology can also be applied to a file or the like that does not have a box structure and stores an image (main image) and other images having a lower resolution than the image.

[0296] In addition, the present technology can be applied not only to the case where external data is associated with the main image in the HEIF file, but also to the case where external data is associated with the screen nail image or thumbnail image in the HEIF file.

[0297] In addition to this, for example, the present technology can also be applied to a case where external data is associated with internal data that is not an image like a main image in a HEIF file.

[0298] <First Embodiment of Image Processing System to Which the Present Technology is Applied>

[0299] Figure 31 is a block diagram showing an example configuration of a first embodiment of an image processing system to which the present technology is applied.

[0300] exist Figure 31 In the example, the image processing system 100 includes Figure 1 The digital camera 10 and the external computer 101, such as a cloud computer, shown in FIG. 1 can communicate with each other via a network such as the Internet.

[0301] Here, there are cases where the user wishes to restrict viewing of images captured by the digital camera 10 to protect so-called copyrights or to protect the portrait rights of people appearing in the images, or the like.

[0302] Therefore, in the image processing system 100 , an encrypted HEIF file using an associated HEIF file is generated by the digital camera 10 , so that viewing of an image captured by the digital camera 10 can be restricted.

[0303] Note that in the following description, for ease of explanation, viewing will be restricted to the primary image stored in the HEIF file. However, with this technology, viewing can be restricted to any desired media data other than the primary image stored in the HEIF file. This technology can also be applied to (files that conform to) ISO base media files other than HEIF files, and further, can be applied to any appropriate file other than ISO base media files.

[0304] In digital camera 10( Figure 1 ), the file control unit 43 can generate a HEIF file as an encrypted HEIF file, in which an encrypted image obtained by encrypting a main image stored in the HEIF file with a first encryption key and an encrypted encryption key obtained by encrypting the first encryption key with a second encryption key are associated with each other and stored.

[0305] When associating the encrypted image stored in the encrypted HEIF file with the encrypted encryption key obtained by encrypting the first encryption key used in the encryption for obtaining the encrypted image, the file control unit 43 can use the association between the main image stored in the associated HEIF file and the specific information about the external data. That is, the file control unit 43 can associate the encrypted image stored in the encrypted HEIF file with the encrypted encryption key in a manner similar to the association between the main image stored in the associated HEIF file and the specific information about the external data.

[0306] The file control unit 43 may also decrypt the encrypted encryption key of the encrypted HEIF file into a first encryption key, and decrypt the encrypted image into the original main image using the first encryption key obtained by the decryption.

[0307] The digital camera 10 can request the external computer 101 to supply the first encryption key and the second encryption key. When there is a request for the first encryption key and the second encryption key from the digital camera 10, the external computer 101 generates the first encryption key and the second encryption key and supplies (transmits) the first encryption key and the second encryption key to the digital camera 10. Note that the first encryption key and the second encryption key can be generated by the digital camera 10. Alternatively, one of the first encryption key and the second encryption key can be generated by the digital camera 10, while the other can be generated by the external computer 101.

[0308] Any appropriate encryption method can be used as the first encryption key and the second encryption key. Furthermore, the first encryption key and the second encryption key can use the same encryption method or different encryption methods.

[0309] In this embodiment, for example, a common key of a common key encryption system is used as a first encryption key, and a public key of a public key encryption system is used as a second encryption key.

[0310] Since the process of generating a common key has a smaller load than the process of generating a public key and a private key for public key encryption, the common key can be easily generated in the digital camera 10. In addition, using the common key, an image with a relatively large amount of data can be encrypted or decrypted in a shorter time than using a public key and a private key.

[0311] Furthermore, by adopting a public key of a public key encryption system as the second encryption key, it is possible to improve the confidentiality of the common key encrypted with the public key.

[0312] In this embodiment, a common key is generated by the digital camera 10 , and a public key and a private key are generated and managed by the external computer 101 .

[0313] As described above, in the digital camera 10, an encrypted HEIF file is generated in which an encrypted image obtained by encrypting a main image stored in a HEIF file with a common key as a first encryption key and an encrypted encryption key obtained by encrypting the common key with a public key as a second encryption key are associated with each other and stored. Therefore, it is possible to restrict viewing of the main image so that only a user who can obtain (acquire) a private key corresponding to the public key used in encryption for obtaining the encrypted encryption key (or the common key before being encrypted into the encrypted encryption key) can view the main image.

[0314] In the following description, for example, the use of associated HEIF files is explained Figure 12 The encrypted HEIF file of the third associated set file (the association in) shown in . However, as the encrypted HEIF file, a HEIF file using associated HEIF files other than the third associated set file (the first associated set file, the second associated set file, or the associated sequence file) can be generated.

[0315] Note that in the following description of encrypted HEIF files, the common parts with the already described HEIF files (including associated HEIF files) will not be explained.

[0316] <Encrypted HEIF file>

[0317] Figure 32 is a diagram illustrating an example of a first encrypted HEIF file.

[0318] exist Figure 32 , items Item#1, Item#2, Item#201, and Item#202 are stored in the mdat box. Item Item#1 is an encrypted image obtained by encrypting the main image 1 with a common key (by common key encryption), and item Item#2 is an encrypted image obtained by encrypting the main image 2 with a common key. Different common keys can be used to encrypt different main images. For example, the common key used in the encryption to obtain the encrypted image as item Item#1 (this encrypted image will also be written as encrypted image Item#1 hereinafter) is a different common key from the common key used in the encryption to obtain the encrypted image Item#2. Note that, if necessary, the same common key can be used to encrypt different main images.

[0319] Item #201 is an encrypted encryption key obtained by encrypting the shared key used in the decryption of main image 1 with a public key (by public key encryption), and item #202 is an encrypted encryption key obtained by encrypting the shared key used in the encryption of main image 2 with a public key. Different public keys (corresponding to private keys) can be used to encrypt different shared keys. For example, the public key used for encryption to obtain the encrypted encryption key as item #201 (this encrypted encryption key will also be written as encrypted encryption key Item #201 hereinafter) is a different public key from the public key used to obtain the encrypted encryption key Item #202. Note that, if necessary, the same public key can be used to encrypt different shared keys.

[0320] exist Figure 32 In FIG, the white lock indicator is encrypted with a shared key, while the shaded lock indicator is encrypted with a public key. The same applies to the drawings described later.

[0321] In the first encrypted HEIF file, association information for associating the encrypted image Item#i stored in the mdat box with the encrypted encryption key Item#200+i obtained by encrypting the common key used in the encryption for obtaining the encrypted image Item#i is stored in the meta box so that the encrypted image Item#i and the encrypted encryption key Item#200+i are associated with each other, as shown in the third association set file ( Figure 12 ), where, for example, association information for associating a main image stored in an mdat box with specific information is stored in a meta box so that the main image and the specific information are associated with each other.

[0322] The association information for associating the encrypted image Item#i with the encrypted encryption key Item#200+i is information associating the item ID#i of the encrypted image Item#i with the item ID of the encrypted key Item#200+i, and is stored in the cdsc box in the iref box in the meta box, as in the third association set file.

[0323] exist Figure 32In the example, a cdsc box storing associated information in which item ID#1 of the encrypted image Item#1 and item ID#201 of the encrypted encryption key Item#201 obtained by encrypting the shared key used in the encryption of the encrypted image Item#1 with a public key are associated with each other as a reference source and a reference destination is stored in the iref box, and a cdsc box storing associated information in which item ID#2 of the encrypted image Item#2 and item ID#202 of the encrypted encryption key Item#202 obtained by encrypting the shared key used in the encryption of the encrypted image Item#2 with a public key are associated with each other as a reference source and a reference destination is stored in the iref box.

[0324] As with encrypted image Item#i stored in the first encrypted HEIF file, if a private key corresponding to encrypted image Item#i is available, the private key corresponding to the public key used to encrypt the shared key used to encrypt encrypted image Item#i, the file control unit 43 can decrypt the encrypted encryption key Item#200+i associated with encrypted image Item#i using the private key to obtain the shared key, and decrypt the encrypted image Item#i using the shared key obtained by decryption to return it to the original main image i. Therefore, viewing of the main image i can be limited to users who can obtain the private key corresponding to encrypted image Item#i.

[0325] In the infe box of the encrypted encryption key Item#200+i in the first encrypted HEIF file, the item type (Item Type) of the encrypted encryption key Item#200+i is set to ECKI (Encryption key for Item), indicating, for example, an encryption key (in this case, a common key). The attribute value indicating the item type of the encryption key is not necessarily ECKI. When decrypting the encrypted image Item#i, the file control unit 43 specifies the item ID of the reference destination in the cdsc box with the reference source indicating the item ID#i of the encrypted image Item#i, as the item ID#200+i of the encrypted encryption key Item#200+i corresponding to the encrypted image Item#i, or the item ID of the encrypted encryption key Item#200+i obtained by encrypting the common key used in encrypting the encrypted image Item#i. Based on the item ID#200+i, the file control unit 43 obtains (reads) the encrypted encryption key Item#200+i having the item ID#200+i from the mdat box and performs decryption. Whether or not the item specified by the item ID of the reference destination in the cdsc box is an encrypted encryption key can be identified (determined) based on the item type set in the infe box of the item specified by the item ID.

[0326] Figure 33 is a diagram illustrating an example of a second encrypted HEIF file.

[0327] exist Figure 33 In the example, the encrypted images Item#1 and Item#2, and the encrypted encryption keys Item#201 and Item#202 are stored in the mdat box, as shown in FIG. Figure 32 In addition, Figure 33 In the example, plaintext shared keys Item #301 and Item #302 are stored in the mdat box. The encrypted encryption key Item #200+i is obtained by encrypting the plaintext shared key Item #300+i as an item with the private key. Here, plaintext refers to data in an unencrypted state.

[0328] In the second encrypted HEIF file, the plaintext common key Item#300+i is associated with the encrypted image Item#i (corresponding main image i), as is the encrypted encryption key Item#200+i corresponding to the common key Item#300+i or the encrypted encryption key Item#200+i obtained by encrypting the common key Item#300+i.

[0329] exist Figure 33 In addition to Figure 32 In addition to the cdsc box shown in the figure, a cdsc box is stored in the iref box. This cdsc box stores information indicating that the item ID #1 of the encrypted image Item #1 and the item ID #301 of the plaintext common key Item #301 corresponding to the encrypted image Item #1 (the common key Item #301 used in the encryption to obtain the encrypted image Item #1) are associated with each other as a reference source and a reference destination. Furthermore, the iref box stores a cdsc box storing information indicating that the item ID #2 of the encrypted image Item #2 and the item ID #302 of the plaintext common key Item #302 corresponding to the encrypted image Item #2 are associated with each other as a reference source and a reference destination.

[0330] In the second encrypted HEIF file, the plaintext common key Item #300+i stored in the mdat box can be deleted based on user operations. For example, if viewing of the main image i obtained by decrypting the encrypted image Item #i is limited to a specific user (a user who can obtain the private key corresponding to the encrypted image Item #i), the file control unit 43 deletes the plaintext common key Item #300+i stored in the mdat box based on the user's operation. With this arrangement, the encrypted image Item #i cannot be decrypted unless the private key corresponding to the encrypted image Item #i is successfully obtained.

[0331] That is, if the plaintext common key Item#300+i is not stored in the mdat box, the encrypted image Item#i cannot be decrypted unless the private key corresponding to the encrypted image Item#i is successfully obtained. If the private key corresponding to the encrypted image Item#i can be obtained, the file control unit 43 can use the corresponding private key to decrypt the encrypted encryption key Item#200+i associated with the encrypted image Item#i stored in the second encrypted HEIF file into the common key, and use the common key obtained by decryption to decrypt the encrypted image Item#i into the original main image i, just like the first encrypted HEIF file.

[0332] On the other hand, when the plaintext common key Item#300+i is stored in the mdat box, the file control unit 43 can decrypt the encrypted image Item#i into the original main image i using the plaintext common key Item#300+i associated with the encrypted image Item#i stored in the second encrypted HEIF file.

[0333] Accordingly, when the plaintext shared key Item#300+i is stored in the mdat box in the second encrypted HEIF file, the encrypted image Item#i can be decrypted into the main image i, and any user can view the main image i without having to obtain the private key corresponding to the encrypted image Item#i.

[0334] In view of the above, using the second encrypted HEIF file, by leaving the plaintext shared key associated with the main image (the encrypted image obtained by encryption) in the mdat box, the user can allow any user to view the main image that does not need to be restricted. In addition, for the main image to be restricted, the user can restrict any user from viewing by deleting the plaintext shared key associated with the main image from the mdat box.

[0335] Note that in the infe box of the plaintext common key Item#300+i in the second encrypted HEIF file, the item type (Item Type) of the plaintext common key Item#300+i is set to ECKI indicating the encryption key, as in the infe box of the encrypted encryption key Item#200+i.

[0336] Figure 34 is a diagram illustrating an example of a third encrypted HEIF file.

[0337] exist Figure 34 In the example, the encrypted images Item#1 and Item#2, and the encrypted encryption keys Item#201 and Item#202 are stored in the mdat box, as shown in FIG. Figure 32 As shown in .

[0338] However, in the third encrypted HEIF file, the encrypted encryption key Item#200+i is an encryption key containing specific information obtained by encrypting the uuid into specific information for specifying external data associated with the main image i and the common key used when encrypting the main image i with the private key, and will also be written as the encryption key Item#200+i containing specific information below.

[0339] exist Figure 34 In the third encrypted HEIF file, encrypted image Item#1 obtained by encrypting main image 1 with the common key, UUID#1 assigned to (the RAW data stored in) RAW file RAW#1 associated with main image 1, and encryption key Item#201 containing specific information obtained by encrypting the common key used in encryption of main image 1 with the private key are associated with each other through association information stored in the cdsc box in the iref box and stored in the mdat box. Furthermore, encrypted image Item#2 obtained by encrypting main image 2 with the common key, UUID#2 assigned to WAV file WAV#2 associated with main image 2, and encryption key Item#202 containing specific information obtained by encrypting the common key used in encryption of main image 2 with the private key are associated with each other through association information stored in the cdsc box in the iref box and stored in the mdat box.

[0340] For the third encrypted HEIF file, if the private key corresponding to the encrypted image Item#i stored in the third encrypted HEIF file is available, the file control unit 43 can use the private key to decrypt the encryption key Item#200+i containing the specific information associated with the encrypted image Item#i into the common key and UUID#i. Then, using the common key obtained by decryption, the encrypted image Item#i is decrypted back to the original primary image i, just as in the first encrypted HEIF file. Therefore, viewing of the primary image i can be restricted to users who can obtain the private key corresponding to the encrypted image Item#i.

[0341] Furthermore, using the third encrypted HEIF file, if the private key corresponding to encrypted image Item#i is available, access to external data specified by UUID#i, obtained by decrypting encrypted key Item#200+i containing specific information with the private key, or external data associated with primary image i, is possible. Therefore, access to external data associated with primary image i can be restricted to users who have access to the private key corresponding to encrypted image Item#i. Furthermore, forgery of UUIDs as specific information regarding external data associated with primary image i can be prevented.

[0342] Figure 35 is a diagram illustrating an example of a fourth encrypted HEIF file.

[0343] exist Figure 35 The fourth encrypted HEIF file is designed in a similar way to the third encrypted HEIF file ( Figure 34 ).

[0344] However, for the fourth encrypted HEIF file, the external data associated with the main image i is encrypted with the common key used in the encryption of the main image i. Figure 35 In the example, RAW file ARW#1 (the RAW data stored therein) is encrypted using the shared key used to encrypt the main image 1 associated with RAW file ARW#1. In addition, WAV file WAV#2 (the sound stored therein) is encrypted using the shared key used to encrypt the main image 2 associated with WAV file WAV#2.

[0345] For the fourth encrypted HEIF file, if the private key corresponding to the encrypted image Item#i stored in the fourth encrypted HEIF file is available, the file control unit 43 can use the private key to decrypt the encryption key Item#200+i containing the specific information associated with the encrypted image Item#i into the common key and UUID#i. The encrypted image Item#i can then be decrypted into the original primary image i using the common key obtained by decryption, similar to the third encrypted HEIF file. Therefore, viewing of the primary image i can be restricted to users who can obtain the private key corresponding to the encrypted image Item#i.

[0346] Furthermore, for the fourth encrypted HEIF file, if the private key corresponding to encrypted image Item#i is available, the encrypted external data specified by UUID#i, obtained by decrypting encryption key Item#200+i containing specific information with the private key, or the encrypted external data associated with primary image i, can be accessed. Furthermore, the encrypted external data can be decrypted back to the original external data using the shared key obtained by decrypting encryption key Item#200+i containing specific information. Therefore, viewing of external data associated with primary image i can be restricted to users who have access to the private key corresponding to encrypted image Item#i. Furthermore, the forgery of UUIDs as specific information regarding external data associated with primary image i can be prevented.

[0347] Figure 36 is a diagram illustrating an example of a fifth encrypted HEIF file.

[0348] exist Figure 36 In the example, the encrypted images Item#1 and Item#2, and the encryption keys Item#201 and Item#202 containing specific information are stored in the mdat box. Figure 34 In addition, Figure 36 In the mdat box, a set of a plaintext common key and Item#301 of UUID#1 as specific information (hereinafter also referred to as a common key set) and a common key set of a plaintext common key and Item#302 of UUID#2 as specific information are stored.

[0349] The common key forming the common key set Item #300+i is the common key used to encrypt the main image i. Furthermore, the UUID #i forming the common key set Item #300+i is a UUID used to specify specific information about external data associated with the main image i (encrypted image Item #i obtained by encryption). The encryption key Item #200+i containing this specific information is obtained by encrypting the common key set Item #300+i using the private key.

[0350] In the fifth encrypted HEIF file, the plaintext common key and common key set Item#300+i of UUID#i are associated with the encrypted image Item#i and stored, as is the encryption key Item#200+i containing specific information obtained by encrypting the common key set Item#300+i.

[0351] exist Figure 36 In addition to Figure 34 In addition to the cdsc box shown in FIG, the iref box also stores a cdsc box storing association information in which Item ID#1 of the encrypted image Item#1 and Item ID#301 of the common key set Item#301 are associated with each other as a reference source and a reference target. Furthermore, the iref box also stores a cdsc box storing association information in which Item ID#2 of the encrypted image Item#2 and Item ID#302 of the common key set Item#302 are associated with each other as a reference source and a reference target.

[0352] In the fifth encrypted HEIF file, the (plaintext) shared key set Item#300+i stored in the mdat box can be deleted according to the user's operation. For example, in the case where viewing of the main image i obtained by decrypting the encrypted image Item#i is limited to a specific user, the file control unit 43 deletes the shared key set Item#300+i stored in the mdat box according to the user's operation. With this arrangement, the encrypted image Item#i cannot be decrypted unless the private key corresponding to the encrypted image Item#i is successfully obtained. In addition, unless the private key corresponding to the encrypted image Item#i is successfully obtained, the external data associated with the encrypted image Item#i (the corresponding main image i) cannot be accessed.

[0353] That is, in the case where the common key set Item#300+i is not stored in the mdat box, the encrypted image Item#i cannot be decrypted, and the external data associated with the encrypted image Item#i cannot be accessed unless the private key corresponding to the encrypted image Item#i is successfully obtained. In the case where the private key corresponding to the encrypted image Item#i can be obtained, the file control unit 43 can use the corresponding private key to decrypt the encryption key Item#200+i containing specific information associated with the encrypted image Item#i stored in the fifth encrypted HEIF file into the common key and UUID#i, and use the common key obtained by decryption to decrypt the encrypted image Item#i into the original main image i, just like the third encrypted HEIF file ( Figure 34). In addition, external data associated with the encrypted image Item#i (in this case, the RAW file ARW#1 or the WAV file WAV#2) can be accessed based on the UUID#i obtained by decrypting the encryption key Item#200+i containing specific information.

[0354] On the other hand, if the shared key set Item #300+i is stored in the mdat box, the file control unit 43 can decrypt the encrypted image Item #i into the original main image i using the shared key forming the shared key set Item #300+i associated with the encrypted image Item #i stored in the fifth encrypted HEIF file. Furthermore, the file control unit 43 can access external data associated with the encrypted image Item #i based on the UUID #i forming the shared key set Item #300+i associated with the encrypted image Item #i stored in the fifth encrypted HEIF file.

[0355] Accordingly, when the (plaintext) shared key set Item#300+i is stored in the mdat box of the fifth encrypted HEIF file, any user can view the main image i and access external data associated with the main image i, regardless of obtaining the private key corresponding to the encrypted image Item#i.

[0356] In view of the above, with the fifth encrypted HEIF file, the user can allow any appropriate user to view the main image that does not require viewing restrictions by leaving the shared key set associated with the main image in the mdat box. In addition, it is possible to allow any appropriate user to access external data associated with such a main image.

[0357] Simultaneously, for the main image that will be limited to be checked, the user can limit any user to check by deleting the shared key set associated with this main image from the mdat box.In addition, it is possible to limit any user to access the external data associated with this main image.

[0358] Note that in the infe box of the common key set Item#300+i (infe box of item ID#300+i) in the fifth encrypted HEIF file, the item type (Item Type) of the common key set Item#300+i is set to ECKI indicating the encryption key, as in the infe box of the encryption key Item#200+i containing specific information.

[0359] Figure 37 is a diagram illustrating an example of a sixth encrypted HEIF file.

[0360] exist Figure 37The sixth encrypted HEIF file is designed in a similar way to the fifth encrypted HEIF file ( Figure 36 ).

[0361] However, for the sixth encrypted HEIF file, the external data associated with the main image i is encrypted with the common key used in the encryption of the main image i, just like the fourth encrypted HEIF file ( Figure 35 ).exist Figure 37 , the RAW file ARW#1 is encrypted using the shared key used in encrypting the main image 1 associated with the RAW file ARW#1, and the WAV file WAV#2 is encrypted using the shared key used in encrypting the main image 2 associated with the WAV file WAV#2.

[0362] For the sixth encrypted HEIF file, if the shared key set Item #300+i is not deleted and remains stored in the mdat box, the file control unit 43 can decrypt the encrypted image Item #i into the original primary image i using the shared key forming the shared key set Item #300+i associated with the encrypted image Item #i stored in the sixth encrypted HEIF file, similar to the fifth encrypted HEIF file. Furthermore, the file control unit 43 can access the encrypted external data associated with the encrypted image Item #i using the UUID #i forming the shared key set Item #300+i associated with the encrypted image Item #i stored in the sixth encrypted HEIF file, and decrypt the encrypted external data into the original external data using the shared key forming the shared key set Item #300+i.

[0363] Accordingly, where the (plaintext) shared key set Item#300+i is stored in the mdat box of the sixth encrypted HEIF file, any appropriate user can view the main image i and, for viewing, etc., access external data associated with the main image i, without having to obtain the private key corresponding to the encrypted image Item#i.

[0364] At the same time, for a primary image i whose viewing is restricted, the user can restrict viewing by any user by deleting the shared key set Item #300+i associated with primary image i from the mdat box. Furthermore, access and viewing of external data associated with this primary image i can be restricted for any user. Furthermore, forgery of a uuid as specific information about external data associated with primary image i can be prevented.

[0365] Furthermore, for the sixth encrypted HEIF file, even if the private key corresponding to the encrypted image Item #i obtained by encrypting the main image i is available after the common key set Item #300+i associated with the main image i is deleted from the mdat box, the file control unit 43 can decrypt the encryption key Item #200+i containing specific information associated with the encrypted image Item #i using the private key into the common key and UUID #i, and decrypt the encrypted image Item #i using the common key obtained by decryption to return it to the original main image i. Therefore, viewing of the main image i can be restricted to users who can obtain the private key corresponding to the encrypted image Item #i.

[0366] Furthermore, with the sixth encrypted HEIF file, if the private key corresponding to encrypted image Item #i is available, it is possible to access the encrypted external data specified by UUID #i, obtained by decrypting encryption key Item #200+i containing specific information with the private key, or the encrypted external data associated with main image i, and further decrypt the encrypted external data back to the original external data using the shared key obtained by decrypting encryption key Item #200+i containing specific information. Therefore, access to external data associated with main image i can be restricted to users who can obtain the private key corresponding to encrypted image Item #i. Furthermore, the forgery of UUIDs as specific information regarding external data associated with main image i can be prevented.

[0367] Figure 38 is a flowchart for explaining an example of pre-processing to be performed in the case where a first encrypted HEIF file is handled by the image processing system 100 .

[0368] The pre-processing is performed, for example, immediately after the manufacture of the digital camera 10 , before imaging by the digital camera 10 , or the like.

[0369] In pre-processing, in step S211 , the digital camera 10 of the image processing system 100 transmits a public key request to the external computer 101 , and then the process moves to step S212 .

[0370] In step S212, the external computer 101 receives the public key request from the digital camera 10 and generates a private key corresponding to the public key in response to the public key request. The process then moves to step S213.

[0371] In step S213 , the external computer 101 transmits the public key to the digital camera 10 , and the process moves to step S214 .

[0372] In step S214 , the digital camera 10 stores the public key supplied from the external computer 101 in the medium 14 or the like, and the pre-processing ends.

[0373] Note that, for example, the digital camera 10 may transmit a request to the external computer 101 for the same number of public keys as the number of main images that can be stored in the medium 14, depending on the remaining capacity of the medium 14 storing the first encrypted HEIF file. In this case, the external computer 101 generates the same number of public keys as the requested public keys, and the generated public keys are stored in the digital camera 10. The digital camera 10 encrypts the common key used in encrypting the main image using a different public key for each main image.

[0374] Figure 39 1 is a flowchart for explaining an example of a generation process of generating a first encrypted HEIF file in a case where the first encrypted HEIF file is handled by the image processing system 100 .

[0375] For example, the generation process is started when the user operates the digital camera 10 to capture an image (main image) or the like.

[0376] In the generation process, in step S221 , the digital camera 10 captures an image according to, for example, a user's operation, and the process then moves to step S222 .

[0377] In step S222 , the file control unit 43 of the digital camera 10 generates a common key, and then moves to step S223 .

[0378] In step S223, the file control unit 43 generates an encrypted image by encrypting the main image obtained by capturing the image in step S221 with the common key generated in step S222. The process then moves to step S224.

[0379] In step S224, the file control unit 43 selects a public key of interest from among the public keys stored in the preprocessing, which is one of the public keys that has not yet been used to encrypt the shared key. Furthermore, the file control unit 43 generates an encrypted encryption key by encrypting the shared key used to encrypt the main image with the public key of interest. The process then moves from step S224 to step S225.

[0380] In step S225, the file control unit 43 generates a first encrypted HEIF file in which the encrypted image generated in step S223 and the encrypted encryption key generated in step S224 are associated with each other and stored. The generation process then ends.

[0381] Figure 40 1 is a flowchart for explaining an example of a reproduction process of reproducing a first encrypted HEIF file in a case where the first encrypted HEIF file is handled by the image processing system 100 .

[0382] For example, the reproduction process is started when the user operates the digital camera 10 to reproduce an image (main image) or the like.

[0383] During the reproduction process, in step S231, the file control unit 43 of the digital camera 10 selects a first encrypted HEIF file to be reproduced, for example, based on a user operation, and selects a reproduction target encrypted image from the first encrypted HEIF file. Furthermore, the file control unit 43 obtains (reads) the encrypted encryption key associated with the reproduction target encrypted image (or the encrypted encryption key corresponding to the encrypted image) from the first encrypted HEIF file to be reproduced, and the process then moves from step S231 to step S232.

[0384] In step S232 , the file control unit 43 attempts to acquire a private key corresponding to the reproduction target encrypted image, and then the process moves to step S233 .

[0385] For example, the file control unit 43 requests the external computer 101 to supply the private key corresponding to the encrypted image of the reproduction target. For example, the external computer 101 pre-stores a user ID and password associated with the private key corresponding to the encrypted image. The external computer 101 prompts the user of the digital camera 10 to enter the user ID and password, waits for the user ID and password to be entered, and then performs authentication. If the authentication is successful, or if the user ID and password stored in association with the private key corresponding to the encrypted image of the reproduction target match the user ID and password entered by the user, the external computer 101 transmits the private key corresponding to the encrypted image of the reproduction target to the digital camera 10.

[0386] On the other hand, if the authentication fails, the external computer 101 notifies the digital camera 10 of this fact.

[0387] In the digital camera 10 , when a private key corresponding to a reproduction target encrypted image is transmitted from the external computer 101 , the file control unit 43 acquires the private key.

[0388] In step S233, a check is made to determine whether the file control unit 43 has successfully acquired the private key in the manner described above.

[0389] If it is determined in step S233 that the private key has not been successfully acquired, the process moves to step S234. In step S234, for example, the digital camera 10 performs error processing and causes the liquid crystal panel 19 to display an error message indicating that image decryption has failed. The reproduction process then ends. Therefore, a user who cannot obtain the private key corresponding to the reproduction target encrypted image cannot view the main image obtained by decrypting the reproduction target encrypted image.

[0390] On the other hand, if it is determined in step S233 that the private key has been successfully acquired, the process moves to step S235.

[0391] In step S235, the file control unit 43 decrypts the encrypted encryption key acquired in step S231 into a common key using the private key acquired in step S232. The process then moves to step S236.

[0392] In step S236, the file control unit 43 decrypts the reproduction target encrypted image into a main image using the common key decrypted in step S235, and causes the liquid crystal panel 19 to display the main image. The copy process then ends.

[0393] Figure 41 is a flowchart for explaining an example of pre-processing to be performed in the case where the second encrypted HEIF file is handled by the image processing system 100.

[0394] exist Figure 41 In the preprocessing shown in FIG, in steps S251 to S254, the same Figure 38 The corresponding steps S211 to S214 of the pre-processing shown in FIG are similar to the process.

[0395] Figure 42 1 is a flowchart for explaining an example of a generation process of generating a second encrypted HEIF file in a case where the second encrypted HEIF file is handled by the image processing system 100 .

[0396] exist Figure 42 In the generation process shown in FIG, in steps S261 to S264, the Figure 39 The corresponding steps S221 to S224 of the generation process shown in FIG. 1 are similar to the process.

[0397] In addition, in step S265, the file control unit 43 generates a second encrypted HEIF file in which the encrypted image generated in step S263, the common key generated in step S262, and the encrypted encryption key generated in step S264 are associated with each other and stored. The generation process then ends.

[0398] Figure 43 This is a flowchart for explaining an example of a restriction process of restricting viewing of a main image obtained by decrypting an encrypted image to only users who can obtain a private key corresponding to the encrypted image when a second encrypted HEIF file is handled by the image processing system 100.

[0399] For example, when the user operates the digital camera 10 to execute the restriction process, the file control unit 43 selects, for example, the second encrypted HEIF file to be reproduced according to the user's operation in step S271, and obtains (reads) the plaintext common key and the encrypted image associated with the plaintext common key from the second encrypted HEIF file. The process then moves to step S272.

[0400] In step S272, the file control unit 43 decrypts the encrypted image acquired in step S271 into a main image using the common key also acquired in step S271, and causes the liquid crystal panel 19 to display the main image. The process then moves to step S273.

[0401] In step S273 , the file control unit 43 determines whether the user has performed a limiting operation of operating the digital camera 10 to limit viewing of the main image displayed in step S272 .

[0402] If it is determined in step S273 that the restriction operation has not been performed, the restriction process ends.

[0403] On the other hand, if it is determined in step S273 that the restriction operation has been performed, the process moves to step S274. The file control unit 43 deletes the plaintext common key acquired in step S271 from the second encrypted HEIF file, and the restriction process ends.

[0404] As described above, since the plaintext common key associated with the encrypted image is deleted from the second encrypted HEIF file, it is possible to limit viewing of the main image obtained by decrypting the encrypted image to only users who can obtain the private key corresponding to the encrypted image.

[0405] Figure 44 1 is a flowchart for explaining an example of a reproduction process of reproducing a second encrypted HEIF file in a case where the second encrypted HEIF file is handled by the image processing system 100 .

[0406] exist Figure 44 In the reproduction process shown in FIG, in step S281, the file control unit 43 selects a second encrypted HEIF file to be reproduced, for example, based on a user operation, and selects a reproduction target encrypted image from the second encrypted HEIF file. The file control unit 43 then determines whether the plaintext common key associated with the reproduction target encrypted image is stored in the second encrypted HEIF file to be reproduced.

[0407] If it is determined in step S281 that the plain text common key is stored, the process moves to step S287.

[0408] In step S287, the file control unit 43 decrypts the reproduction target encrypted image into a main image using the plain text common key associated with the encrypted image, and causes the liquid crystal panel 19 to display the main image. The reproduction process then ends.

[0409] On the other hand, if it is determined in step S281 that no plain text common key is stored, or if the plain text common key has been Figure 43 If the restriction process shown in is deleted, the process moves to step S282.

[0410] In steps S282 to S287, the Figure 40 The process is similar to the process in the corresponding steps S231 to S236 in the reproduction process.

[0411] Figure 45 is a flowchart for explaining an example of pre-processing to be performed in the case where the third encrypted HEIF file is handled by the image processing system 100 .

[0412] exist Figure 45 In the preprocessing shown in FIG, in steps S311 to S314, the Figure 38 The corresponding steps S211 to S214 of the pre-processing shown in FIG are similar to the process.

[0413] Figure 46 1 is a flowchart for explaining an example of a generation process of generating a third encrypted HEIF file in a case where the third encrypted HEIF file is handled by the image processing system 100 .

[0414] exist Figure 46 In the generation process shown in FIG, in steps S321 to S323, the Figure 39 The corresponding steps S221 to S223 of the generation process shown in FIG are similar to the process.

[0415] In addition, in step S324, the file control unit 43 selects a public key of interest from among the public keys stored in the preprocessing, which is one of the public keys that has not yet been used to encrypt the common key. The file control unit 43 also assigns a UUID as specific information to the external data associated with the main image (e.g., the RAW data of the main image). The file control unit 43 then encrypts the common key used in encrypting the main image and the UUID of the external data associated with the main image using the public key of interest to generate an encryption key containing the specific information. The process then moves from step S324 to step S325.

[0416] In step S325, the file control unit 43 generates a third encrypted HEIF file in which the encrypted image generated in step S323 and the encryption key containing the specific information generated in step S324 are associated with each other and stored. The generation process then ends.

[0417] Figure 47 1 is a flowchart for explaining an example of a reproduction process of reproducing the third encrypted HEIF file in a case where the third encrypted HEIF file is handled by the image processing system 100 .

[0418] exist Figure 47 In the reproduction process shown in FIG, in step S331, the file control unit 43 of the digital camera 10 selects a third encrypted HEIF file to be reproduced, for example, based on a user operation, and selects a reproduction target encrypted image from the third encrypted HEIF file. Furthermore, the file control unit 43 obtains an encryption key containing specific information associated with the reproduction target encrypted image from the third encrypted HEIF file to be reproduced, and the process then moves from step S331 to step S332.

[0419] In step S332, the file control unit 43 uses the Figure 40 In a similar manner to step S233, the process attempts to obtain the private key corresponding to the encrypted image of the reproduction target. Then, the process moves to step S333.

[0420] In step S333, a check is made to determine whether the file control unit 43 has successfully acquired the private key.

[0421] If it is determined in step S333 that the private key has not been successfully acquired, the process moves to step S334. In step S334, the digital camera 10 Figure 40 The error handling is performed in a similar manner to step S234. Then the copy process ends.

[0422] On the other hand, if it is determined in step S333 that the private key has been successfully acquired, the process moves to step S335.

[0423] In step S335, the file control unit 43 decrypts the encryption key containing the specific information acquired in step S331 into the uuid and the common key as the specific information using the private key acquired in step S332. The process then moves to step S336.

[0424] In step S336, the file control unit 43 decrypts the reproduction target encrypted image into a main image using the common key decrypted in step S335, and causes the liquid crystal panel 19 to display the main image. The process then moves to step S337.

[0425] In step S337, the file control unit 43 acquires the external data specified by the uuid decrypted in step S336, and the reproduction process ends. For example, the external data is reproduced in response to a user's operation or the like.

[0426] Figure 48 is a flowchart for explaining an example of pre-processing to be performed in the case where the fourth encrypted HEIF file is handled by the image processing system 100.

[0427] exist Figure 48 In the preprocessing shown in FIG, in steps S351 to S354, the Figure 38 The corresponding steps S211 to S214 of the preprocessing shown in FIG. 2 are processed similarly.

[0428] Figure 49 1 is a flowchart for explaining an example of a generation process of generating a fourth encrypted HEIF file in a case where the fourth encrypted HEIF file is handled by the image processing system 100 .

[0429] exist Figure 49 In the generation process shown in FIG, in steps S361 to S363, the Figure 46 The corresponding steps S321 to S323 of the generation process shown in FIG. 1 are processed similarly.

[0430] In addition, in step S364, the file control unit 43 encrypts the external data associated with the main image (for example, the RAW data of the main image) using the common key used in encrypting the main image. The process then moves to step S365.

[0431] In step S365, the file control unit 43 selects a public key of interest from among the public keys stored in the preprocessing, which is one of the public keys that has not yet been used to encrypt the common key. The file control unit 43 also assigns a UUID as specific information to the external data associated with the main image (the external data encrypted in step S364). The file control unit 43 then encrypts the common key used in encrypting the main image and the UUID of the external data associated with the main image using the public key of interest to generate an encryption key containing the specific information. The process then moves from step S365 to step S366.

[0432] In step S366, the file control unit 43 generates a fourth encrypted HEIF file in which the encrypted image generated in step S363 and the encryption key containing the specific information generated in step S365 are associated with each other and stored. The process then moves to step S367.

[0433] In step S367, for example, the file control unit 43 generates a file storing the external data encrypted in step S364, and stores the file into the medium 14. The generation process then ends.

[0434] Figure 50 1 is a flowchart for explaining an example of a reproduction process of reproducing the fourth encrypted HEIF file in a case where the fourth encrypted HEIF file is handled by the image processing system 100 .

[0435] exist Figure 50 In the reproduction process shown in FIG, in steps S371 to S377, the same Figure 47 The process is similar to the process in the corresponding steps S331 to S337.

[0436] Furthermore, in step S378, the file control unit 43 decrypts the external data acquired in step S377 using the common key decrypted in step S375. The copy process then ends.

[0437] That is, for the fourth encrypted HEIF file, the external data obtained in step S377 is encrypted with the common key, and thus the encrypted external data is decrypted with the common key in step S378.

[0438] Figure 51 is a flowchart for explaining an example of pre-processing to be performed in the case where the fifth or sixth encrypted HEIF file is handled by the image processing system 100 .

[0439] exist Figure 51 In the preprocessing shown in FIG, in steps S451 to S454, the Figure 38 The corresponding steps S211 to S214 of the preprocessing shown in FIG. 2 are processed similarly.

[0440] Figure 52 1 is a flowchart for explaining an example of a generation process of generating the fifth or sixth encrypted HEIF file in a case where the fifth or sixth encrypted HEIF file is handled by the image processing system 100 .

[0441] exist Figure 52 In the generation process shown in FIG, in steps S461 and S462, the Figure 39 The process is similar to the process in corresponding steps S221 and S222 in the generation process shown in .

[0442] In addition, in step S463, the file control unit 43 generates an encrypted image by encrypting the main image obtained by capturing the image in step S461 with the common key generated in step S462. The process then moves to step S464.

[0443] Note that, for the sixth encrypted HEIF file, in step S463 , the file control unit 43 also encrypts the external data associated with the main image using the common key used in the encryption of the main image.

[0444] In step S464, the file control unit 43 selects a public key of interest from among the public keys stored in the preprocessing, one of the public keys that has not yet been used to encrypt the shared key. The file control unit 43 also assigns a UUID as specific information to the external data associated with the main image. The file control unit 43 then encrypts the shared key set of the shared key used in encrypting the main image and the UUID of the external data associated with the main image using the public key of interest, generating an encryption key containing the specific information. The process then moves from step S464 to step S465.

[0445] In step S465, the file control unit 43 generates a fifth or sixth encrypted HEIF file in which the encrypted image generated in step S463, the encryption key containing the specific information generated in step S464, and the (plaintext) shared key set previously encrypted with the encryption key containing the specific information are associated with each other and stored. The generation process then ends.

[0446] Note that, for the sixth encrypted HEIF file, in step S465 , for example, the file control unit 43 also generates a file storing the external data encrypted in step S463 , and stores the file in the medium 14 .

[0447] Figure 53 This is a flowchart for explaining an example of a restriction process of limiting viewing of a main image obtained by decrypting an encrypted image to only users who can obtain a private key corresponding to the encrypted image when the fifth or sixth encrypted HEIF file is handled by the image processing system 100.

[0448] exist Figure 53 In the restricted process shown in FIG, in step S471, the file control unit 43 selects the fifth or sixth encrypted HEIF file to be reproduced, for example, based on a user operation, and obtains the (plaintext) common key set and the encrypted image associated with the common key set from the fifth or sixth encrypted HEIF file. The process then moves to step S472.

[0449] In step S472, the file control unit 43 decrypts the encrypted image acquired in step S471 into a main image using the common key forming the common key set also acquired in step S471, and causes the liquid crystal panel 19 to display the main image. The process then moves to step S473.

[0450] In step S473 , the file control unit 43 determines whether the user has performed a limiting operation of operating the digital camera 10 to limit viewing of the main image displayed in step S472 .

[0451] If it is determined in step S473 that the restriction operation has not been performed, the restriction process ends.

[0452] On the other hand, if it is determined in step S473 that the restriction operation has been performed, the process moves to step S474. The file control unit 43 deletes the common key set acquired in step S471 from the fifth or sixth encrypted HEIF file, and the restriction process ends.

[0453] As described above, since the plaintext shared key associated with the encrypted image is deleted from the fifth or sixth encrypted HEIF file, it is possible to limit viewing of the main image obtained by decrypting the encrypted image and limit access to external data associated with the main image to only users who can obtain the private key corresponding to the encrypted image.

[0454] Figure 54 1 is a flowchart for explaining an example of a reproduction process of reproducing the fifth or sixth encrypted HEIF file in a case where the fifth or sixth encrypted HEIF file is handled by the image processing system 100 .

[0455] exist Figure 54 In the reproduction process shown in FIG, in step S481, the file control unit 43 selects the fifth or sixth encrypted HEIF file to be reproduced, for example, based on a user operation, and selects a reproduction target encrypted image from the fifth or sixth encrypted HEIF file. The file control unit 43 then determines whether the plaintext shared key set associated with the reproduction target encrypted image is stored in the fifth or sixth encrypted HEIF file to be reproduced.

[0456] If it is determined in step S481 that the common key set is stored, the process moves to step S487.

[0457] In step S487, the file control unit 43 decrypts the reproduction target encrypted image into a main image using the common key forming the common key set associated with the encrypted image, and causes the liquid crystal panel 19 to display the main image. The process then moves to step S488.

[0458] In step S488, the file control unit 43 acquires the external data specified by the uuid forming the common key set associated with the encrypted image of the reproduction target. The reproduction then ends.

[0459] Note that for the sixth encrypted HEIF file, the external data is encrypted. Therefore, in step S488, the file control unit 43 also obtains the encrypted external data specified by the uuid forming the common key set, and then decrypts the encrypted external data using the common key forming the common key set.

[0460] On the other hand, if it is determined in step S481 that no common key set is stored, or if the common key set has been Figure 53 If the restriction process shown in is deleted, the process moves to step S482.

[0461] In steps S482 to S488, the Figure 47 The reproduction process is similar to the process in the corresponding steps S331 to S337. Then, the reproduction process ends.

[0462] Note that for the sixth encrypted HEIF file, the external data is encrypted. Therefore, in step S488, the file control unit 43 also obtains the encrypted external data specified by the uuid obtained by decrypting the encryption key containing the specific information in step S486, and then decrypts the encrypted external data with the common key obtained by decrypting the encryption key containing the specific information in step S486.

[0463] <Second Embodiment of Image Processing System to Which the Present Technology is Applied>

[0464] Figure 55 is a block diagram illustrating an example configuration of a second embodiment of an image processing system to which the present technology is applied.

[0465] Note that in this figure, Figure 31 The components of the image processing system 100 in FIG. 1 correspond to the components of the image processing system 100 in FIG. Figure 31 The same reference numerals used in the drawings are used, and explanation thereof will not be repeated below.

[0466] exist Figure 55 In FIG. 1 , the image processing system 200 includes a digital camera 210 and an external computer 101 .

[0467] Accordingly, the image processing system 200 and Figure 31 The image processing system 100 in FIG. 1 is similar to the image processing system 100 in that it includes an external computer 101 , but is different from the image processing system 100 in that it includes a digital camera 210 instead of the digital camera 10 .

[0468] Figure 56 is a block diagram illustrating an example configuration of the digital camera 210 .

[0469] Note that in this figure, Figure 1The components of the digital camera 10 correspond to the components of Figure 1 The same reference numerals used in the drawings are denoted, and explanation thereof will not be repeated below.

[0470] The digital camera 210 includes components from the optical system 11 to the interface 21 , and the signal processing unit 13 includes components from the optical system / image sensor control unit 41 to the UI control unit 47 , and the recognition unit 211 .

[0471] Accordingly, the digital camera 210 is similar to the optical camera 21 in terms of components including the optical system 11 to the interface 21. Figure 1 The digital camera 10 is the same as that in FIG. 1 , and the signal processing unit 13 is the same as that in FIG. 1 in terms of components including the optical system / image sensor control unit 41 to the UI control unit 47. Figure 1 The same as in .

[0472] However, the digital camera 210 is different from the digital camera 10 in that a recognition unit 211 is added to the signal processing unit 13 .

[0473] The recognition unit 211 receives (data of) the main image supplied from the optical system / image sensor control unit 41 .

[0474] The recognition unit 211 performs image recognition on the main image supplied from the optical system / image sensor control unit 41. Through the image recognition, the recognition unit 211 recognizes a subject appearing in the main image as a recognition target, and supplies the recognition result to the file control unit 43.

[0475] In the recognition unit 211, the recognition algorithm used to recognize the subject can be an algorithm for recognizing any appropriate subject appearing in the main image as a recognition target and identifying (classifying) the type of subject as a recognition target (such as, for example, a person, a car, or a trademark), or it can be a recognition algorithm for recognizing a specific type of subject as a recognition subject and identifying an individual, such as a recognition algorithm for identifying an individual.

[0476] In addition, other sensors 212 may be provided in the digital camera 210, and the recognition unit 211 may recognize a subject appearing in the main image using sensing results supplied from the other sensors 212 in addition to the main image. For example, the other sensors 212 may include a global positioning system (GPS) sensor, an orientation sensor, a depth (distance measurement) sensor, and the like. The recognition unit 211 may improve the recognition accuracy of a subject appearing in the main image by specifying the current position using the sensing results supplied from the GPS sensor or the orientation sensor and limiting the recognition target to a subject that may be present at the current position. Moreover, using a distance image in addition to the main image, the recognition unit 211 may improve the recognition accuracy in low illumination by generating a distance image having the distance to the subject as its pixel value from the sensing results supplied from the depth sensor and recognizing the subject appearing in the main image.

[0477] Here, there is "grid" as an item type of HEIF. For a main image with an item type of "grid", each of the multiple segmented images (tiles) obtained by dividing the main image into segmented images of the same size is stored as an item in the HEIF file. The main image with the item type of "grid" is reconstructed from the multiple segmented images as items. The item with the item type of "grid" (in this example, the main image) or the main image to be divided into multiple segmented images (and the main image to be reconstructed from the multiple segmented images) is also called a grid item.

[0478] In the digital camera 210, the file control unit 43 may encrypt not the entire main image but the segmented images obtained by segmenting the main image. For example, the file control unit 43 may encrypt the segmented images in which a predetermined subject appears in the main image based on the recognition result supplied from the recognition unit 211.

[0479] In the following description, we will explain how to process encrypted HEIF files when encryption is performed in units of split images. However, before that, we will describe a HEIF file that stores a main image whose item type is "grid" (this HEIF file will also be referred to as a grid file hereinafter).

[0480] <mesh file>

[0481] Figure 57 is a diagram showing an example of a mesh file.

[0482] exist Figure 57 In the grid file of , the main image is divided into 3×3 divided images Item#1 to Item#9 arranged in vertical and horizontal directions, and the divided images Item#1 to Item#9 are stored as items in the mdat box.

[0483] exist Figure 57In , the items stored in the mdat box are the nine items of segmented images Item#1 to Item#9, but the number of items in the iinf box and the iloc box is 10. This is because, in addition to the nine items of segmented images Item#1 to Item#9, the main image (reconstructed image) of the grid item to be reconstructed from the segmented images Item#1 to Item#9 is also counted as an item. Figure 57 , the item ID of the main image as a grid item is 10, and for ease of explanation, this main image is referred to as main image Item#10.

[0484] For main image #10, which is a grid item, media data is not stored in the mdat box. Instead, the idat box is stored in the meta box. The idat box stores the number of horizontal grids, the number of vertical grids, the horizontal output size, and the vertical output size. The horizontal grid number and the vertical grid number indicate the number of segmented images #1 to #9 that make up main image #10, which is a grid item, in the horizontal and vertical directions, respectively. The horizontal output size and the vertical output size indicate the horizontal and vertical sizes of main image #10, which is a grid item reconstructed from segmented images #1 to #9, respectively.

[0485] In addition, regarding the main image #10 as a grid item, information (offset and size) about the storage location of the main image #10 is stored in the iloc box, and this information indicates the storage location of the idat box of the main image #10 as a grid item.

[0486] Figure 57 The grid file in includes ten infe boxes because ten segmented images Item#1 to Item#9 and the main image Item#10 to be reconstructed from the segmented images Item#1 to Item#9 are stored therein. Figure 9 As described above, the item ID and item type for specifying an item are stored (registered) in the infe box, and "grid" indicating a grid item is stored as the item type of the main image Item#10 in the infe box of the main image Item#10 which is a grid item.

[0487] In the mesh file, the iref box stores the dimg box. The dimg box stores information for associating a mesh item with a segmented image constituting the mesh item. For example, the dimg box storing the item ID of the segmented image is stored as a reference destination, and the item ID of the main image that is the mesh item to be reconstructed from the segmented image is stored as a reference source. Figure 57 , item IDs #1 to #9 of divided images Item #1 to Item #9 are stored as reference destinations, and item ID #10 of main image Item #10 is stored as a reference source. The dimg box also stores a reference counter indicating the number of divided images.

[0488] For the grid file described above, file control unit 43 can identify, from the item type "grid" stored in the infe box of main image Item #10, that main image Item #10 is a reconstructed image (grid item) to be reconstructed from a plurality of segmented images. Furthermore, file control unit 43 can specify, from the reference source and reference destination stored in the dimg box, item ID #10 of main image Item #10 to be reconstructed from the segmented images, and item IDs #1 to #9 of segmented images Item #1 to Item #9 to be used to reconstruct main image Item #10. Based on the horizontal and vertical grid numbers and the horizontal and vertical output sizes stored in the idat box, file control unit 43 can also specify the number of segmented images #1 to #9 that constitute main image #10 in the horizontal and vertical directions, as well as the horizontal and vertical sizes of main image #10 to be reconstructed from segmented images #1 to #9.

[0489] Based on the split images Item#1 to Item#9 of the item ID#1 to #9 specified from the dimg box, the file control unit 43 can reconstruct the main image Item#10 having the item ID#10 specified from the dimg box, having the size specified from the idat box and formed by the same number of split images in the horizontal and vertical directions as the number specified from the idat box.

[0490] In the digital camera 210 ( Figure 56 ), the file control unit 43 may generate an encrypted mesh file in which an encrypted divided image obtained by dividing a main image is encrypted with a common key (first encryption key) and an encrypted encryption key obtained by encrypting the common key with a public key (second encryption key) are associated with each other and stored. The encrypted mesh file is one type of encrypted HEIF file.

[0491] When associating the encrypted segmented image stored in the encrypted mesh file with the encrypted encryption key obtained by encrypting the common key used in the encryption for obtaining the encrypted segmented image, the file control unit 43 can use the association between the main image stored in the associated HEIF file and the specific information about the external data. That is, in a manner similar to the association between the main image stored in the associated HEIF file and the specific information about the external data, the file control unit 43 can associate the encrypted segmented image stored in the encrypted mesh file with the encrypted encryption key.

[0492] The file control unit 43 can also decrypt the encryption key of the encrypted grid file into a common key, and decrypt the encrypted segmented image into the original segmented image using the common key obtained by decryption. In addition, the file control unit 43 can reconstruct the reconstructed image into a main image by arranging the segmented images.

[0493] <Handling of Encrypted Mesh Files>

[0494] Figure 58 is a diagram for explaining an outline of encrypted mesh file processing to be executed by the digital camera 210 .

[0495] When generating the encrypted grid file, the digital camera 210 ( Figure 56 ) sets a human face as a recognition target, recognizes a human face appearing in a main image, for example, and supplies the recognition result to the file control unit 43.

[0496] Based on the recognition result supplied from the recognition unit 211, for example, the file control unit 43 encrypts the segmented image in which the recognition target has been recognized among the segmented images obtained by dividing the main image (the segmented image in which the face of the person being the recognition target appears) with the common key as an encrypted segmented image, and stores the encrypted segmented image in the encrypted grid file. Segmented images other than the segmented image in which the recognition target has been recognized may be encrypted with the common key as encrypted segmented images and stored in the encrypted grid file, just like the segmented image in which the recognition target has been recognized, or may be stored (as plain text) in the encrypted grid file without being encrypted. In this example, the segmented images other than the segmented image in which the recognition target has been recognized are stored in the encrypted grid file without being encrypted.

[0497] As in the case of the encrypted HEIF file, the file control unit 43 encrypts the common key used in the encryption of the split image into an encrypted encryption key with a public key, and stores the encrypted encryption key in the encrypted grid file by associating the encrypted encryption key with the corresponding encrypted split image (the split image encrypted with the public key obtained by decrypting the encrypted encryption key).

[0498] When reproducing the encrypted mesh file, the file control unit 43 in the digital camera 210 reconstructs a reconstructed image as a main image based on the divided images and the encrypted divided images stored in the encrypted mesh file.

[0499] If the private key corresponding to the public key is available, when reconstructing the reconstructed image into the main image, file control unit 43 uses the private key to decrypt the encrypted encryption key associated with the encrypted segmented image into the common key, and then uses the common key to decrypt the encrypted segmented image into the original segmented image. Furthermore, file control unit 43 reconstructs the reconstructed image into the main image by aligning the (plaintext) segmented image stored in the encrypted grid file with the segmented image obtained by decrypting the encrypted segmented image. Thus, it is possible to obtain a reconstructed image that shows the subject, in this case, the human face, that appeared in the original main image as the recognition target.

[0500] On the other hand, if the private key corresponding to the public key cannot be obtained, when reconstructing the reconstructed image into the main image, the file control unit 43 reconstructs the reconstructed image into the main image by arranging the (plain text) segmented images stored in the encrypted grid file and the encrypted segmented images (or, for example, a patternless monochrome image representing the encrypted segmented images). Therefore, it is possible to obtain a reconstructed image that does not show (or hides) the subject, in this case, the human face, which was the recognition target appearing in the original main image.

[0501] As described above, with the encrypted grid file, viewing of a subject as a recognition target can be limited to users who can obtain a private key corresponding to a public key.

[0502] Note that in the digital camera 210, the encrypted segmented image can be encrypted with a common key that varies with each subject appearing in the corresponding segmented image, and the common key used in the encryption of the segmented image can be encrypted with a public key that varies with each subject.

[0503] That is, in the digital camera 210, the segmented image can be encrypted with a common key that is different for each subject that is the recognition target appearing in the segmented image. Alternatively, the common key that is different for each subject that is the recognition target appearing in the segmented image (and the private key that is also different accordingly) can be encrypted with a public key that is different for each subject that is the recognition target.

[0504] For example, when the face is the target of recognition and Figure 58 In the case of identifying the face of person A and the face of person B appearing in the main image as shown in , the segmented image in which the face of person A appears and the segmented image in which the face of person B appears may be encrypted with different common keys. In addition, the common key used in encrypting the segmented image in which the face of person A appears and the common key used in encrypting the segmented image in which the face of person B appears may be encrypted with different public keys KA and KB.

[0505] In this example, if the private key corresponding to the public key KA is available, a reconstructed image showing the face of person A is reconstructed, and if the private key corresponding to the public key KB is available, a reconstructed image showing the face of person B is reconstructed. Accordingly, viewing of person A's face can be restricted to users who can obtain the private key corresponding to the public key KA, and viewing of person B's face can be restricted to users who can obtain the private key corresponding to the public key KB.

[0506] As described above, for each subject, users who can view the subject can be limited.

[0507] Furthermore, in this example, individuals are identified, where the identification target is a specific type of subject, i.e., a person. However, multiple types of subjects can be set as identification targets, and the type of subject used as the identification target can be identified. For example, a person and a trademark can be identified as the identification target. In this case, encryption can be performed using different shared keys and public keys for the corresponding types of subjects used as identification targets (i.e., a person and a trademark).

[0508] In addition, the recognition unit 211 can set the subject to be recognized according to the user's operation. For example, it can be set according to the user's operation whether to recognize an individual as a specific type of subject such as a person as the recognition target, or whether to recognize a person and a trademark as the recognition target as described above.

[0509] <Encrypted mesh file>

[0510] Figure 59 is a diagram showing an example of an encrypted mesh file.

[0511] Figure 59 The encrypted grid file in Figure 57 The encrypted grid file corresponding to the grid file in . That is, Figure 59 The encrypted grid files in the image are stored based on the segmented images. Figure 57 The encrypted image obtained by encrypting the divided images Item#1 to Item#9 in the mesh file in the encrypted mesh file is the encrypted divided images Item#1 to Item#9.

[0512] Note that the contents of the iloc and idat boxes are in Figure 59 Not shown, but with Figure 57 The content is similar.

[0513] Figure 59 The encrypted grid file in Figure 57The grid file in is different in that, instead of the (plaintext) divided images Item#1 to Item#9, encrypted divided images Item#1 to Item#9 are stored as items in the mdat box. Figure 59 The encrypted grid file in Figure 57 The grid file in is also different in that the encrypted encryption keys Item#11 to Item#19 obtained by encrypting the shared key used in the encryption for obtaining the encrypted segmented images Item#1 to Item#9 with the public key are newly stored as items in the mdat box. In addition, Figure 59 The encrypted grid file in Figure 57 The difference between the grid files in is that the cdsc box that stores the association information is now stored in the iref box.

[0514] Note that, in Figure 59 In the encrypted grid file in Figure 57 The number of encrypted encryption keys Item#11 to Item#19 is increased in the grid file in . Figure 59 In the encrypted grid file, the number of entries in the iinf box is 19, and in this respect, Figure 59 The encrypted grid file in Figure 57 The difference in the number of terms in is ten. In addition, Figure 59 The encrypted grid file in Figure 57 The difference between the grid files in the example is that the infe boxes for encryption keys Item#11 to Item#19 are newly provided, which results in a Figure 57 The increase in items in the grid file.

[0515] exist Figure 59 For example, before being encrypted into encrypted segmented images Item #1 to Item #9, subjects serving as different recognition targets are captured in the respective segmented images 1 to 9. Therefore, segmented images 1 to 9 are encrypted using different public keys into encrypted segmented images Item #1 to Item #9. Furthermore, the common key used in encrypting segmented images 1 to 9, in which subjects serving as different recognition targets are captured, is encrypted using different public keys into encrypted encryption keys Item #11 to Item #19.

[0516] In the encrypted grid file, association information for associating the encrypted segmented image Item#i with the encrypted encryption key Item#10+i obtained by encrypting the common key used in the encryption for obtaining the encrypted segmented image Item#i with a public key is stored in the cdsc box, so that the encrypted segmented image Item#i and the encrypted encryption key Item#10+i are associated with each other, for example, as in the (first to sixth) encrypted HEIF files.

[0517] For encrypted segmented image Item#i stored in the encrypted grid file, if the private key corresponding to the encrypted segmented image Item#i (the private key corresponding to the public key used to encrypt the common key used to encrypt the encrypted segmented image Item#i) is available, the file control unit 43 can decrypt the encrypted encryption key Item#10+i associated with the encrypted segmented image Item#i using the private key to convert it into the common key, and decrypt the encrypted segmented image Item#i using the public key obtained by decryption to return it to the original segmented image i. Therefore, for each segmented image i obtained by dividing the main image, viewing of (the subject appearing in) the segmented image i can be limited to users who can obtain the private key corresponding to the encrypted segmented image Item#i.

[0518] When decrypting encrypted divided image Item#i, file control unit 43 specifies the item ID of the reference destination in the cdsc box containing the reference source of item ID#i of encrypted divided image Item#i as item ID#10+i of the encrypted encryption key Item#10+i corresponding to encrypted divided image Item#i, or the item ID of the encrypted encryption key Item#10+i obtained by encrypting the common key used in encrypting encrypted divided image Item#i. Based on item ID#10+i, file control unit 43 obtains the encrypted encryption key Item#10+i from the mdat box and performs decryption.

[0519] Note that, in Figure 59 In the encrypted grid file, all nine segmented images 1 to 9 that make up the main image are encrypted as encrypted segmented images Item#1 to Item#9. However, not all segmented images 1 to 9 that make up the main image need to be encrypted. That is, among the segmented images 1 to 9 that make up the main image, only the segmented images that contain the subject to be recognized can be encrypted.

[0520] Figure 60 is a flowchart for explaining a first example of pre-processing to be performed in the case where an encrypted mesh file is handled by the image processing system 200 .

[0521] In the first example of pre-processing, in step S511 , the digital camera 210 of the image processing system 200 transmits a request for a predetermined number of public keys to the external computer 101 , and then the process moves to step S512 .

[0522] For example, in a case where a person is set as an identification target in the identification unit 211 to identify an individual, facial recognition is performed using an image captured by the digital camera 210 to display a so-called through-the-lens image, thereby specifying the number of people (individuals) as identification targets, and requesting as many public keys as the number of people as identification targets.

[0523] In addition, in the digital camera 210 , for example, the user inputs information on persons whose faces are to be hidden, and requests as many public keys as the number of persons whose faces are to be hidden according to the information.

[0524] In steps S512 to S514, the Figure 38 Therefore, for example, the digital camera 210 stores a required number of public keys (such as the number of people whose faces are to be hidden).

[0525] Figure 61 1 is a flowchart for explaining a first example of a generation process of generating an encrypted mesh file in a case where the encrypted mesh file is handled by the image processing system 200 .

[0526] In the first example of the generation process, in step S521 , the digital camera 210 captures an image according to, for example, an operation of a user, and then the process moves to step S522 .

[0527] In step S522 , the recognition unit 211 performs image recognition using the image captured in step S521 , and the process moves to step S523 .

[0528] In step S523, the file control unit 43 controls the decode control unit 42 to divide the main image obtained by capturing the image in step S521 according to the recognition targets recognized in the image recognition performed by the recognition unit 211. Under the control of the file control unit 43, the decode control unit 42 divides the main image into a plurality of divided images so that, for example, different recognition targets appear in different divided images (so that one or fewer recognition targets appear in one divided image), and supplies the plurality of divided images to the file control unit 43. The process then moves from step S523 to step S524.

[0529] In step S524 , the file control unit 43 generates different common keys for the respective recognition targets recognized in the image recognition by the recognition unit 211 , and the process moves to step S525 .

[0530] In step S525, the file control unit 43 generates an encrypted divided image by encrypting the divided image showing the recognition target recognized by the recognition unit 211 in the image recognition among the divided images obtained by dividing the main image using the common key for the recognition target. The process then moves to step S526.

[0531] In step S526, the file control unit 43 encrypts the common key for the corresponding recognition target identified by the recognition unit 211 in the image recognition process using the public key stored in the first example of pre-processing, thereby generating an encrypted encryption key for the corresponding recognition target. The process then moves to step S527. Note that the common keys for different recognition targets are encrypted using different public keys.

[0532] In step S527, the file control unit 43 generates an encrypted grid file that stores the segmented images obtained by dividing the main image, in which no recognition target appears, and the encrypted segmented images generated in step S525 in association with the encrypted encryption key generated in step S526 and corresponding to the encrypted segmented images (the encrypted encryption key obtained by encrypting the common key used in the encryption for obtaining the encrypted segmented images). The generation process then ends.

[0533] Note that according to HEIF, the main image is divided into multiple segmented images of the same size. Therefore, if the main image is divided into segmented images in which one or fewer identification targets appear, a single identification target may appear in multiple segmented images. If a single identification target appears in multiple segmented images, the multiple segmented images can be encrypted using the same shared key.

[0534] In this example, a common key is generated for each recognition target identified by image recognition, and the segmented image showing the recognition target is encrypted using the common key for the recognition target. However, for example, a common key may be generated for each segmented image showing the recognition target identified by image recognition, and the segmented image showing the recognition target may be encrypted using the common key for the segmented image.

[0535] Figure 62 is a flowchart for explaining an example of a reproduction process of reproducing an encrypted mesh file in a case where the encrypted mesh file is handled by the image processing system 200 .

[0536] During the reproduction process, in step S531, the file control unit 43 of the digital camera 210 selects an encrypted mesh file to be reproduced, for example, based on a user operation, and selects a reproduction target main image from the encrypted mesh file. Furthermore, the file control unit 43 reconstructs the main image using the segmented images constituting the reproduction target main image and the encrypted segmented images, and causes the liquid crystal panel 19 to display the main image.

[0537] In the main image reconstructed in step S531, the subject appearing in the region of the encrypted divided image is not visible. After such a main image is displayed in step S531, the process moves to step S532.

[0538] In step S532, for example, when the user specifies an encrypted divided image in the main image displayed in step S531, the file control unit 43 selects the encrypted divided image specified by the user as the encrypted divided image of interest. Furthermore, the file control unit 43 obtains the encrypted encryption key associated with the encrypted divided image of interest (or the encrypted encryption key corresponding to the encrypted divided image) from the encrypted mesh file to be reproduced, and the process then moves from step S532 to step S533.

[0539] In step S533, for example, the file control unit 43 attempts to Figure 40 The private key corresponding to the encrypted segmented image of interest (the private key corresponding to the public key used to encrypt the encrypted segmented image of interest) is acquired in a manner similar to that in step S232 in . The process then moves to step S534.

[0540] In step S534 , a check is made to determine whether the file control unit 43 has successfully acquired the private key corresponding to the encrypted divided image of interest.

[0541] If it is determined in step S534 that the private key has not been successfully acquired, the process moves to step S535 and the same steps as above are performed. Figure 40 The reproduction process then ends. Therefore, a user who cannot obtain the private key corresponding to the encrypted segmented image of interest cannot view the main image shown in the segmented image obtained by decrypting the encrypted segmented image of interest.

[0542] On the other hand, if it is determined in step S534 that the private key has been successfully acquired, the process moves to step S536.

[0543] In step S536, the file control unit 43 decrypts the encrypted encryption key acquired in step S532 into a common key using the private key acquired in step S533. The process then moves to step S537.

[0544] In step S537, the file control unit 43 decrypts the encrypted divided image of interest into divided images using the common key decrypted in step S536. The process then moves to step S538.

[0545] In step S538, the file control unit 43 reconstructs the main image using the divided images obtained by decrypting the encrypted divided image of interest instead of the encrypted divided image of interest, and causes the liquid crystal panel 19 to display the main image. The reproduction process then ends.

[0546] Figure 63 is a flowchart for explaining a second example of pre-processing to be performed in the case where an encrypted mesh file is handled by the image processing system 200 .

[0547] In the second example of pre-processing, in step S611 , the digital camera 210 of the image processing system 200 transmits a request for a predetermined number of public keys to the external computer 101 , and then the process moves to step S612 .

[0548] For example, in a method for dividing a main image into divided images of the same size, or presetting the number of divided images in the horizontal and vertical directions of the main image, as many public keys as the number of divided images constituting the main image are requested.

[0549] In steps S612 to S614, the Figure 38 Therefore, in the digital camera 210, as many public keys as the maximum number that may be required, or as many public keys as the number of divided images constituting the main image, are acquired and stored as public keys for the corresponding divided images constituting the main image.

[0550] Figure 64 1 is a flowchart for explaining a second example of a generation process of generating an encrypted mesh file in a case where the encrypted mesh file is handled by the image processing system 200 .

[0551] In the second example of the generation process, in step S621 , the digital camera 210 captures an image according to, for example, an operation of a user, and the process moves to step S622 .

[0552] In step S622, the file control unit 43 controls the decode control unit 42 to divide the main image obtained by capturing the image in step S621. Under the control of the file control unit 43, the decode control unit 42 divides the main image into a plurality of divided images of the same size, for example, using a preset division method, and supplies the plurality of divided images to the file control unit 43. The process then moves from step S622 to step S623.

[0553] In step S623 , the recognition unit 211 performs image recognition using the image captured in step S621 , and the process moves to step S624 .

[0554] In step S624, the file control unit 43 generates a common key for each of the divided images constituting the main image showing the recognition target recognized in the image recognition by the recognition unit 211. The process then moves to step S625.

[0555] In step S625, the file control unit 43 generates an encrypted divided image by encrypting the divided image showing the recognition target recognized by the recognition unit 211 in the image recognition among the divided images obtained by dividing the main image using the common key for the divided image. The process then moves to step S626.

[0556] In step S626, the file control unit 43 encrypts the shared key representing the segmented image of the recognition target identified by the recognition unit 211 in the image recognition process using the public key stored in the second example of preprocessing. This public key is used to segment the image, thereby generating an encrypted encryption key for the corresponding segmented image representing the recognition target. The process then moves from step S626 to step S627.

[0557] In step S627, the file control unit 43 generates an encrypted grid file that stores the segmented images obtained by dividing the main image, in which no recognition target appears, and the encrypted segmented images generated in step S625 and associated with the encryption key corresponding to the encrypted segmented images generated in step S626. The generation process then ends.

[0558] The reproduction process for reproducing the encrypted mesh file generated in the second example of the generation process is, for example, the same as Figure 62 , so this article will not explain it again.

[0559] Figure 65 1 is a flowchart for explaining a third example of a generation process of generating an encrypted mesh file in a case where the encrypted mesh file is handled by the image processing system 200 .

[0560] In the first and second examples of the generation process, the common key is encrypted with the public key pre-stored in the digital camera 210 by the first and second examples of preprocessing. However, in the third example of the generation process, the necessary public key is acquired in the generation process without preprocessing.

[0561] In the third example of the generation process, in steps S721 and S722, the Figure 61The process is similar to the process in steps S521 and S522 in . Then the process moves to step S723.

[0562] In step S723, the digital camera 210 acquires and stores a public key for a new recognition target among the recognition targets recognized in the image recognition in step S722 by the recognition unit 211. The process then moves to step S724.

[0563] Specifically, the digital camera 210 stores the recognition targets identified by the recognition unit 211 during image recognition, and when a new recognition target is identified, acquires and stores the public key for the new recognition target. Specifically, the digital camera 210 transmits requests for as many public keys as the number of new recognition targets to the external computer 101. In response to the requests for public keys from the digital camera 210, the external computer 101 generates a public key and a corresponding private key for the new recognition target, and transmits the public key to the digital camera 210. The digital camera 210 receives and stores the public key provided for the new recognition target from the external computer 101.

[0564] In step S724, the file control unit 43 controls the decoding control unit 42 to divide the main image obtained by capturing the image in step S721 according to the recognition target recognized in the image recognition performed by the recognition unit 211. For example, under the control of the file control unit 43, the decoding control unit 42 divides the main image obtained by capturing the image in step S721 in a manner similar to Figure 61 The main image is divided into a plurality of divided images in the manner of step S523 in the above process, and the plurality of divided images are supplied to the file control unit 43. The process then moves from step S724 to step S725.

[0565] In step S725 , the file control unit 43 generates a common key for the corresponding recognition target recognized by the recognition unit 211 in the image recognition, and the process moves to step S726 .

[0566] In step S726, the file control unit 43 generates an encrypted divided image by encrypting the divided image showing the recognition target recognized by the recognition unit 211 in the image recognition among the divided images obtained by dividing the main image using the common key for the recognition target. The process then moves to step S727.

[0567] In step S727, the file control unit 43 encrypts the common key for the corresponding recognition target identified by the recognition unit 211 in the image recognition with the public key for the recognition target, thereby generating an encrypted encryption key for the corresponding recognition target. The process then moves to step S728.

[0568] In step S728, the file control unit 43 generates an encrypted grid file that stores the segmented images obtained by dividing the main image, in which no recognition target appears, and the encrypted segmented images generated in step S726 and associated with the encryption key that was generated in step S727 and corresponds to the encrypted segmented images. The generation process then ends.

[0569] Note that in this example, a shared key is generated for each recognition target identified by image recognition, and the segmented image showing the recognition target is encrypted using the shared key for the recognition target. However, for example, a shared key may be generated for each segmented image showing the recognition target identified by image recognition, and the segmented image showing the recognition target may be encrypted using the shared key for the segmented image.

[0570] <Description of Computer to Which the Present Technology is Applied>

[0571] Next, the file control unit 43 and the signal processing unit 13 ( Figure 1 ) etc. can be executed by hardware or software. In the case of executing a series of processes by software, a program forming the software is installed in a computer etc.

[0572] Figure 66 : is a block diagram showing an example configuration of an embodiment of a computer installed with a program for executing the above-described series of processes.

[0573] The program can be recorded in advance in the hard disk 905 or the ROM 903 provided as a recording medium in the computer.

[0574] Alternatively, the program may be stored (recorded) in a removable recording medium 911 driven by the drive 909. Such a removable recording medium 911 may be provided as so-called packaged software. Here, the removable recording medium 911 may be, for example, a floppy disk, a compact disc read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a magnetic disk, a semiconductor memory, or the like.

[0575] Note that the program can be installed into the computer from the above-mentioned removable recording medium 911, but can also be downloaded into the computer via a communication network or a broadcasting network and installed into the internal hard disk 905. That is, the program can be wirelessly transmitted from a download site, for example, to the computer via an artificial satellite for digital satellite broadcasting, or can be transmitted to the computer by a cable via a network such as a local area network (LAN) or the Internet.

[0576] The computer includes a central processing unit (CPU) 902 , and an input / output interface 910 is connected to the CPU 902 via a bus 901 .

[0577] When a user inputs an instruction by operating the input unit 907 or the like via the input / output interface 910, the CPU 902 executes a program stored in a read-only memory (ROM) 903 according to the instruction. Alternatively, the CPU 902 loads a program stored in the hard disk 905 into a random access memory (RAM) 904 and executes the program.

[0578] By doing so, the CPU 902 executes the process according to the above flowchart, or executes the process using the components shown in the above block diagram. Then, the CPU 902 outputs the process result from the output unit 906, for example, or transmits the process result from the communication unit 908 via the input / output interface 910, and further stores the process result in the hard disk 905 or the like as necessary.

[0579] Note that the input unit 907 is formed of a keyboard, a mouse, a microphone, etc. Meanwhile, the output unit 906 is formed of a liquid crystal display (LCD), a speaker, and the like.

[0580] In this specification, the processes executed by a computer according to a program are not necessarily executed in chronological order according to the order shown in the flowcharts. In other words, the processes executed by a computer according to a program include processes executed in parallel or independently of each other (such as parallel processes or subject-based processes).

[0581] Moreover, the program may be executed by one computer (processor), or may be executed in a distributed manner by a plurality of computers. In addition, the program may be transferred to a remote computer and executed therein.

[0582] In this specification, a system refers to an assembly of multiple components (devices, modules (parts), etc.), and not all components need to be provided in the same housing. In view of this, multiple devices housed in different housings and connected to each other via a network form a system, and a device that houses multiple modules in a single housing also forms a system.

[0583] Note that embodiments of the present technology are not limited to the above-described embodiments, and various modifications may be made to them without departing from the scope of the present technology.

[0584] For example, the present technology can be implemented in a cloud computing configuration in which one function is shared among multiple devices via a network, and processing is performed by the devices cooperating with each other.

[0585] In addition, the corresponding steps described with reference to the above flowcharts may be performed by one device or may be shared among a plurality of devices.

[0586] Furthermore, in the case where a plurality of processes are included in one step, the plurality of processes included in one step may be performed by one device or may be shared among a plurality of devices.

[0587] Meanwhile, the advantageous effects described in this specification are merely examples, and the advantageous effects of the present technology are not limited to them and may include other effects.

[0588] Note that the present technology can also be implemented in the configurations described below.

[0589] <1> A file processing device comprising

[0590] A file control unit that generates a file that stores

[0591] an encrypted image obtained by encrypting the image with a first encryption key, and

[0592] an encrypted encryption key obtained by encrypting the first encryption key with the second encryption key,

[0593] The encrypted image and the encrypted encryption key are associated with each other in the file.

[0594] <2> according to <1> The file processing device, wherein

[0595] The first encryption key in plain text is also stored in the file.

[0596] <3> according to <1> The file processing device, wherein

[0597] The encrypted encryption key is an encryption key obtained by encrypting specific information and the first encryption key with the second encryption key, the specific information being used to specify external data, which is data external to the file and associated with the image.

[0598] <4> according to <3> The file processing device, wherein

[0599] The external data is encrypted using a first encryption key.

[0600] <5> according to <3> or <4> The file processing device, wherein

[0601] Specific information in plain text and the first encryption key are also stored in the file.

[0602] <6> according to <1> or <2> The file processing device, wherein

[0603] The encrypted image is an encrypted divided image obtained by encrypting the divided images obtained by dividing the image with the first encryption key.

[0604] <7> according to <6> The file processing device, wherein

[0605] The encrypted divided images are encrypted with a first encryption key that varies with each subject appearing in the corresponding divided image, and

[0606] The first encryption key used in encryption of the divided image is encrypted with the second encryption key that changes for each subject.

[0607] <8> according to <6> or <7> The file processing device, wherein

[0608] The image is divided into segmented images according to the subjects captured in the image.

[0609] <9> according to <1> to <8> The file processing device according to any one of

[0610] The first encryption key is a common key of a common key encryption method.

[0611] The second encryption key is a public key for public key encryption, and

[0612] The file is a High Efficiency Image File Format (HEIF) file.

[0613] <10> A file processing method includes generating a file, the file storing

[0614] an encrypted image obtained by encrypting the image with a first encryption key, and

[0615] an encrypted encryption key obtained by encrypting the first encryption key with the second encryption key,

[0616] The encrypted image and the encrypted encryption key are associated with each other in the file.

[0617] <11> A file processing device comprising

[0618] The file control unit decrypts the encrypted image into an image using a first encryption key, wherein the first encryption key is obtained by decrypting the encrypted encryption key in the file, wherein the file is stored

[0619] an encrypted image obtained by encrypting the image with a first encryption key, and

[0620] an encrypted encryption key obtained by encrypting the first encryption key with the second encryption key,

[0621] The encrypted image and the encrypted encryption key are associated with each other in the file.

[0622] <12> according to <11> The file processing device, wherein:

[0623] When the first encryption key in plain text is also stored in the file, the file control unit decrypts the encrypted image into the image using the first encryption key in plain text, and

[0624] When the first encryption key in plain text is not stored in the file, the file control unit decrypts the encrypted encryption key into the first encryption key and decrypts the encrypted image into the image using the first encryption key obtained by decrypting the encrypted encryption key.

[0625] <13> according to <11> The file processing device, wherein

[0626] The encrypted encryption key is an encryption key containing specific information obtained by encrypting the specific information and the first encryption key with the second encryption key, the external data is data external to the file and associated with the image, and

[0627] The file control unit decrypts the encryption key containing the specific information into the specific information and the first encryption key, and acquires external data specified by the specific information obtained by decrypting the encryption key containing the specific information.

[0628] <14> according to <13> The file processing device, wherein:

[0629] When the external data is encrypted using the first encryption key, the file control unit decrypts the encrypted external data using the first encryption key.

[0630] <15> according to <13> or <14> The file processing device, wherein:

[0631] When specific information in plain text and the first encryption key are also stored in the file, the file control unit acquires external data specified by the specific information in plain text, and,

[0632] When the specific information and the first encryption key in plain text are not stored in the file, the file control unit decrypts the encryption key containing the specific information into the specific information and the first encryption key, and obtains external data specified by the specific information obtained by decrypting the encryption key containing the specific information.

[0633] <16> according to <11> or <12> The file processing device, wherein

[0634] The encrypted image is an encrypted divided image obtained by encrypting the divided image obtained by dividing the image with the first encryption key, and

[0635] The file control unit decrypts the encrypted encryption key into a first encryption key, and decrypts the encrypted divided image into the divided image using the first encryption key obtained by decrypting the encrypted encryption key.

[0636] <17> according to <16> The file processing device, wherein

[0637] The encrypted divided images are encrypted with a first encryption key that varies with each subject appearing in the corresponding divided image.

[0638] A first encryption key used in encryption of the divided image is encrypted with a second encryption key that changes for each subject, and

[0639] The file control unit decrypts an encrypted encryption key obtained by encrypting the first encryption key for the subject of interest into a first encryption key, and decrypts the encrypted segmented image showing the subject of interest into a segmented image using the first encryption key for the subject of interest obtained by decrypting the encrypted encryption key.

[0640] <18> according to <16> or <17> The file processing device, wherein

[0641] The image is divided into segmented images according to the subjects captured in the image.

[0642] <19> according to <11> to <18> The file processing device according to any one of

[0643] The first encryption key is a common key of a common key encryption method.

[0644] The second encryption key is a public key for public key encryption, and

[0645] The file is a High Efficiency Image File Format (HEIF) file.

[0646] <20> A file processing method, comprising:

[0647] The encrypted image is decrypted into an image using a first encryption key obtained by decrypting the encrypted encryption key in a file stored

[0648] an encrypted image obtained by encrypting the image with a first encryption key, and

[0649] an encrypted encryption key obtained by encrypting the first encryption key with the second encryption key,

[0650] The encrypted image and the encrypted encryption key are associated with each other in the file.

[0651] Reference Signs List

[0652] 10. Digital Camera

[0653] 11 Optical System

[0654] 13 Signal Processing Unit

[0655] 14 Medium

[0656] 15, 16 interfaces

[0657] 17 buttons / keys

[0658] 18 Touch Panel

[0659] 19 LCD panel

[0660] 20 Viewfinder

[0661] 21 Interface

[0662] 41 Optical system / image sensor control unit

[0663] 42 Decoding control unit

[0664] 43 File Control Unit

[0665] 44 Media Control Unit

[0666] 45 Operation control unit

[0667] 46 Display Control Unit

[0668] 47 UI control unit

[0669] 901 Bus

[0670] 902 CPU

[0671] 903 ROM

[0672] 904 RAM

[0673] 905 Hard Drive

[0674] 906 Output Unit

[0675] 907 Input Unit

[0676] 908 Communication Unit

[0677] 909 Driver

[0678] 910 Input / Output Interface

[0679] 911 Removable Recording Media

Claims

1. A file processing device, comprising a file control unit, which generates a file, the file storage an encrypted image obtained by encrypting the image with a first encryption key, and an encrypted encryption key obtained by encrypting the first encryption key with the second encryption key, The encrypted image and the encrypted encryption key are associated with each other in the file, The encrypted encryption key is an encryption key including specific information obtained by encrypting specific information and the first encryption key with the second encryption key, the specific information being used to specify external data, which is data external to the file and associated with the image.

2. The file processing device according to claim 1, wherein The first encryption key in plain text is also stored in the file.

3. The file processing device according to claim 1, wherein The external data is encrypted using a first encryption key.

4. The file processing device according to claim 1, wherein Specific information in plain text and the first encryption key are also stored in the file.

5. The file processing device according to claim 1, wherein The encrypted image is an encrypted divided image obtained by encrypting the divided images obtained by dividing the image with the first encryption key. The document processing device according to claim 5 , wherein The encrypted divided images are encrypted with a first encryption key that varies with each subject appearing in the corresponding divided image, and The first encryption key used in encryption of the divided image is encrypted with the second encryption key that changes for each subject.

7. The file processing device according to claim 5, wherein The image is divided into segmented images according to the subjects captured in the image.

8. The file processing device according to claim 1, wherein The first encryption key is a common key of a common key encryption method. The second encryption key is a public key for public key encryption, and The file is a High Efficiency Image File Format (HEIF) file.

9. A file processing method, comprising generating a file, the file storing an encrypted image obtained by encrypting the image with a first encryption key, and an encrypted encryption key obtained by encrypting the first encryption key with the second encryption key, The encrypted image and the encrypted encryption key are associated with each other in the file, The encrypted encryption key is an encryption key including specific information obtained by encrypting specific information and the first encryption key with the second encryption key, the specific information being used to specify external data, which is data external to the file and associated with the image.

10. A file processing device comprising The file control unit decrypts the encrypted image into an image using a first encryption key, the first encryption key being obtained by decrypting the encrypted encryption key in the file, the file storing an encrypted image obtained by encrypting the image with a first encryption key, and an encrypted encryption key obtained by encrypting the first encryption key with the second encryption key, The encrypted image and the encrypted encryption key are associated with each other in the file, The encrypted encryption key is an encryption key including specific information obtained by encrypting specific information and the first encryption key with the second encryption key, the specific information being used to specify external data, which is data external to the file and associated with the image.

11. The file processing device according to claim 10, wherein: When the first encryption key in plain text is also stored in the file, the file control unit decrypts the encrypted image into the image using the first encryption key in plain text, and When the first encryption key in plain text is not stored in the file, the file control unit decrypts the encrypted encryption key into the first encryption key and decrypts the encrypted image into the image using the first encryption key obtained by decrypting the encrypted encryption key.

12. The file processing device according to claim 10, wherein The file control unit decrypts the encryption key containing the specific information into the specific information and the first encryption key, and acquires external data specified by the specific information obtained by decrypting the encryption key containing the specific information.

13. The file processing device according to claim 12, wherein: When the external data is encrypted using the first encryption key, the file control unit decrypts the encrypted external data using the first encryption key.

14. The file processing device according to claim 12, wherein: When specific information in plain text and the first encryption key are also stored in the file, the file control unit acquires external data specified by the specific information in plain text, and, When the specific information and the first encryption key in plain text are not stored in the file, the file control unit decrypts the encryption key containing the specific information into the specific information and the first encryption key, and obtains external data specified by the specific information obtained by decryption of the encryption key containing the specific information.

15. The file processing device according to claim 10, wherein The encrypted image is an encrypted divided image obtained by encrypting the divided image obtained by dividing the image with the first encryption key, and The file control unit decrypts the encrypted encryption key into a first encryption key, and decrypts the encrypted divided image into the divided image using the first encryption key obtained by decrypting the encrypted encryption key.

16. The file processing device according to claim 15, wherein The encrypted divided images are encrypted with a first encryption key that varies with each subject appearing in the corresponding divided image. A first encryption key used in encryption of the divided image is encrypted with a second encryption key that changes for each subject, and The file control unit decrypts an encrypted encryption key obtained by encrypting a first encryption key for the object of interest into a first encryption key, and decrypts the encrypted segmented image showing the object of interest into a segmented image using the first encryption key for the object of interest obtained by decrypting the encrypted encryption key.

17. The file processing device according to claim 15, wherein The image is divided into segmented images according to the subjects captured in the image.

18. The file processing device according to claim 10, wherein The first encryption key is a common key of a common key encryption method. The second encryption key is a public key for public key encryption, and The file is a High Efficiency Image File Format (HEIF) file.

19. A file processing method comprising The encrypted image is decrypted into an image using a first encryption key obtained by decrypting the encrypted encryption key in a file stored an encrypted image obtained by encrypting the image with a first encryption key, and an encrypted encryption key obtained by encrypting the first encryption key with the second encryption key, The encrypted image and the encrypted encryption key are associated with each other in the file, The encrypted encryption key is an encryption key including specific information obtained by encrypting specific information and the first encryption key with the second encryption key, the specific information being used to specify external data, which is data external to the file and associated with the image.

Images