File processing device, file processing method and medium

By generating a proxy file in the efficient image file format HEIF file, the problem of low image index display efficiency in HEIF files is solved, and fast and efficient index display is achieved.

CN114128249BActive Publication Date: 2024-11-15SONY GROUP CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080052758.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-07-30
Filing Date
2020-07-16
Publication Date
2024-11-15
Estimated Expiration
2040-07-16

AI Technical Summary

Technical Problem

The prior art is difficult to quickly perform index display images in the HEIF file format, resulting in inefficient index display.

Method used

By generating image-related proxy files stored in the HEIF file format, the proxy file is another HEIF file for fast indexing display.

Benefits of technology

Fast index display is implemented, reducing the time required for index display, and avoiding redundant processing during index display, improving efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114128249B_ABST
    Figure CN114128249B_ABST
Patent Text Reader

Abstract

The present technology relates to a file processing device, a file processing method, and a program that enable quick execution of index display. The file control unit generates a proxy file, which is another HEIF file in which related images related to the images in the HEIF file are stored. In addition, the file control unit detects an image corresponding to a selected related image selected from the related images stored in the proxy file. The present technology can be applied to situations such as executing index display of thumbnail images, for example.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present technology relates to a file processing device, a file processing method and a program, and more particularly to a file processing device, a file processing method and a program that implement, for example, a quick index display. Background Art

[0002] As a file format for efficiently storing images, there is High Efficiency Image File Format (HEIF) (see Non-Patent Literature 1).

[0003] Reference List

[0004] Non-patent literature

[0005] Non-patent document 1: ISO / IEC 23008-12: 2017, Information technology--Highefficiency coding and media delivery in heterogeneous environments--Part 12:Image File Format Summary of the invention

[0006] Problems to be solved by the present invention

[0007] It is expected that HEIF files compliant with the High Efficiency Image File Format (HEIF) will be required to quickly perform index display.

[0008] The present technology is made in view of such circumstances and aims to realize fast index display.

[0009] Solution to the problem

[0010] The first file processing device or program of the present technology is a file processing device including a file control unit, or a program for causing a computer to be used as such a file processing device, wherein the file control unit generates a proxy file storing related images related to an image in a High Efficiency Image File Format (HEIF) file, the proxy file being another HEIF file.

[0011] A first file processing method of the present technology is a file processing method including generating a proxy file storing a related image related to an image in a High Efficiency Image File Format (HEIF) file, the proxy file being another HEIF file.

[0012] In a first file processing device, a file processing method, and a program of the present technology, a proxy file storing a related image related to an image in a High Efficiency Image File Format (HEIF) file is generated, the proxy file being another HEIF file.

[0013] The second file processing device or program of the present technology is a file processing device including a file control unit, or a program for causing a computer to be used as such a file processing device, wherein the file control unit detects an image in a High Efficiency Image File Format HEIF file, which corresponds to a selected related image selected from related images related to the image and is stored in a proxy file as another HEIF file.

[0014] The second file processing method of the present technology is a file processing method, including detecting an image in a High Efficiency Image File Format HEIF file, which corresponds to a selected related image selected from related images related to the image and is stored in a proxy file as another HEIF file.

[0015] In the second file processing device, file processing method and program of the present technology, an image in a high-efficiency image file format HEIF file is detected, which corresponds to a selected related image selected from related images related to the image and is stored in a proxy file as another HEIF file.

[0016] Note that the first and second file processing devices may be independent devices, or may be internal blocks forming one device.

[0017] Furthermore, the first and second programs may be provided by being recorded on a recording medium or by being transmitted via a transmission medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a block diagram illustrating a configuration example of an embodiment of a digital camera to which the present technology is applied.

[0019] Figure 2 is a view illustrating an example of the format of a Joint Photographic Experts Group (JPEG) file conforming to JPEG.

[0020] Figure 3 is a view illustrating an example of the ISO base media file format.

[0021] Figure 4 is a view illustrating a format example of a HEIF file conforming to HEIF.

[0022] Figure 5 is a view illustrating a format example of a HEIF file of an image item format.

[0023] Figure 6 is a view illustrating an example of an iprp box.

[0024] Figure 7 is a view illustrating an example of the format of a HEIF file in an image sequence format.

[0025] Figure 8 is a view illustrating an example of a trak box.

[0026] Fig. 9 is a view illustrating an example of a normal collection file in which a main image and thumbnail images are stored.

[0027] Fig.10 is a view illustrating an example of a first association type collection file.

[0028] Fig.11 is a view illustrating an example of a second association type collection file.

[0029] Fig.12 is a view illustrating an example of a third association type collection file.

[0030] Fig.13 is a view illustrating an example of a normal sequence file in which a track of a main image and a track of thumbnail images of the main image are stored.

[0031] Fig.14 is a view illustrating an example of an association type sequence file.

[0032] Fig.15 : is a flowchart for explaining an outline of an example of a generation process of generating an association type HEIF file.

[0033] Fig.16 : is a flowchart for explaining an outline of an example of a reproduction process of reproducing an association type HEIF file.

[0034] Fig.17 is a flowchart for explaining an example of a reproduction process of a reproduction set file.

[0035] Fig.18 : is a flowchart for explaining an example of a process of reading a reproduction target image in step S32.

[0036] Fig.19 is a flowchart for explaining a first example of the process of acquiring the reproduction target item ID in step S31.

[0037] Fig. 20 is a flowchart for explaining a second example of the process of acquiring the reproduction target item ID in step S31.

[0038] Fig.21 is a flowchart for explaining an example of a process of acquiring uuid of designation information of a RAW file which is a predetermined main image from a first association type set file.

[0039] Fig. 22is a flowchart for explaining an example of a process of acquiring uuid of designation information of a RAW file which is a predetermined main image from a second association type set file.

[0040] Fig.23 is a flowchart for explaining an example of a process of acquiring uuid of designation information of a RAW file which is a predetermined main image from a third association type set file.

[0041] Fig.24 is a flowchart for explaining an example of a process of acquiring an item ID list of a main image from a collection file.

[0042] Fig.25 is a flowchart for explaining an example of a process of reproducing thumbnail images of (frames of) a main image with respect to predetermined time information from a sequence file.

[0043] Fig.26 is a flowchart for explaining an example of a process of acquiring uuid of designation information of a RAW file which is a predetermined main image (frame) from an association type sequence file.

[0044] Fig. 27 is a view illustrating an example of storing uuid in a RAW file in a case where a RAW file of a main image is adopted as external data and an association type collection file is generated.

[0045] Fig.28 is a view illustrating an example of storing uuid into a RAW file in the case of adopting a RAW file of a main image as external data and generating an association type sequence file.

[0046] Fig.29 is a view illustrating an example of storing uuid into a WAV file in a case where a WAV file of a main image is adopted as external data and an associated type collection file is generated.

[0047] Fig.30 is a view illustrating an example of storing uuid into a WAV file in a case where a WAV file of a main image is adopted as external data and an associated type sequence file is generated.

[0048] Fig.31 1 is a view for explaining an example of index display on the digital camera 10 .

[0049] Fig.32 is a diagram for explaining an example of index display on an external device other than the digital camera 10 .

[0050] Fig.33 is a view illustrating an example of index display for a first association type collection file.

[0051] Fig.34 is a view illustrating another example of index display for a first association type collection file.

[0052] Fig.35 is a diagram illustrating an example of an index display for an association type sequence file.

[0053] Fig.36 is a view for explaining a first example of a proxy file.

[0054] Fig.37 is a view for explaining a second example of a proxy file.

[0055] Fig.38 is a view for explaining a third example of a proxy file.

[0056] Fig.39 is a view for explaining a fourth example of a proxy file.

[0057] Fig.40 is a view for explaining an example of writing uuid which is designation information of a main image which is external data to sequence file #1 which is a proxy target file.

[0058] Fig.41 This is a view for explaining an example of generating a proxy file using a first association type collection file as a proxy target file.

[0059] Fig.42 This is a view for explaining another example of generating a proxy file using a first association type collection file as a proxy target file.

[0060] Fig.43 This is a view for explaining an example of generating a proxy file by associating a type sequence file as a proxy target file.

[0061] Fig.44 A view for explaining the mode adopted by a proxy object file and a set of proxy files generated by a proxy file generation process executed for the proxy object file.

[0062] Fig.45 is a view illustrating a detailed example of the second mode adopted by a collection of a proxy target file and a proxy file.

[0063] Fig.46 is a view illustrating a detailed example of the third mode adopted by a collection of a proxy target file and a proxy file.

[0064] Fig.47 2 is a view for explaining an example of index display using a proxy file on the digital camera 10.

[0065] Fig.48 10 is a view for explaining an example of index display using a proxy file on an external device other than the digital camera 10.

[0066] Fig.49 is a view for explaining an example of selecting a HEIF file used as a proxy target file.

[0067] Fig.50 is a view for explaining another example of selecting a HEIF file used as a proxy target file.

[0068] Fig.51 It is a view for explaining a proxy file generation process for generating a proxy file of the first mode and an adding process for adding a thumbnail image to the proxy file.

[0069] Fig.52 is a flowchart for explaining an example of a proxy file generation process for generating a proxy file of the first mode.

[0070] Fig.53 is a flowchart for explaining an example of an adding process of adding a thumbnail image to a proxy file of the first mode.

[0071] Fig.54 It is a view for explaining a proxy file generation process for generating a proxy file of the second or third mode and an adding process for adding a thumbnail image to the proxy file.

[0072] Fig.55 is a flowchart for explaining an example of a proxy file generation process for generating a proxy file of the second or third mode.

[0073] Fig.56 is a flowchart for explaining an example of adding processing of adding a thumbnail image to a proxy file of the second or third mode.

[0074] Fig.57 is a view for explaining an example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file in the second mode.

[0075] Fig.58 is a view for further explaining an example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the second mode.

[0076] Fig.59 is a flowchart for explaining an example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file in the second mode.

[0077] Fig.60is a view for explaining an example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode.

[0078] Fig.61 is a view for further explaining an example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode.

[0079] Fig.62 is a view for explaining a first example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode.

[0080] Fig.63 is a flowchart for explaining a first example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file in the third mode.

[0081] Fig.64 is a view for explaining a second example of the detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode.

[0082] Fig.65 is a view for explaining a third example of the detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode.

[0083] Fig.66 is a view for explaining a fourth example of the detection process for detecting a main image corresponding to a thumbnail image stored in the proxy file of the third mode.

[0084] Fig.67 It is a view for explaining the proxy file generation processing and the adding processing performed in the case of generating a thumbnail image to be stored in the proxy file of the first mode when generating a proxy target file.

[0085] Fig.68 is a flowchart for explaining an example of a proxy file generation process performed in the case of generating a thumbnail image to be stored in a proxy file of the first mode when generating a proxy target file.

[0086] Fig.69 is a flowchart for explaining an example of adding processing performed in the case of generating a thumbnail image to be stored in a proxy file of the first mode when generating a proxy target file.

[0087] Fig.70 It is a view for explaining the proxy file generation processing and the adding processing performed in the case of generating a thumbnail image to be stored in the proxy file of the third mode when generating a proxy target file.

[0088] Fig.71 is a flowchart for explaining an example of a proxy file generation process performed in the case of generating a thumbnail image to be stored in a proxy file of the third mode when generating a proxy target file.

[0089] Fig.72 is a flowchart for explaining an example of adding processing performed in the case of generating a thumbnail image to be stored in a proxy file of the third mode when generating a proxy target file.

[0090] Fig.73 is a flowchart for explaining an example of proxy file generation processing in which an already existing HEIF file is used as a proxy target file.

[0091] Fig.74 : is a flowchart for explaining an example of adding processing in which an already existing HEIF file is used as a proxy target file.

[0092] Fig.75 is a block diagram illustrating a configuration example of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION

[0093] <One embodiment of a digital camera to which the present technology is applied>

[0094] Figure 1 is a block diagram illustrating a configuration example of an embodiment of a digital camera to which the present technology is applied.

[0095] The digital camera 10 includes an optical system 11 , an image sensor 12 , a signal processing unit 13 , a medium 14 , interfaces 15 and 16 , buttons / keys 17 , a touch panel 18 , a liquid crystal panel 19 , a viewfinder 20 , an interface 21 , and the like.

[0096] The optical system 11 condenses light from a subject on the image sensor 12 .

[0097] The image sensor 12 receives light from the optical system 11 and performs imaging of photoelectric conversion to generate image data as an electric signal and supplies it to the signal processing unit 13 .

[0098] The signal processing unit 13 includes an optical system / image sensor control unit 41 , an encoding control unit 42 , a file control unit 43 , a medium control unit 44 , an operation control unit 45 , a display control unit 46 , and a UI control unit 47 .

[0099] The optical system / image sensor control unit 41 controls the optical system 11 and the image sensor 12 , and supplies (data of) an image obtained by imaging performed according to the control to the encoding control unit 42 .

[0100] The encoding control unit 42 supplies the image from the optical system / image sensor control unit 41 to the display control unit 46, and encodes the image as needed and supplies it to the file control unit 43. Furthermore, the encoding control unit 42 decodes the image supplied from the file control unit 43 as needed and supplies it to the display control unit 46.

[0101] The file control unit 43 generates a file storing the image supplied from the encoding control unit 42, and supplies it to the medium control unit 44. In addition, the file control unit 43 reproduces the file supplied from the medium control unit 44, that is, reads data such as the image stored in the file, etc. For example, the image read from the file is supplied from the file control unit 43 to the encoding control unit 42.

[0102] The medium control unit 44 controls file exchange between the medium 14 and the interfaces 15 and 16. For example, the medium control unit 44 causes a file from the file control unit 43 to be recorded on the medium 14 or transferred from the interfaces 15 and 16. In addition, the medium control unit 44 reads a file from the medium 14 or causes the interfaces 15 and 16 to receive a file and supplies it to the file control unit 43.

[0103] The operation control unit 45 supplies an operation signal corresponding to the operation to a necessary block according to the user's operation on the button / key 17 or the touch panel 18 .

[0104] The display control unit 46 performs display control and the like to supply the image and the like supplied from the encoding control unit 42 to the liquid crystal panel 19 , the viewfinder 20 , and the interface 21 for display.

[0105] The UI control unit 47 manages user interface (UI) control.

[0106] The medium 14 is, for example, a storage medium such as an SD card. The interface 15 is, for example, an interface of a local area network (LAN) such as Wi-Fi (registered trademark) or Ethernet (registered trademark). The interface 16 is, for example, a universal serial bus (USB) interface. When a command or other information is input to the digital camera 10, the button / key 17 and the touch panel 18 are operated by the user. The touch panel 18 can be formed integrally with the liquid crystal panel 19. The liquid crystal panel 19 and the viewfinder 20 display images supplied from the display control unit 46, etc. The interface 21 is an interface for transmitting at least an image, such as a high-definition multimedia interface (HDMI) (registered trademark) or a display port (DP).

[0107] In the digital camera 10 configured as described above, the optical system / image sensor control unit 41 generates, for example, a YUV image having the same resolution (number of pixels) as that of the RAW image as the main image of the HEIF file based on an image of RAW data obtained by imaging by the image sensor 12 (hereinafter, also referred to as a RAW image), and supplies it to the encoding control unit 42.

[0108] The encoding control unit 42 generates, for example, a YUV image (hereinafter, also referred to as a screen nail image) having a lower resolution than the main image from the YUV main image for display on the liquid crystal panel 19 or an external display as a first other image based on the main image, and generates, for example, a YUV image (hereinafter, also referred to as a thumbnail image) having a resolution lower than that of the screen nail image for index display (list display) as a second other image based on the main image. For example, the encoding control unit 42 supplies the screen nail image to the liquid crystal panel 19 via the display control unit 46 to be displayed as a so-called through image. As the thumbnail image, for example, an image having a size of 320 pixels or less on the long side can be used. For example, the ratio of the size (number of pixels) between the main image and the screen nail image as the first other image based on the main image or the thumbnail image as the second other image based on the main image can be, for example, 200 times or less. Similarly, the ratio of the size between the main image and the screen nail image as the first other image based on the main image and the thumbnail image as the second other image based on the main image can also be 200 times or less. As a screen nail image, for example, an image with a resolution of 4K or higher can be used. In addition, as a screen nail image, for example, a 4K (QFHD) or FHD image can be used according to the user's selection. Moreover, an image with the same resolution can be used as the main image and the screen nail image. In the case of using images with the same resolution as the main image and the screen nail image, both the main image and the screen nail image can be stored in a HEIF file, or the main image can be stored without storing the screen nail image. In the case where the main image is stored in a HEIF file without storing the screen nail image, the size of the main image can be adjusted and used as a screen nail image.

[0109] Furthermore, the encoding control unit 42 encodes the main image, the screen nail image, and the thumbnail images corresponding to the RAW image (the main image, the screen nail image, and the thumbnail images generated from the same RAW image) when necessary, and supplies them to the file control unit 43 together with the RAW image.

[0110] The file control unit 43 generates a RAW file that stores the RAW image, and generates a HEIF file, a JPEG file, etc. that store the corresponding main image, screen thumbnail image, and thumbnail image (the main image, screen thumbnail image, and thumbnail image generated from the same RAW image), and supplies them to the medium control unit 44. The HEIF file is a file compliant with the High Efficiency Image File Format (HEIF), and the JPEG file is a file compliant with the Joint Photographic Experts Group (JPEG).

[0111] The medium control unit 44 records the RAW file and the HEIF file or JPEG file from the file control unit 43 on the medium 14, or causes the RAW file and the HEIF file or JPEG file to be transmitted from the interface 15 or 16.

[0112] In the file control unit 43, for example, it is possible to select which of the HEIF file and the JPEG file to generate according to the user's operation. In addition, although the HEIF file includes the image item format and the image sequence format described later, it is possible to select which of the image item format and the image sequence format to adopt according to the user's operation. Moreover, the file control unit 43 can perform mutual conversion between the HEIF file and the JPEG file according to the user's operation.

[0113] Moreover, when generating the HEIF file, the file control unit 43 can associate the internal data in the HEIF file (the data stored in the HEIF file, which is in the HEIF file and is to be associated with the external data (the data not stored in the HEIF file)) with the specified information that specifies the external data, and store it in the HEIF file. The HEIF file in which the internal data and the specified information of the external data to be associated with the internal data are stored in association with each other is also called an associated type HEIF file. The associated type HEIF file can store the internal data and the specified information in association with each other, for example, by storing the association information that associates the internal data and the specified information.

[0114] <JPEG File>

[0115] Figure 2 It is a view showing an example of the format of a file compliant with the Joint Photographic Experts Group (JPEG) of JPEG.

[0116] The JPEG file is configured by storing, for example, Exif metadata, a thumbnail image, and Extensible Metadata Platform (XMP) (registered trademark) metadata, MPF indicating the storage location (position) of the main image and the simple display image, the main image, and the simple display image. As the simple display image, for example, a screen thumbnail image can be adopted.

[0117] <ISO Base Media File Format>

[0118] Figure 3 It is a view showing an example of the ISO base media file format.

[0119] HEIF (ISO / IEC 23008-12) is a file format that conforms to the ISO base media file format (ISO / IEC 14496-12), so HEIF files conform to the ISO base media file format.

[0120] The ISO base media file format is formed with units called boxes as containers for storing data and has a structure called a box structure.

[0121] A box includes a type (box type), actual data (data), etc. The type indicates the type of the actual data in the box. As the actual data, it is possible to adopt reproducible media data such as images (still images or moving images), audio and subtitles, attribute names (field names), and attribute values (field values) of (variables represented by) attribute names, as well as various other data.

[0122] Moreover, a box can be adopted as the actual data. That is, a box can have a box as the actual data, which makes a hierarchical structure possible.

[0123] A basic media file that conforms to the ISO base media file format can have an ftyp box, a moov box (MovieBox), a meta box (MetaBox), an mdat box (MediaDataBox), etc. The ftyp box stores identification information for identifying the file format. The moov box can store trak boxes, etc. The meta box can store iinf boxes, iprp boxes, iref boxes, iloc boxes, etc. The mdat box can store media data (AV data) and any other data.

[0124] HEIF conforms to the above ISO base media file format.

[0125] <HEIF file>

[0126] Figure 4 It is a view showing an example of the format of a HEIF file that conforms to HEIF.

[0127] HEIF files are roughly divided into an image item format and an image sequence format. Moreover, the image item format includes a single image format with only one item and an image set format with multiple items, which will be described later.

[0128] A HEIF file in the image item format has an ftyp box, a meta box, and an mdat box.

[0129] HEIF files in image sequence format have ftyp box, moov box and mdat box.

[0130] Note that HEIF files can also have both a meta box and a moov box, not just one or the other.

[0131] The ftyp box stores identification information for identifying the format of a file, such as, for example, the fact that the file is a HEIF file in the image item format or the image sequence format.

[0132] The meta box and the moov box store metadata necessary for reproduction, management, etc. of the media data stored in the mdat box, for example, metadata such as the storage location of the media data.

[0133] The mdat box stores media data (AV data) and the like.

[0134] For example, in the digital camera 10, which HEIF file of the HEIF file in the image item format and the image sequence format is to be generated can be selected according to the user's operation. In addition, when the image is encoded and stored in the mdat box of the HEIF file, the image item format only allows intra coding, while the image sequence format allows intra coding and inter coding. Therefore, for example, in the case where priority is given to high-speed access to data stored in the HEIF file, it is possible to choose to generate a HEIF file in the image item format. And in the case where priority is given to reducing the size (data amount) of the HEIF file, it is possible to choose to generate a HEIF file in the image sequence format.

[0135] Figure 5 is a view illustrating an example of the format of a HEIF file of an image item format.

[0136] The HEIF file in the image item format stores information indicating that the HEIF file is in the image item format, such as mif1, etc. (as an attribute value) in the ftyp box.

[0137] The meta box stores the iinf box, iref box, iprp box, and iloc box.

[0138] The iinf box stores (indicates) the number of items (attribute names and attribute values) such as media data (AV data) stored in the mdat box. The item is one data stored in the mdat box of the HEIF file in the image item format. For example, one image (screen) is an item. In this specification, whether it is a still image or a moving image, one image is also called a frame. One frame is one item.

[0139] The iref box stores information indicating the relationship between items. For example, in the mdat box, each of the corresponding main image, screen nail image, and thumbnail image may be stored as an item. In the case where the mdat box stores that item I1 is the main image, item I2 is the screen nail image, and item I3 is the thumbnail image, the iref box stores information indicating that item I2 is the screen nail image as the main image of item I1 and information indicating that item I3 is the thumbnail image as the main image of item I1.

[0140] The iprp box stores information about the properties of an item.

[0141] The iloc box stores information about the storage location of the items stored in the mdat box.

[0142] The mdat box (of a HEIF file) in the image item format stores, for example, frames of an image as items. In the mdat box, one or more items can be stored. In addition, the frames as items can be encoded and stored in the mdat box. However, the encoding of the frames as items stored in the mdat box in the image item format is limited to intra-frame encoding. As a coding method (codec) for encoding frames as items, for example, HEVC, etc. can be used.

[0143] Figure 6 It is a graphic Figure 5 A view of an example of an iprp box.

[0144] The iprp box stores the ipco box and the ipma box related to the attributes of the item. The ipco box stores the attributes of the item stored in the mdat box, for example, codec information about the codec of the image as the item and image size information about the size. The ipma box stores the index (pointer) to the property stored in the ipco box of the item stored in the mdat box.

[0145] Figure 7 is a view illustrating an example of the format of a HEIF file in an image sequence format.

[0146] A HEIF file in an image sequence format stores information indicating that the HEIF file is in an image sequence format, such as msf1, in an ftyp box.

[0147] The moov box stores the trak box. The trak box stores information about the track stored in the mdat box.

[0148] A track includes a piece of independent media data, such as an image or audio, to be reproduced according to a timeline. For example, a track includes an image that will become one or more frames of an elementary stream. For tracks stored in an mdat box, multiple tracks can be reproduced simultaneously, such as individual tracks of images and audio recorded simultaneously.

[0149] The media data of a track is formed in units called samples. A sample is the minimum unit (access unit) when accessing media data in a HEIF file. Therefore, the media data in a HEIF file cannot be accessed in units finer than a sample.

[0150] For example, for image media data, one frame is one sample. Also, for audio media data, for example, one audio frame defined in the standard of audio media data is one sample.

[0151] In the mdat box (of the HEIF file) of the image sequence format, the media data of the track is arranged in units called blocks. A block is a collection of one or more samples arranged at logically consecutive addresses.

[0152] In the case where a plurality of tracks as media data are stored in the mdat box, the plurality of tracks are interleaved and arranged in units of blocks.

[0153] As described above, in the mdat box of the image sequence format, one or more tracks including media data such as images and audio are stored.

[0154] In the mdat box, the image frames constituting the track can be encoded and stored. For the encoding of the frames constituting the track stored in the mdat box of the image sequence format, a long group of pictures (GOP) can be used as the GOP, and either intra-frame encoding or inter-frame encoding can be used. As a codec for encoding the frames constituting the track, for example, HEVC can be used.

[0155] Figure 8 is a view illustrating an example of a trak box.

[0156] The trak box can store a tkhd box and an mdia box. The tkhd box stores header information of the track, such as the creation date and time of the pipeline managed by the trak box. The mdia box stores a minf box, etc. The minf box stores a stbl box. The stbl box stores a stsd box, a stsc box, a stsz box, and a stco box, which store samples of the track and thus store information for accessing blocks. The stsd box stores codec information about the codec of the track. The stsc box stores a block size (the number of samples of one block). The stsz box stores a sample size. The stco box stores a block offset, that is, an offset of the arrangement position of each block of the track stored in the mdat box.

[0157] Here, the HEIF file in the image item format is also referred to as a collection file, and the HEIF file in the image sequence format is also referred to as a sequence file. Moreover, the associated type HEIF file in the image item format is also referred to as an associated type collection file, and the associated type HEIF file in the image sequence format is also referred to as an associated type sequence file.

[0158] In the digital camera 10, it is possible to generate a HEIF file (including an associated type HEIF file) that stores a main image, and also stores one or both of necessary screen nail images and thumbnail images.

[0159] <Collection File>

[0160] Fig. 9 is a view illustrating an example of a normal collection file in which a main image and thumbnail images are stored.

[0161] Here, a normal collection file refers to a collection file in which internal data in the collection file is not associated with designated information of external data.

[0162] Now, assume that the frames (items) are encoded by HEVC and stored in the mdat box of the collection file.

[0163] The ftyp box stores heic indicating that the format is the image item format and the codec is HEVC, as identification information for identifying the file format.

[0164] The iinf box stores the number of items stored in the mdat box (number of items). Fig. 9 In the mdat box, a total of four items (frames) are stored: a main image specified by item ID #1 (hereinafter, also referred to as main image Item #1); main image Item #2; a thumbnail image specified by item ID #101 (hereinafter, also referred to as thumbnail image Item #101); and thumbnail image Item #102. Therefore, the number of items is 4. Note that thumbnail image Item #101 is a thumbnail image of main image Item #1, and thumbnail image Item #102 is a thumbnail image of main image Item #2.

[0165] Furthermore, the iinf box also stores an infe box for each item stored in the mdat box. In the infe box, an item ID and an item type for specifying an item are registered. Fig. 9 In the image, there is an infe box for each of the main images Item#1 and Item#2 and the thumbnail images Item#101 and Item#102.

[0166] The iref box stores, for example, a thmb box as information for associating items stored in the mdat box. In the thmb box, a reference source and a reference destination, which are information for associating a main image with a thumbnail image of the main image, are associated with each other and stored. In the thmb box, the reference source represents the item ID of the main image, and the reference destination represents the item ID of the thumbnail image of the main image specified by the item ID of the reference source. Therefore, based on the reference destination associated with the reference source, the item ID of the thumbnail image of the main image specified by the item ID represented by the reference source can be identified. In addition, based on the reference source associated with the reference destination, the item ID of the main image of the thumbnail image specified by the item ID represented by the reference destination can be identified.

[0167] like Figure 6 As described in , the ipco box and ipma box are stored in the iprp box. Figure 6 As described in , the ipco box stores the characteristics of the frame as items stored in the mdat box, for example, codec information about the codec and image size information about the size. Figure 6 As described in, the ipma box stores references to the properties stored in the ipco box for the items stored in the mdat box.

[0168] like Figure 6 As described in , the iloc box stores information about the storage locations of items in the mdat box. Fig. 9 In the iloc box, the number of items is stored as 4. Also, in the iloc box, the offset and size to each storage location of the main image Item#1 and Item#2 and thumbnail image items Item#101 and Item#102 IDs stored in the mdat box are stored in association with the item IDs.

[0169] In the following, it will be described Fig. 9 An associative collection file that stores designated information of internal data and external data in association with each other in a normal collection file.

[0170] Fig.10 is a view illustrating an example of a first association type collection file.

[0171] Here, it is assumed that as external data to be associated with a main image as internal data in a HEIF file, for example, (a RAW file in which) a RAW image of the main image is adopted (stored).

[0172] In the first association type set file, by storing association information associating the main image as internal data with the designated information of the RAW file (RAW image stored in) storing the RAW file as external data, the main image and the designated information of the RAW file storing the RAW image are stored in association with each other. Also, in the first association type set file, the association information is stored in a meta box.

[0173] As specifying information of a RAW file that stores a RAW image as external data, in addition to the file name of the RAW file, the universally unique identifier (uuid) and uniform resource locator (URL) published for the RAW file, it is possible to adopt any information that can specify the RAW file (the RAW image stored therein).

[0174] For the first association type set file, as a new box to be stored in the meta box, an association information storage box storing association information is defined and stored in the meta box. The association information storage box of the first association type set file stores association information in which, for example, an item ID for specifying a main image is associated with a uuid as designation information specifying a RAW file (RAW image stored in) to be associated with the main image. Moreover, the association information storage box stores the number of main images (the number of main images) to be associated with the RAW file (RAW image stored in). The number of main images stored in the association information storage box is the number of main images associated with the RAW file, and therefore is a value equal to or less than the number of main images stored in the mdat box.

[0175] exist Fig.10 In the example, the uuid of the RAW file of the main image Item#1 (the uuid of the RAW image associated with the main image Item#1) is UUID#1, and the uuid of the RAW file of the main image Item#2 is UUID#2. Now, assuming that the RAW file whose uuid is UUID#i is described as RAW file UUID#i, in Fig.10 In the associated information storage box, associated information is stored, in which item ID#1 of main image Item#1 is associated with the uuid of RAW file UUID#1, and item ID#2 of main image Item#2 is associated with the uuid of RAW file UUID#2.

[0176] Fig.11 is a view illustrating an example of a second association type collection file.

[0177] In the second association type set file, similar to the first association set file, by storing association information associating the main image as internal data with the designated information of the RAW file as external data, the main image and the designated information of the RAW file are stored in association with each other. However, in the second association type set file, the association information is stored in the mdat box.

[0178] For the second associative collection file, for example, associative information similar to that of the first associative collection file is stored as an item in the mdat box. Fig.11 In the example, the associated information is stored in the mdat box as an item with item ID #201.

[0179] As described above, in the second association type set file, in response to storing the association information as item Item#201 in the mdat box, the information stored in the meta box is the same as Fig. 9 In the second associative collection file, metadata as associative information of item Item#201 is stored in the meta box.

[0180] Specifically, in the second association type collection file, the number of items stored in the iinf box and the iloc box is 5, which is from Fig. 9 In the case of 4 plus 1 of item Item#201, it is obtained. Moreover, the infe box for item Item#201 is added to the iinf box, and the offset and size of the storage position of item Item#201 are added to the iloc box. The infe box for item Item#201 stores the item ID#201 of item Item#201 and item type identification data information (IDIF) indicating that item Item#201 is associated information. IDIF is a newly defined attribute value (field value) indicating that the item is associated information.

[0181] Fig.12 is a view illustrating an example of a third association type collection file.

[0182] In the third association type set file, by storing the designated information of the RAW file as external data in the mdat box as an item of each designated information, and storing the associated information associating the main image as internal data with the designated information of the RAW file as external data in the meta box, the main image and the designated information of the RAW file are stored in association with each other. However, in the third association type set file, the associated information is information in which the item ID of the main image as an item is associated with the item ID of the designated information (of the RAW file) as an item, and is stored in the cdsc box stored in the iref box in the meta box.

[0183] The cdsc box can store a reference source and a reference destination as information for associating a main image with items of respective pieces of designated information in a RAW file as the main image. In the cdsc box, the reference source represents the item ID of the main image, and the reference destination represents the item ID of the designated information of the item of the RAW file as the main image designated by the item ID of the reference source.

[0184] exist Fig.12 , UUID#1 which is the uuid of the designated information of the RAW file of the main image Item#1 is stored in the mdat box as item Item#201, and UUID#2 which is the uuid of the designated information of the RAW file of the main image Item#2 is stored in the mdat box as item Item#202. Moreover, the iref box stores a cdsc box storing association information in which the item ID#1 of the main image Item#1 and the item ID#201 of the designated information UUID#1 are associated with each other as a reference source and a reference destination, respectively, and the iref box stores a cdsc box storing association information in which the item ID#2 of the main image Item#2 and the item ID#202 of the designated information UUID#2 are associated with each other as a reference source and a reference destination, respectively.

[0185] <sequence file>

[0186] Fig.13 is a view illustrating an example of a normal sequence file in which a track of main images and a track of thumbnail images of the main images are stored.

[0187] Here, a normal sequence refers to a sequence file in which internal data in the sequence file is not associated with designated information of external data.

[0188] Now, assume that the frames are encoded by HEVC and stored in the mdat box of the sequence file.

[0189] The ftyp box stores hevc indicating that the format is an image sequence format and the codec is HEVC as identification information for identifying the file format.

[0190] like Figure 7 As described in , the moov box stores and manages the trak boxes of each track stored in the mdat box. Fig.13 In the moov box, the mdat box stores a track of the main image specified by track ID #1 (hereinafter, also referred to as track #1) and track #2 of the thumbnail image of the main image specified by track #1. Therefore, in the moov box, a trak box managing track #1 and a trak box managing track #2 are stored. The nth thumbnail (frame) of track #2 (from the head) is a thumbnail image of the nth main image of track #1.

[0191] For example, in the case where continuous shooting is performed by the digital camera 10 , the sequence file is useful in recording main images and thumbnail images of a plurality of frames obtained by continuous shooting as one track or the like.

[0192] The tkhd box of the trak box of track #1 managing the main images stores: track ID #1 specifying track #1; the image size of the main image constituting track #1; rotation information indicating the orientation of the digital camera 10 when the main image was captured; and the creation date and time of track #1. In the tkhd box of the trak box of track #2 managing the thumbnail images, track ID #2 specifying track #2 and the creation date and time of track #2 are stored.

[0193] In the trak box, in addition to Figure 7 In addition to the tkhd box and the mdia box described in , a tref box may be stored. The tref box stores a track ID for specifying another track associated with the track managed by the trak box in which the tref box is stored, information indicating the content of the track, and the like. Fig.13 In the example, the tref box is provided in the trak box managing track #2. Then, the tref box stores information indicating that another track associated with track #2 is track #1 (track_ID=1) and that data constituting track #2 is a thumbnail image (track #2 is a track of thumbnail images) (type=thmb).

[0194] In the trak box's mdia box, in addition to Figure 8 In addition to the minf box described in , an hdlr box may be stored. The hdlr box stores information indicating the type of data constituting the track managed by the trak box in which the hdlr box is stored. In the hdlr box stored in the trak box (stored in the mdia box) of track #1 managing the main image, information (pict) indicating that the data constituting track #1 is a picture (frame) is stored. In the hdlr box stored in the trak box of track #2 managing the thumbnail image, information indicating that the data constituting track #2 is a picture is stored.

[0195] minf box Figure 8 As described in.

[0196] In the following, it will be described Fig.13 An association type sequence file that stores designated information of internal data and external data in association with each other in a normal sequence file.

[0197] Fig.14 is a view illustrating an example of an association type sequence file.

[0198] In the associated type sequence file, track #3 of the (elementary) stream (Meta ES) of the uuid of the designated information of the RAW file as external data is added to the mdat box, and the trak box managing the track #3 is added to the moov box.

[0199] Here, track #1 is a time sequence of one or more frames of the main image aligned on the timeline, and track #3 is a time sequence of uuids of the RAW files of each frame of the main image aligned on the timeline.

[0200] The nth uuid (starting from the head) of track #3 is the designated information of the RAW file of the frame of the nth main image of track #1. In addition, multiple tracks (data) stored in the mdat box can be synchronously reproduced according to the time information on a timeline. Therefore, by storing the uuid (stream) of the RAW file of track #1 of the main image and track #3 of each frame of the main image constituting track #1 in the mdat box, the frame of the nth main image of track #1 and the uuid of the RAW file of the main image (frame) are stored in association with each other. In this case, it can be said that the uuid of the frame of the main image of track #1 and the RAW file of the main image (frame) are associated with each other through the time information on the timeline.

[0201] Note that the nth uuid (starting from the head) of track #3 is the designation information of the RAW file of the nth frame of track #1, and it can also be understood that the uuids of the main image (frame) constituting track #1 and the uuids constituting track #3 are associated with each other according to the arrangement order in the track.

[0202] In the associated type sequence file, in response to track #3 of the uuid of the RAW file being added to the mdat box, a trak box managing track #3 is added to the moov box.

[0203] In the trak box of track #3 that manages the uuid of the RAW file, a tkhd box, a tref box, an mdia box, and the like are stored.

[0204] In the tkhd box of the trak box managing track #3, track ID #3 specifying track #3 and the creation date and time of track #3 are stored.

[0205] The tref box of the trak box managing track #3 stores a track ID for specifying another track associated with track #3 managed by the trak box in which the tref box is stored, information indicating the contents of track #3, etc. The uuid constituting track #3 is the designation information of the RAW file of the main image constituting track #1. Since track #3 is associated with track #1, Fig.14The tref box of the trak box that manages track #3 stores information indicating that another track associated with track #3 is track #1 (track_ID = 1) and that track #3 is a track of metadata (here, specified information) (type = cdsc).

[0206] In the mdia box of the trak box that manages track #3, an hdlr box and a minf box are stored. In the trak box that manages track #3, the hdlr box stores information indicating that the data constituting track #3 is metadata (of the main image), and the minf box stores an stsc box, a stsc box, an stsz box, and an stco box for track #3.

[0207] <Generation and Reproduction of HEIF Files>

[0208] Fig.15 It is a flowchart outlining an example of a generation process for generating an associated type HEIF file.

[0209] In the generation process, in step S11, the file control unit 43 generates a uuid as the specified information for the RAW file (RAW image) of the frame of the main image, and the process proceeds to step S12.

[0210] In step S12, the file control unit 43 assigns the uuid generated in step S11 to the RAW file (RAW image) of the frame of the main image, and the process proceeds to step S13.

[0211] In step S13, the file control unit 43 generates an associated type HEIF file in which the frame of the main image and the uuid of the RAW file of that frame are stored in association with each other in the HEIF file, and the generation process ends.

[0212] Fig.16 It is a flowchart outlining an example of a reproduction process for reproducing an associated type HEIF file.

[0213] In the reproduction process, in step S21, the file control unit 43 generates, for example, a handle list for identifying the handles of the respective frames of the main image stored in the HEIF file stored in the medium 14, and the process proceeds to step S22.

[0214] Here, the handle of the frame of the main image includes the file name of the HEIF file in which the frame is stored. Moreover, the handle of the frame (item) of the main image stored in the collection file also includes the item ID of the frame. Moreover, the handle of the frame of the main image stored in the sequence file also includes the time information of the frame. Based on the handle of the frame of the main image, the frame corresponding to the handle can be uniquely identified (specified).

[0215] Note that the handle of the frame of the main image stored in the sequence file may include the track ID of the track containing the frame and the order of the frame in the track (what number the frame is), rather than the time information of the frame.

[0216] Regardless of whether one or more tracks including the frame of the main image are stored in the sequence file, the time information of each frame is unique. Therefore, based on the time information of the frame, even if multiple tracks are stored in the sequence file, the frame of the time information included in the handle can be uniquely specified from the frames constituting each of the multiple tracks. Therefore, in the case where the time information of the frame is included in the handle of the frame of the main image, even if there is no track ID of the track where the frame is located, the frame corresponding to the time information can be uniquely specified.

[0217] The handle list may be generated for all frames of the main image stored in the HEIF file stored in the medium 14, or may be generated only for frames that are reduced under certain conditions, such as frames of a certain creation date and time.

[0218] After generating the handle list, the file control unit 43 accesses the HEIF file with reference to the handle list when necessary.

[0219] In step S22, for example, after waiting for the user to operate the digital camera 10 to display the thumbnail image, the UI control unit 47 requests the file control unit 43 to display the thumbnail image. In response to the thumbnail image display request from the UI control unit 47, the file control unit 43 reads the thumbnail image (frame) of the frame of the main image identified by the handle of the handle list from the HEIF file. Then, the file control unit 43, for example, causes the liquid crystal panel 19 ( Figure 1 ) displays a list of thumbnail images read from the HEIF file, and processing proceeds from step S22 to step S23.

[0220] In step S23, for example, after waiting for the user to select a desired thumbnail (frame) in the thumbnail image list, the UI control unit 47 requests the file control unit 43 to transmit a main image corresponding to the thumbnail image selected by the user. The file control unit 43 reads the main image from the HEIF file in response to the request for the main image from the UI control unit 47. If necessary, the file control unit 43 can cause the liquid crystal panel 19 to display the main image read from the HEIF file.

[0221] Alternatively, the UI control unit 47 requests the uuid of the RAW file of the main image corresponding to the thumbnail image selected by the user from the file control unit 43. In response to the uuid request from the UI control unit 47, the file control unit 43 reads the uuid from the associated type HEIF file. If necessary, the file control unit 43 can access the RAW file specified by the uuid read from the associated type HEIF file.

[0222] Fig.17 is a flowchart for explaining an example of a reproduction process of a reproduction set file.

[0223] In step S31 , the file control unit 43 acquires the item ID of the reproduction target image which is the image (item) to be reproduced (hereinafter, also referred to as the reproduction target item ID), and the process proceeds to step S32 .

[0224] In acquiring the reproduction target item ID, the item ID of the reproduction target image (reproduction target item ID) is acquired by taking as the reproduction target image a main image identified by, for example, any handle in a handle list, a thumbnail image of the main image, a thumbnail image selected by a user from a list of thumbnail images (hereinafter, also referred to as a selected thumbnail image), a main image of the selected thumbnail image, and the like.

[0225] In step S32 , the file control unit 43 reads the reproduction target image according to the reproduction target item ID acquired in step S31 .

[0226] In reading of the reproduction target image, the reproduction target image specified by the reproduction target item ID is read from the set file.

[0227] Fig.18 It is used to explain Fig.17 Flowchart of an example of a process of reading a reproduction target image in step S32 of FIG.

[0228] In step S41, the file control unit 43 collects files ( Figures 9 to 12 )'s iloc box, and the processing proceeds to step S42.

[0229] In step S42, the file control unit 43 reads the offset and size associated with the reproduction target item ID searched in step S41 in the iloc box, and the process proceeds to step S43.

[0230] In step S43, the file control unit 43 reads the reproduction target image stored in the mdat box of the set file according to the offset and size associated with the reproduction target item ID, and the processing ends.

[0231] Fig.19is used to explain Fig.17 Flowchart of a first example of the process of acquiring the reproduction target item ID in step S31 of FIG.

[0232] Right now, Fig.19 An example of acquiring the item ID of a thumbnail image as a reproduction target image is illustrated.

[0233] Note that in Fig.19 In the above description, it is assumed that the file control unit 43 has recognized the item ID of the main image which is the thumbnail image of the reproduction target image from the handle, for example.

[0234] In step S51, the file control unit 43 collects the files ( Figures 9 to 12 ) in the thmb box of the iref box searches for a thmb box whose reference source matches the item ID of the main image, and the process proceeds to step S52.

[0235] In step S52, as the item ID of the thumbnail which is the reproduction target image, the file control unit 43 reads the reference destination in the thmb box searched in step S51 and the reference source having the item ID matching the main image, and the processing ends.

[0236] Fig. 20 is used to explain Fig.17 Flowchart of a second example of the process of acquiring the reproduction target item ID in step S31 of FIG.

[0237] Right now, Fig. 20 An example of acquiring the item ID of a main image as a reproduction target image is illustrated.

[0238] Note that in Fig. 20 In the description, for example, it is assumed that the user has selected a thumbnail image (selected thumbnail image) from the list of thumbnail images, and the file control unit 43 has recognized the item ID of the selected thumbnail image.

[0239] In step S61, the file control unit 43 collects the files ( Figures 9 to 12 )'s thmb box in the iref box is searched for a thmb box whose reference destination matches the item ID of the selected thumbnail, and the processing proceeds to step S62.

[0240] In step S62, the file control unit 43 reads the reference source in the thmb box searched in step S61 as the item ID of the main image of the reproduction target image and has a reference destination matching the item ID of the selected thumbnail image, and the processing ends.

[0241] Fig.21 is used to explain Fig.10 A flowchart of an example of a process of acquiring uuid of designation information of a RAW file which is a predetermined main image in a first association type collection file.

[0242] Note that in Fig.21 In the example, it is assumed that the file control unit 43 has already recognized the item ID of a predetermined main image through a handle list or the like.

[0243] In step S71, the file control unit 43 collects files of the first association type ( Fig.10 ) is searched for the item ID of the predetermined main image in the associated information in the associated information storage box, and the processing proceeds to step S72.

[0244] In step S72 , the file control unit 43 reads the uuid associated with the item ID of the predetermined main image searched in step S71 in the associated information, and the process ends.

[0245] The file control unit 43 can access the RAW file of the predetermined main image through the uuid read as described above.

[0246] Fig. 22 is used to explain Fig.11 A flowchart of an example of a process of acquiring uuid of designation information of a RAW file which is a predetermined main image in a second association type set file.

[0247] Note that in Fig. 22 In the example, it is assumed that the file control unit 43 has already recognized the item ID of a predetermined main image through a handle list or the like.

[0248] In step S81, the file control unit 43 generates a second association type set file ( Fig.11 ) searches for an infe box of item type IDIF indicating that the item is associated information, and the processing proceeds to step S82.

[0249] In step S82, the file control unit 43 reads the item ID of the associated information as an item from the infe box of the item type IDIF searched in step S81, and the process proceeds to step S83.

[0250] In step S83, the file control unit 43 searches the iloc box of the second association type set file for the item ID of the association information read in step S82, and the process proceeds to step S84.

[0251] In step S84 , the file control unit 43 reads the offset and size associated with the item ID of the associated information searched in step S83 in the iloc box, and the process proceeds to step S85 .

[0252] In step S85, the file control unit 43 reads association information as an item stored in the mdat box of the second association type set file based on the offset and size associated with the item ID of the association information read in step S84, and the process proceeds to step S86.

[0253] In step S86 , the file control unit 43 searches for the item ID of the predetermined main image from the associated information read in step S85 , and the process proceeds to step S87 .

[0254] In step S87 , the file control unit 43 reads uuid associated with the item ID of the predetermined main image searched in step S86 in the association information, and the process ends.

[0255] The file control unit 43 can access the RAW file of the predetermined main image through the uuid read as described above.

[0256] Fig.23 is used to explain Fig.12 A flowchart of an example of a process of acquiring uuid of designation information of a RAW file which is a predetermined main image in a third association type set file.

[0257] Note that in Fig.23 In the example, it is assumed that the file control unit 43 has already recognized the item ID of a predetermined main image through a handle list or the like.

[0258] In step S91, the file control unit 43 generates a third association type set file ( Fig.12 )'s cdsc box in the iref box is searched for a cdsc box whose reference source matches the item ID of the predetermined main image, and the processing continues to step S92.

[0259] In step S92, as the item ID of the designating information of the RAW file of the predetermined main image as the item, the file control unit 43 reads the reference destination in the cdsc box searched in step S91 and the reference source having the item ID matching the predetermined main image, and the processing proceeds to step S93.

[0260] In step S93, the file control unit 43 searches the iloc box of the third association type set file for the item ID which is the specification information of the item read in step S92, and the process proceeds to step S94.

[0261] In step S94 , the file control unit 43 reads the offset and size associated with the item ID of the designated information searched in step S93 in the iloc box, and the process proceeds to step S95 .

[0262] In step S95, the file control unit 43 reads uuid as designated information of the RAW file of the predetermined main image stored in the mdat box of the third association type set file according to the offset and size associated with the item ID of the designated information read in step S94, and the processing ends.

[0263] The file control unit 43 can access the RAW file of the predetermined main image through the uuid read as described above.

[0264] Fig.24 is a flowchart for explaining an example of a process of acquiring an item ID list of a main image from a collection file.

[0265] For example, in the case of generating a handle list or the like, a process of acquiring an item ID list of a main image from a collection file is performed.

[0266] In step S101, the file control unit 43 reads the item ID ( Figures 9 to 12 ), and is registered in the item ID list of the main image (hereinafter, also referred to as the main image list), and the processing proceeds to step S102.

[0267] In step S102, the file control unit 43 reads the item ID as the reference destination from all the boxes in the iref box of the collection file, excludes the item ID from the main image list, and the process ends.

[0268] After the above processing, the item ID registered in the main image list becomes the item ID of the main image.

[0269] Fig.25 is a flowchart for explaining an example of a process of reproducing thumbnail images of (frames of) a main image with respect to predetermined time information from a sequence file.

[0270] Note that in Fig.25 In the example, it is assumed that the file control unit 43 has already recognized the time information (or order) of the predetermined main image through the handle list or the like.

[0271] In step S111, the file control unit 43 generates a track box as a track for managing thumbnail images of main images with respect to predetermined time information in the sequence file ( Fig.13 and 14) searches the trak box in the moov box of the image processing unit for a trak box in which information indicating that data constituting the track is a thumbnail image is stored in the tref box, that is, a trak box in which the type in the tref box is thmb, and the processing proceeds to step S112.

[0272] In step S112, the file control unit 43 reads the track ID in the tkhd box in the trak box searched in step S111 as the track ID of the track of the thumbnail image of the main image with respect to the predetermined time information, and the processing proceeds to step S113.

[0273] In step S113, the file control unit 43 reproduces the track having the track ID read in step S112, and obtains the thumbnail image (frame) corresponding to the predetermined time information (or order) from the track as the thumbnail image of the main file corresponding to the predetermined time information, and the processing ends.

[0274] Note that the processing type of reproducing a track of images stored in a sequence file is similar to the processing of reproducing a moving image of an MP4 file.

[0275] Fig.26 is a flowchart for explaining an example of a process of acquiring uuid of designation information of a RAW file which is a predetermined main image (frame) from an association type sequence file.

[0276] Note that in Fig.26 In the example, it is assumed that the file control unit 43 has already recognized the time information (or order) of the predetermined main image through the handle list or the like.

[0277] In step S121, the file control unit 43 associates the type sequence file ( Fig.14 )'s trak box in the moov box is searched for a trak box in which information indicating that data constituting the track is designated information is stored in the tref box, that is, a trak box in which the type in the tref box is cdsc, as the trak box for managing the track of the designated information, and the processing proceeds to step S122.

[0278] In step S122, the file control unit 43 reads the track ID in the tkhd box in the trak box searched in step S121 as the track ID of the track of the designated information, and the processing proceeds to step S123.

[0279] In step S123, from the track having the track ID read in step S122, the file control unit 43 acquires uuid as designation information relative to time information (or order) of the predetermined main image as the uuid of the RAW file of the predetermined main image, and the processing ends.

[0280] The file control unit 43 can access the RAW file of the predetermined main image through the uuid acquired as described above.

[0281] As described above, the file control unit 43 generates and reproduces an association type HEIF file that stores the main image in the HEIF file and the designation information of the external data to be associated with the main image outside the HEIF file in association with each other in the HEIF file conforming to the HEIF. Therefore, the main image stored in the HEIF file can be associated with the external data outside the HEIF file.

[0282] Furthermore, in the case of using uuid as the designation information, even if the file name of the external data is changed, the association between the main image in the HEIF file and the external data whose file name is changed can be maintained by the uuid.

[0283] <Storage of specified information assigned to external data>

[0284] Fig. 27 : is a view illustrating an example of storing uuid into a RAW file in a case where a RAW file of a main image is adopted as (a file storing) external data and an association type collection file is generated.

[0285] Note that in Fig. 27 In the example, the first association type set file is used as the association type set file.

[0286] The RAW file has an area called a marker comment (MakerNote) as a partial area for storing additional information of Exif as metadata.

[0287] The file control unit 43 may store the uuid assigned to the RAW file (RAW image) into, for example, a mark comment of the RAW file.

[0288] exist Fig. 27, four items of main image Item#1, Item#2, Item#3, and Item#4 are stored in the association type set file, and RAW files #1, #2, #3, and #4 storing the RAW images of main image Item#1, Item#2, Item#3, and Item#4 are generated. Then, UUID#i is assigned to RAW file #i (RAW image), and the association information storage box stores association information in which the item ID#i of main image Item#i is designated to be associated with the UUID#i of the RAW file #i to be associated with the main image Item#i, as association information associating the main image Item#i with the UUID#i of the RAW file #i of the main image Item#i.

[0289] Fig.28 is a view illustrating an example of storing uuid into a RAW file in the case of adopting a RAW file of a main image as external data and generating an association type sequence file.

[0290] Also in the case of generating an associated type sequence file, with Fig. 27 Similar to the case of generating an associated type set file described in , the file control unit 43 can store the uuid assigned to the RAW file in the tag comment of the RAW file.

[0291] exist Fig.28 In the example, track #1 including main images #1, #2, #3 and #4 as four frames is stored in an association type sequence file, and RAW files #1, #2, #3 and #4 storing RAW images of main images #1, #2, #3 and #4 are generated. Then, UUID #i is assigned to RAW file #i, and the association type sequence file stores track #3, which is configured so that UUID #i of RAW file #i is arranged to have the same time information as main image #i corresponding to RAW file #i (RAW image).

[0292] As described above, when track #3 is configured so that the UUID #i of RAW file #i is arranged to have the same time information as the main image #i corresponding to the RAW file #i, the i-th main image #i of track #1 and the i-th UUID #i of track #3 (i.e., the UUID #i of the RAW file #i of the main image #i) are associated and stored in the association type sequence file.

[0293] In the above description, the RAW file (RAW image) of the main image is used as the external data, but other data may be used as the external data. As the external data, for example, audio (sound) recorded together with the imaging of the main image may be used. As the file storing the audio, for example, a WAV file in the WAV format, an MP4 file in the MP4 format, etc. may be used. In the following, for example, it is assumed that a WAV file is used as the file storing the audio.

[0294] Fig.29 : is a view illustrating an example of storing uuid into a WAV file in a case where a WAV file of a main image is adopted as (a file storing) external data and an association type collection file is generated.

[0295] Note that in Fig.29 In the example, the first association type set file is used as the association type set file.

[0296] The WAV file has an area called a list block as a partial area of ​​the area in which metadata is described.

[0297] The file control unit 43 may store the uuid assigned to the WAV file (audio) in, for example, a list block of the WAV file.

[0298] exist Fig.29 , main images Item#1, Item#2, Item#3, and Item#4 are stored as four items in an association type set file, and WAV files #1, #2, #3, and #4 of the main images Item#1, Item#2, Item#3, and Item#4 are generated. Then, UUID#i is assigned to WAV file #i (audio), and an association information storage box stores association information in which item ID#i of main image Item#i is specified to be associated with UUID#i of WAV file #i to be associated with main image Item#i, as association information for associating main image Item#i with UUID#i of WAV file #i of main image Item#i.

[0299] Fig.30 is a view illustrating an example of storing uuid into a WAV file in a case where a WAV file of a main image is adopted as external data and an associated type sequence file is generated.

[0300] Also in the case of generating an associated type sequence file, with Fig.29 Similar to the case of generating an associated type set file described in , the file control unit 43 can store the uuid assigned to the WAV file in the list block of the WAV file.

[0301] exist Fig.30 In the example, track #1 including main images #1, #2, #3 and #4 as four frames is stored in an associated type sequence file, and WAV files #1, #2, #3 and #4 of the main images #1, #2, #3 and #4 are generated. Then, UUID #i is assigned to WAV file #i, and the associated type sequence file stores track #3, which is configured so that UUID #i of WAV file #i is arranged to have the same time information as the main image #i corresponding to WAV file #i.

[0302] As described above, when track #3 is configured with the UUID #i of WAV file #i (which UUID #i is arranged to have the same time information as the main image #i corresponding to WAV file #i), the i-th main image #i of track #1 and the i-th UUID #i of track #3 (i.e., the UUID #i of the WAV file #i of the main image #i) are associated and stored in the association type sequence file.

[0303] Note that in addition to HEIF files, the present technology can also be applied to, for example, ISO base media files having a box structure other than HEIF files, MP4 files, Miaf files, and the like.

[0304] Furthermore, the present technology can be applied to, for example, a file or the like that stores an image (main image) having no box structure and another image in which the resolution of the image is lowered.

[0305] Moreover, in addition to the case where the external data is associated with the main image in the HEIF file, the present technology can also be applied to the case where the external data is associated with the screen nail image or thumbnail image in the HEIF file.

[0306] Furthermore, the present technology can be applied to a case where, for example, external data is associated with internal data other than an image such as a main image in a HEIF file.

[0307] <Index Display>

[0308] Fig.31 is a diagram for explaining an example of index display on the digital camera 10 .

[0309] The digital camera 10 can display a list of images stored in various files such as HEIF files (including associated type HEIF files) as an index display. For ease of explanation, it is assumed that the file as the target of the index display is a HEIF file.

[0310] For index display, the digital camera 10 reads thumbnail images Thm#i stored in the HEIF file, and displays a list screen in which the thumbnail images Thm#i are arranged in a matrix on the liquid crystal panel 19 .

[0311] exist Fig.31 In the above example, a list screen is generated in which the thumbnail images Thm#i are arranged in 3 rows×3 columns.

[0312] In the case where the total number of thumbnail images Thm#i stored in the HEIF file that is the target of index display exceeds the number of thumbnail images that can be arranged on the list screen (9 in this example), a necessary number of tabs are set at the top of the list screen. Fig.31 , the black tab is the selected tab.

[0313] By default, the first tab (from the left) is selected, and a list screen in which thumbnail images Thm#1 to Thm#9 stored in the first nine HEIF files that are targets of index display are arranged is displayed.

[0314] When the user selects the second tab, a list screen in which thumbnail images Thm#10 to Thm#18 stored in the next nine HEIF files that are targets of index display are arranged is displayed.

[0315] Similarly, a list screen in which thumbnail images stored in nine (or less) HEIF files corresponding to the tab selected by the user are arranged is displayed. Here, the thumbnail images in the list screen are not limited to a 3×3 arrangement.

[0316] Note that, Fig.31 The digital camera 10 in the embodiment is provided with a share button and a RAW edit button.

[0317] When the share button is operated in a state where a thumbnail image Thm#i in the list screen is selected, a main image corresponding to the thumbnail image Thm#i selected in the list screen is transmitted to an external device other than the digital camera 10 and shared with the external device.

[0318] When the RAW edit button is operated in a state where a thumbnail image Thm#i in the list screen is selected, in the digital camera 10, for example, a RAW image (associated with a main image) corresponding to the thumbnail image Thm#i selected in the list screen is read from a RAW file, and the RAW image is displayed instead of the list screen on the liquid crystal panel 19. Thus, the digital camera 10 enters a state where the RAW image can be edited.

[0319] In the index display as described above, when a larger number of HEIF files are targeted for index display, it is necessary to open / close such a larger number of HEIF files, and thus it takes time to display the list screen.

[0320] Furthermore, in the case where a HEIF file serving as a target for index display includes a sequence file and thumbnail images Thm#i stored in the sequence file are long-GOP encoded, processing performed for index display may be redundant.

[0321] For example, among 15 frames of thumbnail images Thm#i constituting a GOP, thumbnail images Thm#1 to Thm#9 of the 1st to 9th frames are arranged in a first list screen and thumbnail images Thm#10 to Thm#15 of the 10th to 15th frames are arranged on a second list screen. When the list screen display is switched from a screen other than the second list screen to the second list screen, in order to decode thumbnail images Thm#10 to Thm#15 of the 10th to 15th frames arranged in the second list screen, it is necessary to decode thumbnail images Thm#i necessary for decoding thumbnail images Thm#10 to Thm#15 of the 10th to 15th frames among thumbnail images Thm#1 to Thm#9 of the 1st to 9th frames that are not arranged in the second list screen.

[0322] Fig.32 10 is a view for explaining an example of index display on an external device other than the digital camera 10 .

[0323] Regarding images stored in a HEIF file obtained by the digital camera 10 , index display may be performed on an external device other than the digital camera 10 .

[0324] For example, when index display is performed on an external device, a HEIF file that is a target of index display is transferred to the external device via a network, and index display is performed on the external device in a manner similar to index display on the digital camera 10 .

[0325] HEIF files can be transferred from the digital camera 10 to an external device via the Internet, LAN, USB, or any other wired or wireless network. In addition, HEIF files can also be transferred from the digital camera 10 to an external device via a cloud computer.

[0326] As in the case of performing index display on the digital camera 10 , in the case of performing index display on the external device for images stored in a HEIF file acquired by the digital camera 10 , displaying the list screen takes time and processing performed for the index display may be redundant.

[0327] Furthermore, when index display is performed on an external device, it takes time to transfer the HEIF file necessary for generating a list screen from the digital camera 10 to the external device. For example, responsiveness from when a user requests index display on an external device until a list screen is actually displayed may decrease.

[0328] Fig.33 is a view illustrating an example of index display for a first association type collection file.

[0329] Here, a certain HEIF file A and another HEIF file B different from the HEIF file A may use duplicate item IDs. For example, the item ID of an item in the HEIF file A may be set to 1, and the item ID of an item in the HEIF file B may be set to 1. However, in this embodiment, for easier explanation, unique item IDs are appropriately used in the HEIF file A and the HEIF file B.

[0330] exist Fig.33 In the figure, four first association type set files (HEIF files) #1 to #4 are the targets of index display.

[0331] exist Fig.33 In the first association type set file #i in the mdat box, a main image Item#i of one frame and a thumbnail image Item#(1000+i) of the main image Item#i are stored in the mdat box. In addition, an association information storage box having association information indicating that the number of main images is 1 and associating the item ID#i of the main image Item#i with the UUID#i as designation information of external data associated with the main image Item#i is stored in the meta box.

[0332] In the case of performing index display on four first association type collection files #1 to #4 as described above, for example, thumbnail images Item#1001 to Item#1004 are read from the four first association type collection files #1 to #4, respectively. Then, a list screen in which thumbnail images Item#1001 to Item#1004 are arranged is generated.

[0333] Therefore, when index display is performed on the digital camera 10 and the external device, in order to read the thumbnail images Item#1001 to Item#1004 from the first association type collection files #1 to #4 respectively, it is necessary to open / close each of the four (in other words, multiple) first association type collection files #1 to #4, thereby possibly reducing the responsiveness of the index display.

[0334] Moreover, when index display is performed on an external device, four (in other words, multiple) first association type set files #1 to #4 need to be transferred from the digital camera 10 to the external device, so transferring the files takes time and may reduce the responsiveness of the index display.

[0335] Fig.34 is a view illustrating another example of index display of a first association type collection file.

[0336] exist Fig.34 In the figure, the first associated type set file (HEIF file) #1 is the target of index display.

[0337] exist Fig.34 In the first association type set file #1, main images Item#1 to Item#4 of four (in other words, multiple) frames and their corresponding thumbnail images Item#1001 to Item#1004 of the main images Item#1 to Item#4 are stored in the mdat box.

[0338] In addition, an associated information storage box is stored in the meta box, the associated information storage box having associated information indicating that the number of main images is 4 and associating item ID#i of main image Item#i with UUID#i, which is designated information of external data associated with the main image, for each of main images Item#1 to Item#4 of four frames.

[0339] In the case where index display is performed on the first association type collection file #1 as described above, for example, thumbnail images Item#1001 to Item#1004 are read from the first association type collection file #1. Then, a list screen in which the thumbnail images Item#1001 to Item#1004 are arranged is generated.

[0340] Therefore, when index display is performed on the digital camera 10 and the external device, in order to read only the thumbnail images Item#1001 to Item#1004 from the first association type set file #1, a search is performed on the first association type set file #1, thereby reducing the responsiveness of the index display.

[0341] also, Fig.34 The first association type set file in stores main images Item#1 to Item#4 of four (in other words, a plurality of) frames, and therefore the file size is large.

[0342] Therefore, when index display is performed on an external device, the first association type set file #1 having a large file size needs to be transferred from the digital camera 10 to the external device, so transferring the file takes time and may reduce the responsiveness of the index display.

[0343] Note that the index display for collection files other than the association type collection files can be performed in a similar manner to the index display for the association type collection files.

[0344] Fig.35 is a view illustrating an example of index display for an association type sequence file.

[0345] exist Fig.35 In the example, a sequence file of associated type (HEIF file) #1 is the target of index display.

[0346] exist Fig.35 In the associated type sequence file #1, track #1 including a main image of four (in other words, multiple) frames, track #2 including thumbnail images of four frames corresponding to the main image, and track #3 including UUID #1 to UUID #4 which are designated information of external data associated with the main image of the four frames are respectively stored in the mdat box.

[0347] Also, trak boxes that respectively manage tracks #1 to #3 are stored in the moov box.

[0348] Each of the trak boxes managing tracks #1 to #3 includes a tkhd box storing a track ID of the track managed by the trak box. The trak box managing track #2 also includes a tref box storing (information indicating) that another track related to track #2 is track #1 (track_ID=1) and track #2 is a track of thumbnail images (type=thmb). The trak box managing track #3 also includes a tref box storing (information indicating that another track related to track #3 is track #1 (track_ID=1) and track #3 is a track of metadata (designation information) (type=cdsc).

[0349] When index display is performed on the association type sequence file #1 as described above, for example, thumbnail images of four frames (encoded data) are read from the association type sequence file #1 and decoded. Then, a list screen in which thumbnail images of four frames are arranged is generated.

[0350] For example, the association type sequence file stores thumbnail images in a state where the thumbnail images are long-GOP encoded.

[0351] Therefore, in order to decode (reproduce) an inter-frame-coded thumbnail image, it is necessary to decode another thumbnail image to which the thumbnail image refers. Furthermore, in the case where another thumbnail image refers to yet another thumbnail image, it is necessary to decode such a thumbnail image.

[0352] Since the associated type sequence file can store inter-frame coded images, its file size can be smaller than the file size of the collection file that does not allow the storage of inter-frame coded images. However, due to inter-frame coding, decoding a thumbnail image may require decoding other thumbnail images of one or more frames.

[0353] Therefore, for example, when scrolling or switching list screens between tabs on the digital camera 10 and the external device, decoding thumbnail images to be displayed after scrolling or switching between tabs takes time, and the responsiveness of index display may be reduced.

[0354] Note that the index display for sequence files other than the association type sequence files can be performed in a similar manner to the index display for the association type sequence files.

[0355] Therefore, the digital camera 10 can generate another HEIF file storing related images related to the main image (such as thumbnail images of the main image stored in one or more HEIF files) as a proxy file representing one or more HEIF files. Then, the digital camera 10 and the external device can perform index display using the proxy file.

[0356] By performing index display using a proxy file, the digital camera 10 and the external device can reduce the time required to display a list screen as an index display, thereby realizing a quick index display.

[0357] Furthermore, it is made possible to prevent the processing performed for index display from being redundant.

[0358] Furthermore, by transferring the proxy file from the digital camera 10 to the external device, it is possible to reduce the time required for file transfer and suppress a decrease in responsiveness of the external device, compared with the case of transferring the HEIF file necessary for generating the list screen.

[0359] Furthermore, the proxy file facilitates management of thumbnail images because the proxy file collects thumbnail images of the primary images stored in one or more HEIF files.

[0360] <Proxy File>

[0361] Fig.36 is a view for explaining a first example of a proxy file.

[0362] The file control unit 43 is capable of generating a proxy file that stores a related image related to a main image stored in a HEIF file (including a HEIF file of an associated type) as a main image in the HEIF file.

[0363] As a proxy file, for example, a HEIF file, or in particular a collection file, can be used. In addition, a sequence file or a file other than a HEIF file can be used as a proxy file.

[0364] Furthermore, in addition to storing related images related to a primary image stored in a HEIF file, a proxy file can also store related images related to an image stored in, for example, a JPEG file or a RAW file other than a HEIF file.

[0365] As an associated image related to a main image stored in a HEIF file, an image that is identical to the main image in content but different from the main image in the number of pixels (size) may be used (i.e., for example, a thumbnail image having a smaller number of pixels (size) than the main image, an image that is identical to the main image in content but different from the main image in a YUV signal format, an image that is identical to the main image in content but different from the main image in bit allocation (grayscale) per pixel, and other images related to the main image).

[0366] For example, in the case where a HEIF file stores a main image and thumbnail images, as a related image related to the main image, a thumbnail image generated by reducing pixels from the main image or a thumbnail image obtained by copying a thumbnail image stored in the HEIF file can be used.

[0367] That is, in the case where a thumbnail image of a main image is adopted as an image related to the main image, if a HEIF file stores the main image and the thumbnail image, a thumbnail image may be newly generated from the main image stored in the HEIF file and the thumbnail image may be stored in the proxy file as a related image, or (a copy of) the thumbnail image stored in the HEIF file may be stored in the proxy file as a related image. The following description assumes, for example, that a thumbnail image of a main image is adopted as a related image of the main image.

[0368] Here, an item in a HEIF file that is a proxy file is also referred to as a proxy item in order to distinguish the item from an item in a HEIF file that is a non-proxy file.

[0369] exist Fig.36 In the HEIF file, there are four HEIF files #1 to #4, which are set files, and the mdat box in each HEIF file #i stores a main image Item#i and a thumbnail image Item#(1000+i) of the main image Item#i.

[0370] In addition, Fig.36 , a proxy file is generated in which (copies of) the thumbnail images Item#1001 to Item#1004 stored in HEIF files #1 to #4 are stored as proxy items Item#1 to Item#4.

[0371] As an image, the proxy item Item#i is a thumbnail image of the main image Item#i in the HEIF file #i, but is the main image in the (HEIF file of) the proxy file.

[0372] In the first example of the proxy file, as the main image corresponding to the thumbnail image as proxy item #i, the main image Item#i' that best matches the thumbnail image as proxy item #i or the main image Item#i' corresponding to the thumbnail image Item#(1000+i') that best matches the thumbnail image as proxy item #i can be detected by, for example, matching the thumbnail image as proxy item #i in the proxy file with each of the main images Item#1 to Item#4 in HEIF files #1 to #4 or with each of the thumbnail images Item#1001 to Item#1004.

[0373] Fig.37 is a view for explaining a second example of a proxy file.

[0374] like Fig.36 As shown in Fig.37 , a proxy file is generated in which thumbnail images Item#1001 to Item#1004 stored in HEIF files #1 to #4 are stored as proxy items Item#1 to Item#4.

[0375] Moreover, in Fig.37 In the example, the file generated as a proxy file is an associated type HEIF file, in which when a thumbnail image as a proxy item Item#i (a main image in the proxy file) is used as internal data in the proxy file and a main image Item#i corresponding to the thumbnail image is used as external data outside the proxy file, designated information of the internal data (the thumbnail image as the proxy item Item#i) and the external data (the main image Item#i) are stored in association with each other.

[0376] That is, in Fig.37 In the HEIF file as a proxy file, the meta box stores an associated information storage box, which stores associated information associating the item ID #i of the thumbnail image as the proxy item Item #i (the main image in the proxy file) with the hash value #i as the designated information of the main image Item #i corresponding to the thumbnail image. Fig.37 The proxy file in is the first association type collection file ( Fig.10 ).

[0377] As the hash value #i which is the specifying information of the main image Item#i, for example, a hash value obtained by using the main image Item#i as an input can be adopted.

[0378] also, Fig.37 It is shown that in the associated information storage box in the proxy file, the number of main images is four, which is the number of thumbnail images of the main images Item#1 to Item#4 (as proxy items Item#1 to Item#4 stored in the proxy file), that is, the number of groups of related item IDs and hash values ​​as specified information.

[0379] It is noted that as the proxy file (file format), an associated type HEIF file other than the first associated type set file (ie, the second associated type set file ( Fig.11 ), the third association type collection file ( Fig.12 ) or an associated type sequence file ( Fig.14 )).

[0380] In the second example of the proxy file, for example, among the hash values ​​generated by using the respective main images Item#1 to Item#4 in the HEIF files #1 to #4 as input, the main image specified by the hash value (the hash value associated with the item ID of the thumbnail image) which is the designated information associated with the thumbnail image as the proxy item #i (i.e., the main image Item#i' which provides a hash value matching the hash value #i) can be detected as the main image corresponding to the thumbnail image as the proxy item #i.

[0381] Fig.38 is a view for explaining a third example of a proxy file.

[0382] like Fig.37 As shown in Fig.38 In the example, a proxy file is generated in which thumbnail images Item#1001 to Item#1004 stored in HEIF files #1 to #4 as a collection file are stored as proxy items Item#1 to Item#4.

[0383] Moreover, if Fig.37 As shown in Fig.38 In the example, the file generated as a proxy file is an associated type HEIF file, in which, when a thumbnail image as a proxy item Item#i (a main image in the proxy file) is used as internal data in the proxy file and a main image Item#i corresponding to the thumbnail image is used as external data outside the proxy file, the internal data (the thumbnail image as the proxy item Item#i) and the designated information of the external data (the main image Item#i) are stored in association with each other.

[0384] That is, Fig.37 Like in Fig.38In the HEIF file as a proxy file, the meta box stores an associated information storage box, which stores associated information that associates the item ID #i of the thumbnail image as the proxy item Item #i with the specified information of the main image Item #i corresponding to the thumbnail image. Fig.38 The proxy file in is a first association type collection file.

[0385] However, in Fig.38 In the example, uuid is used as the specifying information of the main image Item#i instead of the hash value.

[0386] In the case where uuid is used as the designation information of the main image Item#i as external data, as shown in the reference Figures 27 to 30 In the above case, it is necessary to write (store) UUID#i as the designation information of the main image Item#i into the HEIF file #i storing the main image Item#i as external data.

[0387] Therefore, the file control unit 43 writes the UUID#i of the main image Item#i as external data in the HEIF file#i storing the main image Item#i.

[0388] In order to write UUID#i into a HEIF file #i storing the primary image Item#i as external data, UUID#i may be written into a meta box or an mdat box in the HEIF file #i. In order to write UUID#i into a HEIF file #i storing the primary image Item#i as external data, UUID#i may be written, for example, according to the format of the associated type HEIF file, i.e., in such a way that the format after writing UUID#i is the format of the associated type HEIF file.

[0389] exist Fig.38 In the example, according to the file format of the first association type set file, UUID#i is written into the HEIF file #i storing the main image Item#i as external data.

[0390] That is, in Fig.38 In the HEIF file #i, an association information storage box in which the UUID#i of the main image Item#i and the item ID#i of the main image Item#i are stored in association with each other is stored in a meta box in the HEIF file #i, so the HEIF file #i is a first association type collection file (conforming to the format of the first association type collection file).

[0391] also, Fig.38It is shown that in the associated information storage box in the HEIF file #i (which has become a first association type collection file), the number of main images is 1, which is the number of main image Item#i stored in the HEIF file #i, that is, the number of groups of associated item IDs and uuids as designated information.

[0392] Here, a HEIF file storing a target main image for a thumbnail image to be stored as a related image in a proxy file is a target HEIF file for generating a proxy file, and such a HEIF file is hereinafter also referred to as a proxy target file.

[0393] UUID #i may be written into HEIF file #i as a proxy target file according to the format of any one of the first to third association type collection files when HEIF file #i is a collection file, or according to the format of an association type sequence file when HEIF file #i is a sequence file.

[0394] Note that for HEIF file #i which is a proxy target file having UUID #i written thereto, main image Item #i in HEIF file #i is internal data of HEIF file #i, and thumbnail image which is a proxy item Item #i in the proxy file is external data of HEIF file #i.

[0395] In addition, for the proxy file, the UUID#i written in the HEIF file#i as the proxy target file is the designation information of the main image#Item#i as the external data of the proxy file. However, the UUID#i can also be regarded as the designation information that designates the proxy item#i.

[0396] In this case, the HEIF file #i which is a proxy target file in which UUID #i has been written can be regarded as a first association type HEIF file in which the main image Item #i as internal data and the proxy item Item #i as external data (i.e., the designated information UUID #i of the thumbnail image (of the main image Item #i) in the proxy file) are stored in association with each other.

[0397] In the third example of the proxy file, for example, the primary image specified by the uuid (uuid associated with the item ID of the thumbnail image) as the designated information associated with the thumbnail image as the proxy item #i (i.e., among the main images Item#1 to Iem#4 in HEIF files #1 to #4, the main image Item#i with item ID#i associated with UUID#i) can be detected as the primary image corresponding to the thumbnail image as the proxy item #i.

[0398] Fig.39is a view for explaining a fourth example of a proxy file.

[0399] Fig.39 The fourth example of a proxy file in Fig.38 The difference between the third example of the proxy file in the example is that the HEIF file as the proxy target file is Fig.39 The single sequence file #1 in , and the HEIF file as the proxy target file in Fig.38 There are four collection files.

[0400] Here, Fig.39 The collection file #1 used as the proxy target file is not an associated type sequence file but a normal sequence file.

[0401] exist Fig.39 In the sequence file #1 in , track #1 including four (in other words, a plurality of) frames of main images and track #2 including four frames of thumbnail images of the main images corresponding to their four frames are stored in the mdat box.

[0402] Also, trak boxes that respectively manage tracks #1 and #2 are stored in the moov box.

[0403] Each of the trak boxes managing tracks #1 to #2 includes a tkhd box storing a track ID of the track managed by the trak box.

[0404] The trak box managing track #2 also includes a tref box storing information indicating that another track related to track #2 is track #1 (track_ID=1) and that track #2 is a track of thumbnail images (type=thmb).

[0405] For example, stored in Fig.39 Thumbnail images of four frames in sequence file #1 in are decoded and intra-coded.

[0406] Then, a proxy file is generated in which the intra-frame-coded thumbnail images of four frames are stored as proxy items Item#1 to Item#4. Moreover, in the proxy file, the thumbnail image as the proxy item Item#i (the main image in the proxy file) is used as internal data in the proxy file and the main image Item#i of the i-th frame (the frame at the i-th position) in the track #1 in the sequence file #1 corresponding to the thumbnail image is used as external data outside the proxy file, and the internal data (the thumbnail image as the proxy item Item#i) and the designated information of the external data (the main image of the i-th frame) are stored in association with each other.

[0407] That is, Fig.38 Like in Fig.39In the example, the associated information storage box is stored in the meta box in the HEIF file as a proxy file, and the associated information storage box stores the associated information associating the item ID #i of the thumbnail image as the proxy item Item #i with the designated information of the main image of the i-th frame corresponding to the thumbnail image. Fig.39 The proxy file in is a first association type collection file.

[0408] like Fig.38 Like in Fig.39 In , uuid is used as the designation information of the main image as external data. The uuid of the main image of the i-th frame as external data is UUID#i.

[0409] As mentioned above, in Fig.39 In , uuid is used as the designation information of the main image as external data. Therefore, Fig.38 As in , uuid which is designation information of the main image as external data is written (stored) to sequence file #1 which is a proxy target file for storing the main image as external data.

[0410] Fig.40 It is a view for explaining an example of writing uuid which is designation information of a main image which is external data into sequence file #1 which is a proxy target file.

[0411] In order to write the uuid of the main image as external data into the sequence file #i storing the main image, the uuid may be written into the moov box or the mdat box in the sequence file #i. In order to write the uuid of the main image into the sequence file #i storing the main image, the uuid may be written, for example, according to the format of the associated type HEIF file (i.e., in such a way that the format after writing the uuid is the format of the associated type HEIF file).

[0412] exist Fig.40 In the example, the uuid is written to the sequence file #1 that stores the main image as external data, according to the file format of the associated type sequence file.

[0413] That is, in Fig.40 In the process, track #3 of UUID #1 to UUID #4 for main images of frames 1 to 4 included in track #1 is generated, a trak box for managing track #3 is generated, track #3 is stored in the mdat box, and the trak box for managing track #3 is stored in the moov box, whereby sequence file #1 is an associated type sequence file (conforming to the format of an associated type sequence file).

[0414] In track #3, UUID#1 to UUID#4 of the main images of the 1st to 4th frames are included and arranged respectively, so that the main image of the i-th frame in track #1 is associated with UUID#i of the main image according to time information on the timeline or the order of arrangement in track #3.

[0415] The trak box managing the track #3 includes a tkhd box storing the track ID of the track #3 managed by the trak box.

[0416] The trak box managing track #3 also includes a tref box storing information indicating that another track related to track #3 is track #1 (track_ID=1) and that track #3 is a track (designation information) of metadata (type=cdsc).

[0417] In the fourth example of the proxy file, for example, associated with UUID #i, the main image of the i-th frame in track #1 can be detected as the main image corresponding to the thumbnail image as proxy item #i among the main images of the four frames included in track #1 in (association type) sequence file #1.

[0418] Note that for sequence file #1 which is a proxy target file with UUID#i written in it, the main image in sequence file #1 is internal data of sequence file #1, and the thumbnail image of the proxy item Item#i in the proxy file is external data of sequence file #1.

[0419] Furthermore, for the proxy file, UUID#i written in sequence file #1 as the proxy target file is the designation information of the i-th frame main image as the external data of the proxy file. However, UUID#i can also be regarded as the designation information designating the proxy item #i.

[0420] In this case, sequence file #1, which is a proxy target file in which UUID#i has been written, can be regarded as an associated type sequence file, in which the i-th frame main image as internal data and the proxy item Item#i as external data (i.e., the designated information UUID#i of the thumbnail image of the i-th frame main image) are stored in association with each other.

[0421] The proxy file as described above makes it possible to quickly perform index display of thumbnail images of related images of the main image stored in the HEIF file, for example. Moreover, it makes it possible to easily detect the main image corresponding to the thumbnail image as the related image in the index display.

[0422] Fig.41 This is a view for explaining an example of generating a proxy file using a first association type collection file as a proxy target file.

[0423] refer to Figures 36 to 40 , a description has been given about a proxy file generated with a normal HEIF file (a HEIF file other than an associated type HEIF file) as a proxy target file. However, in addition to a normal HEIF file, a proxy file can also be generated with an associated type HEIF file as a proxy target file.

[0424] exist Fig.41 In the example, the proxy target files are four first association type set files #1 to #4.

[0425] exist Fig.41 In the first association type set file #i in the mdat box, a main image Item#i of one frame and a thumbnail image Item#(1000+i) of the main image Item#i are stored in the mdat box. In addition, an association information storage box having association information indicating that the number of main images is 1 and associating the item ID#i of the main image Item#i with the UUID#i which is designated information of external data (not shown) associated with the main image Item#i is stored in the meta box.

[0426] As described above, in the first association type set file #i, the meta box stores an association information storage box having association information associating the item ID #i of the main image Item #i with the UUID #i as the designation information of some external data associated with the main image Item #i, and in the case where the UUID #i is used as the designation information of some external data, the main image Item #i is already associated with the external data. Some external data already associated with the main image Item #i in the first association type set file #i is also referred to as already associated external data.

[0427] In the case where a proxy file generation process for generating a proxy file is performed by using the first association type collection file #i as the proxy target file as described above, the file control unit 43 can generate a proxy file by using UUID#i, which has been used as the designation information of the associated external data, in a shared manner as the designation information of the main image Item#i which is the external data of the proxy file.

[0428] That is, in the proxy file generation process, when thumbnail images Item#1001 to Item#1004 stored in the first association type set files #1 to #4 are used as proxy items Item#1 to Item#4, thumbnail images (main images in the proxy files) are used as proxy items Item#i which are internal data in the proxy files, and main image Item#i corresponding to the thumbnail images is used as external data outside the proxy file, the file control unit 43 generates an association type HEIF file as a proxy file, in which designated information of internal data (thumbnail images as proxy item Item#i) and external data (main image Item#i) are stored in association with each other.

[0429] For example, the file control unit 43 generates a first association type set file as a proxy file, in which the meta box stores an association information storage box, and the association information storage box stores association information that associates the item ID#i of the thumbnail image serving as the proxy item Item#i with the specified information of the main image Item#i corresponding to the thumbnail image.

[0430] In the proxy file generation process with the first association type collection file #i as the proxy target file as described above, UUID#i, which is the designation information of the associated external data associated with the main image Item#i in the first association type collection file #i, can be used as the designation information of the main image Item#i as the external data of the proxy file.

[0431] In this case, regarding UUID#i of the designation information of the main image Item#i serving as external data as a proxy file, the same as the reference Fig.38 Unlike the description provided, it is not necessary to write UUID#i into the first association type set file #i. This is because the first association type set file #i is in a state where UUID#i has already been written.

[0432] exist Fig.41 In the proxy file, Fig.38 As in the third example of the proxy file in, for example, among the main images Item#1 to Item#4 in the first HEIF files #1 to #4, the main image Item#i of the item ID#i associated with the UUID#i can be detected as the main image corresponding to the thumbnail image that is the proxy Item#i.

[0433] Fig.42 This is a view for explaining another example of generating a proxy file using a first association type collection file as a proxy target file.

[0434] exist Fig.42 In the example, the proxy target file is a first association type set file #1.

[0435] exist Fig.42 In the first association type set file #1 in the main images Item#1 to Item#4 of four (in other words, a plurality of) frames, and thumbnail images #1001 to Item#1004 of the main images Item#1 to Item#4 corresponding to them are stored in the mdat box. In addition, an association information storage box having association information indicating that the number of main images is 4 and associating the item ID#i of the main image Item#i with the UUID#i as the designated information of the associated external data associated with the main image for each of the main images Item#1 to Item#4 of the four frames is stored in the meta box.

[0436] In the case where the proxy file generation process is performed by using the first association type set file #1 as the proxy target file as described above, Fig.41 As in the case of , the file control unit 43 can generate a proxy file by using UUID#i, which has been used as the designation information of the associated external data, in a shared manner as the designation information of the main image Item#i as the external data of the proxy file.

[0437] That is, in the proxy file generation process, while using thumbnail images Item#1001 to Item#1004 stored in the first association type collection file #1 as proxy items Item#1 to Item#4, using the thumbnail images (main images in the proxy file) as proxy items Item#i which are internal data in the proxy file, and using the main image Item#i corresponding to the thumbnail images as external data outside the proxy file, the file control unit 43 generates an association type HEIF file as a proxy file, in which designated information of the internal data (thumbnail images as proxy item Item#i) and the external data (main image Item#i) are stored in association with each other.

[0438] For example, the file control unit 43 generates a first association type set file as a proxy file, in which the meta box stores an association information storage box, and the association information storage box stores association information that associates the item ID#i of the thumbnail image as the proxy item Item#i with the specified information of the main image Item#i corresponding to the thumbnail image.

[0439] In the proxy file generation process with the first association type collection file #i as the proxy target file as described above, UUID#i, which is the designation information of the associated external data associated with the main image Item#i in the first association type collection file #i, can be used as the designation information of the main image Item#i as the external data of the proxy file.

[0440] In this case, regarding UUID#i of the designation information of the main image Item#i serving as external data as a proxy file, the same as the reference Fig.38 Unlike the description provided, it is not necessary to write UUID#i into the first association type set file #i. This is because the first association type set file #i is in a state where UUID#i has already been written.

[0441] exist Fig.42 In the proxy file, Fig.38 As in the third example of the proxy file in, for example, among the main images Item#1 to Item#4 in the first HEIF files #1 to #4, the main image Item#i of the item ID#i associated with the UUID#i can be detected as the main image corresponding to the thumbnail image that is the proxy Item#i.

[0442] Fig.43 This is a view for explaining an example of generating a proxy file by associating a type sequence file as a proxy target file.

[0443] exist Fig.43 In , the proxy target file is an associated type sequence file #1.

[0444] exist Fig.43 In the association type sequence file #1, track #1 including a main image of four (in other words, multiple) frames, track #2 including thumbnail images of four frames corresponding to the main image, and track #3 including UUID #1 to UUID #4 which are designated information of associated external data associated with the main image of the four frames, are stored in the mdat box.

[0445] Moreover, in Fig.43 In the associated type sequence file #1 in the , trak boxes that manage tracks #1 to #3 respectively are stored in the moov box.

[0446] Each of the trak boxes managing tracks #1 to #3 includes a tkhd box storing a track ID of the track managed by the trak box.

[0447] The trak box managing track #2 also includes a tref box storing information indicating that another track related to track #2 is track #1 (track_ID=1) and track #2 is a track of thumbnail images (type=thmb).

[0448] The trak box managing track #3 also includes a tref box storing information indicating that another track related to track #3 is track #1 (track_ID=1) and track #3 is a track (designation information) of metadata (type=cdsc).

[0449] In the case where the proxy file generation process is performed by using the association type sequence file #1 as the proxy target file as described above, the file control unit 43 can Fig.41 As in the case of , a proxy file is generated by using UUID#i, which has been used as the designation information of the associated external data, in a shared manner as the designation information of the main image Item#i which is the external data of the proxy file.

[0450] That is, in the proxy file generation process, the file control unit 43 causes the encoding control unit 42 to decode the thumbnail images of four frames stored in the association type sequence file #1, and further intra-encode the thumbnail images.

[0451] Then, the file control unit 43 generates a first association type set file as a proxy file, and the first association type set file stores intra-frame encoded thumbnail images of four frames as proxy items Item#1 to Item#4.

[0452] Moreover, in the proxy file, while a thumbnail image as a proxy item Item#i (a main image in the proxy file) is used as internal data in the proxy file and a main image Item#i of the i-th frame in track #1 in sequence file #1 corresponding to the thumbnail image is used as external data outside the proxy file, designated information of the internal data (the thumbnail image as the proxy item Item#i) and the external data (the main image of the i-th frame) are stored in association with each other.

[0453] That is, an association information storage box storing association information associating item ID#i of a thumbnail image as a proxy item Item#i with designation information of a main image of the i-th frame corresponding to the thumbnail image is stored in the meta box of the proxy file.

[0454] In the proxy file generation process with the association type sequence file #1 as the proxy target file as described above, UUID#i which is the designation information of the associated external data associated with the main image of the i-th frame in the association type sequence file #1 can be used as the designation information of the main image of the i-th frame in track #1 which is the external data of the proxy file.

[0455] In this case, UUID#i of the designated information of the i-th frame main image used as external data as a proxy file is referenced Fig.39 and40 Different from the description provided, it is not necessary to write UUID#i into the association type sequence file #1. This is because the association type sequence file #1 is in a state where UUID#i has already been written.

[0456] about Fig.43 The proxy file in Fig.39 and 40 As in the fourth example of the proxy file, for example, from among the main images of the four frames included in track #1 in the associated type sequence file #1, the main image of the i-th frame in track #1 associated with UUID #i can be detected as the main image corresponding to the thumbnail image as proxy item #i.

[0457] Fig.44 A view for explaining the mode adopted by a proxy object file and a set of proxy files generated by a proxy file generation process executed using the proxy object file.

[0458] Fig.44 A shows an example of a first mode adopted by a collection of proxy target files and proxy files.

[0459] In the first mode, the proxy file does not have the designation information of the main image in the proxy target file and corresponding to the thumbnail image as the proxy item in the proxy file. Likewise, the proxy target file does not have the designation information of the main image in the proxy target file.

[0460] In the first mode, a primary image that best matches a thumbnail image as a proxy item may be detected as a primary image corresponding to the thumbnail image as a proxy item by, for example, matching the thumbnail image as a proxy item in the proxy file with the primary image in the proxy target file.

[0461] Fig.44 B shows an example of the second mode adopted by the collection of proxy target files and proxy files.

[0462] In the second mode, the proxy file has designation information of a main image that is in the proxy target file and corresponds to a thumbnail image that is a proxy item in the proxy file.

[0463] On the other hand, the proxy target file does not have the designation information of the main image in the proxy target file. In the second mode, as the designation information of the main image, information that can be generated by using the main image (such as a hash value obtained by using the main image as input) can be adopted.

[0464] In the second mode, for example, among hash values ​​generated by using a main image in a proxy target file as input, a main image that provides a hash value matching a hash value of specified information associated with a thumbnail image in the proxy file can be detected as a main image corresponding to the thumbnail image as a proxy item.

[0465] Fig.44 C shows an example of the third mode adopted by the collection of proxy target files and proxy files.

[0466] In the third mode, the proxy file has designation information of a main image that is in the proxy target file and corresponds to a thumbnail image that is a proxy item in the proxy file.

[0467] Likewise, the proxy target file has designation information of the main image in the proxy target file. In the third mode, as the designation information of the main image, for example, information (such as uuid) that can be used to designate the main image can be adopted.

[0468] In the third mode, for example, a primary image associated with designated information in a proxy target file (the designated information being the same as designated information associated with a thumbnail image serving as a proxy item in the proxy file) may be detected as a primary image corresponding to the thumbnail image serving as a proxy item.

[0469] Fig.45 is a view illustrating a detailed example of the second mode adopted by a collection of a proxy target file and a proxy file.

[0470] exist Fig.45 , a proxy file is generated having two HEIF files #1 and #2 used as proxy target files.

[0471] exist Fig.45 , HEIF file #1 as a proxy target file stores two main images A and B and thumbnail images A and B of the main images A and B, respectively. HEIF file #2 as a proxy target file stores two main images C and D and thumbnail images C and D of the main images C and D, respectively.

[0472] Furthermore, the proxy file stores the thumbnail images A to D as main images in the proxy file, and also stores designation information of the main images A to D corresponding to the thumbnail images A to D. As the designation information of the main images A to D, hash values ​​A to D obtained by using the main images A to D as input are respectively adopted.

[0473] In the second mode, when a thumbnail image stored in a proxy file is selected, among hash values ​​generated by taking as input the respective main images A to D in HEIF files #1 and #2 which are proxy target files, a main image providing a hash value matching a hash value which is designated information associated with the thumbnail image selected in the proxy file can be detected as a main image corresponding to the thumbnail image selected in the proxy file.

[0474] Fig.46 is a view illustrating a detailed example of the third mode adopted by a collection of a proxy target file and a proxy file.

[0475] exist Fig.46 , a proxy file is generated having two HEIF files #1 and #2 used as proxy target files.

[0476] exist Fig.46 , HEIF file #1 as a proxy target file stores two main images A and B and thumbnail images A and B of the main images A and B, respectively. HEIF file #2 as a proxy target file stores two main images C and D and thumbnail images C and D of the main images C and D, respectively.

[0477] In addition, the proxy file stores thumbnail images A to D as main images in the proxy file, and also stores designation information of main images A to D corresponding to the thumbnail images A to D. As the designation information of the main images A to D, uuid (UUID#A, UUID#B, UUID#C, and UUID#D) is used.

[0478] UUID#A, UUID#B, UUID#C, and UUID#D, which are designated information of main images A to D, are stored (written) not only in the proxy file but also in the proxy target file. That is, HEIF file #1, which is a proxy target file storing main images A and B, stores UUID#A and UUID#B of main images A and B, and HEIF file #2, which is a proxy target file storing main images C and D, stores UUID#C and UUID#D of main images C and D.

[0479] In the third mode, when a thumbnail image stored in a proxy file is selected, the primary image associated with the uuid in the proxy target file (which is the same as the uuid associated with the selected thumbnail image in the proxy file) can be detected as the primary image corresponding to the thumbnail image selected in the proxy file.

[0480] <Index display using proxy files>

[0481] Fig.472 is a view for explaining an example of index display using a proxy file on the digital camera 10.

[0482] For index display, the digital camera 10 reads thumbnail images Thm#i stored in the proxy file and displays a list screen in which the thumbnail images Thm#i are arranged in a matrix on the liquid crystal panel 19. Fig.31 Same as in Fig.47 In the example, a list screen is generated in which thumbnail images Thm#i are arranged in 3 rows × 3 columns. In the case where the total number of thumbnail images Thm#i stored in the proxy file as the target of index display exceeds the number of thumbnail images that can be arranged on the list screen (nine in this example), a necessary number of tabs are set at the top of the list screen.

[0483] By default, the first tab is selected, and a list screen in which thumbnail images Thm#1 to Thm#9 stored in the first nine proxy files displayed as targets of index display are arranged is displayed. When the user selects the second tab, a list screen in which thumbnail images Thm#10 to Thm#18 stored in the next nine proxy files displayed as targets of index display are arranged is displayed. Similarly, a list screen in which thumbnail images stored in nine (or less) proxy files corresponding to the tab selected by the user are arranged is displayed.

[0484] Here, with Fig.31 As in the case of , the thumbnail images in the list screen are not limited to a 3×3 arrangement. In addition, the share button and RAW edit button are as shown in the reference Fig.31 described.

[0485] According to the index display using the proxy file as described above, the index display can be appropriately performed.

[0486] That is, according to the index display using the proxy file, only the proxy file is required to be opened / closed, and the HEIF file storing the main image corresponding to the thumbnail image Thm#i in the proxy file does not need to be opened / closed. Therefore, compared with the case of opening / closing the HEIF file storing the main image, the time period for displaying the list screen is shortened to achieve fast display of the list screen.

[0487] In addition, the thumbnail image Thm#i that is not inter-frame encoded is stored in the proxy file that is the target of the index display, that is, in the present embodiment, the thumbnail image is intra-frame encoded and stored, thereby preventing the processing performed for the index display from being redundant. Specifically, when a certain thumbnail image Thm#i is to be displayed on the list screen, the thumbnail image Thm#i is stored in the proxy file that is the target of the index display. Fig.31Unlike the described case in which the thumbnail image Thm#i encoded by the long GOP inter-frame is stored in the sequence file, it is not necessary to decode another thumbnail image Thm#i' in order to decode the thumbnail image Thm#i.

[0488] Fig.48 10 is a view for explaining an example of index display using a proxy file on an external device other than the digital camera 10.

[0489] For example, when index display is performed on an external device other than the digital camera 10 , instead of a HEIF file as a target of the index display, a proxy file generated by using the HEIF file as a proxy target file may be transmitted to the external device via a network.

[0490] The external device receives the proxy file from the digital camera 10. Then, as in the case where index display is performed on the digital camera 10 by using the proxy file, a list screen is generated and displayed on the external device by using the thumbnail images Thm#i stored in the proxy file.

[0491] Therefore, the index display can be appropriately performed on the external device as in the case where the index display is performed on the digital camera 10 by using the proxy file. Therefore, the time period for displaying the list screen is shortened to realize the quick display of the list screen.

[0492] Furthermore, it is made possible to prevent the processing performed for index display from being redundant.

[0493] In addition, the data size of the proxy file storing the thumbnail image is smaller than the total data size of the HEIF file storing the main image corresponding to the thumbnail image stored in the proxy file. Fig.32 Compared with the transmission of the HEIF file, the transmission of the proxy file from the digital camera 10 to the external device can be completed in a shorter time.

[0494] Therefore, it is made possible to improve the responsiveness from when the user requests the index display on the external device until the list screen is actually displayed.

[0495] Note that after the list screen is displayed on the external device, the user can select any thumbnail image Thm#i in the list screen. When the user selects the thumbnail image Thm#i, the external device can transmit the designated information of the main image Main#i corresponding to the thumbnail image Thm#i in the proxy file and associated with the thumbnail image Thm#i to the digital camera 10. The digital camera 10 can acquire the main image Main#i in the HEIF file specified by the designated information, or, for example, the RAW image in the RAW file corresponding to the main image Main#i as the associated external data from the external device, and transmit the acquired image to the external device for display.

[0496] Fig.49 is a view for explaining an example of selecting a HEIF file used as a proxy target file.

[0497] In the proxy file generation process, for example, Fig.41 As shown in , a plurality of set files in which main images and thumbnail images of one frame (or a plurality of frames) are stored may be selected as proxy target files to generate proxy files.

[0498] In addition, for example, Fig.42 As shown in , (at least) one set file in which main images and thumbnail images of a plurality of frames are stored may be selected as a proxy target file to generate a proxy file.

[0499] Moreover, for example, Fig.43 As shown in , you can select (at least) one sequence file as a proxy target file to generate a proxy file.

[0500] In addition, if Fig.49 As shown in , a HEIF file stored in a specific directory can be selected as a proxy target file to generate a proxy file.

[0501] exist Fig.49 In the example, the DCIM directory exists in the root directory, and the 100MSDCF directory exists in the DCIM directory. In addition, the HEIF file under the 100MSDCF directory is selected as the proxy target file to generate the proxy file.

[0502] exist Fig.49 In the 100MSDCF directory, two set files (HEIF files) named "DSC00001.HEIF" and "DSC00002.HEIF" are stored, each of which stores a main image and a thumbnail image of a frame. Then, a proxy file is generated by using these two set files as proxy target files.

[0503] Fig.50is a view for explaining another example of selecting a HEIF file used as a proxy target file.

[0504] For example, Fig.50 As shown in , a proxy file can be generated by selecting a HEIF file stored in a certain medium (eg, the medium 14 attached to the digital camera 10) as a proxy target file.

[0505] exist Fig.50 In the example, the DCIM directory exists in the root directory of the medium, and the 100MSDCF directory, the 101MSDCF directory, ..., and the 999MSDCF directory exist in the DCIM directory. In addition, the HEIF files under the 100MSDCF to 999MSDCF directories are selected as proxy target files to generate proxy files.

[0506] exist Fig.50 In the example, the 100MSDCF to 999MSDCF directories store N set files (HEIF files) as a whole, each set file stores a main image and a thumbnail image of one frame, and a proxy file is generated by using these N set files as proxy target files.

[0507] <Proxy File Generation Processing for Generating Proxy File of First Mode and Additional Processing for Adding Thumbnail Image to Proxy File>

[0508] Fig.51 It is a view for explaining a proxy file generation process for generating a proxy file of the first mode and an attachment process for adding a thumbnail image to the proxy file.

[0509] Fig.51 An example of the structure of a proxy file of the first mode is shown.

[0510] Fig.51 A shows an example of the structure of a generated proxy file. It should be noted here that it is assumed that the generated proxy file stores a thumbnail image of one main image stored in the proxy target file.

[0511] In order to generate a proxy file, the file control unit 43 generates, for example, a general collection file having a free box including an empty area (beat area) and having an mdat box as a proxy file, and a thumbnail image #1 of a main image stored in the proxy target file is stored as a proxy item in the mdat box.

[0512] A free box is a box that can store any data. The capacity of (the empty area of) a free box is determined according to, for example, the number of thumbnail images (planned) to be stored in the proxy file.

[0513] exist Fig.51In , free boxes are set in the file hierarchy level between the meta box and the mdat box. However, free boxes can also be set in the meta box.

[0514] Fig.51 B shows an example of the structure of a proxy file to which a thumbnail image of another main image has been added.

[0515] You can add a thumbnail image to an already generated proxy file. The process of adding a thumbnail image to an already generated proxy file is also called an adding process.

[0516] In another main image thumbnail image #2 is to be stored in the already generated proxy file (such as Fig.51 In the case of the proxy file shown in A), the file control unit Fig.43 Thumbnail image #2 of another main image is stored as a proxy item by adding thumbnail image #2 after thumbnail image #1 already stored in the mdat box of the proxy file.

[0517] Furthermore, the file control unit 43 rewrites the meta box to a state in which metadata of the thumbnail image #2 as a proxy item is added, and reduces the capacity (size) of the free box by the amount by which the capacity of the meta box is rewritten. A thumbnail image of another main image is added to the proxy file in a similar manner.

[0518] In the case where a thumbnail image is added as a proxy item after the proxy file is generated, it is necessary to add metadata of the attached thumbnail image to the meta box in the proxy file. In this case, without taking measures to add metadata, the offset of the data already stored in the mdat box may be shifted when the metadata is added to the proxy file.

[0519] When the offset of data stored in the mdat box is shifted, it is necessary to calculate the offset shift amount for each piece of data stored in the mdat box and rewrite the iloc box with the offset reflecting the shift amount, which increases the burden of adding processing for adding thumbnail images to proxy files.

[0520] To this end, in this embodiment, a free box is set in the proxy file when the proxy file is generated. Then, when performing the adding process for adding the thumbnail image to the proxy file, metadata is added by using the free box, thereby preventing the offset of the data already stored in the mdat box from shifting.

[0521] Note that in the add process, thumbnail image #2 is stored (written) after thumbnail image #1 has been stored in the mdat box in the proxy file. Therefore, adding thumbnail image #2 to the mdat box itself neither consumes (any empty area in) the free box nor causes any offset of thumbnail image #1 already stored in the mdat box to shift.

[0522] Fig.52 is a flowchart for explaining an example of a proxy file generation process for generating a proxy file of the first mode.

[0523] In step S211 , the file control unit 43 acquires the thumbnail image of the main image stored in the proxy target file as a thumbnail image to be stored in the proxy file as a proxy item, and the process proceeds to step S212 .

[0524] For example, in the case where the proxy target file is a collection file or a sequence file in which all thumbnail images are intra-frame encoded and stored, the file control unit 43 reads a thumbnail image from the collection file or sequence file serving as the proxy target file, thereby obtaining the thumbnail image as a proxy item to be stored in the proxy file.

[0525] Furthermore, for example, in the case where the proxy target file is a sequence file in which thumbnail images are inter-coded and stored by a long GOP, the file control unit 43 generates thumbnail images from the sequence file serving as the proxy target file, thereby acquiring thumbnail images.

[0526] Specifically, for example, the file control unit 43 generates a YUV thumbnail image from a main image (or a RAW image corresponding to the main image) stored in a sequence file serving as a proxy target file and intra-encodes the thumbnail image, thereby acquiring the thumbnail image as a proxy item to be stored in the proxy file. Alternatively, the file control unit 43 decodes and intra-encodes an inter-encoded YUV thumbnail image stored in a sequence file serving as a proxy target file, thereby acquiring the thumbnail image as a proxy item to be stored in the proxy file.

[0527] In step S212 , the file control unit 43 calculates the size of the free box to be set in the proxy file, and the process proceeds to step S213 .

[0528] For example, the file control unit 43 determines the maximum value of the number of thumbnail images to be stored in the proxy file (hereinafter also referred to as the maximum number of thumbnails). Then, the file control unit 43 calculates the size of a free box that allows storage of metadata of at least the maximum number of thumbnails according to the maximum number of thumbnails, so that the thumbnail images of the maximum number of thumbnails can be stored in the proxy file without shifting the offset of the data stored in the mdat box in the proxy file.

[0529] The maximum number of thumbnails may be determined according to, for example, the remaining capacity of a medium (such as the medium 14) in which the proxy target file and the proxy file are recorded.

[0530] In addition, assuming that taking a photo (image) is referred to as a single shot as compared to a continuous shot in which multiple photos are taken in a single shutter operation on the digital camera 10, the maximum number of thumbnails can be determined based on, for example, the imaging mode of the digital camera 10, where the imaging mode is either a continuous shot mode or a single shot mode and can be set for each HEIF file that serves as a proxy target file.

[0531] Furthermore, the maximum number of thumbnails may be determined based on, for example, a determined maximum number of primary images that may be stored in one HEIF file serving as a proxy target file.

[0532] Furthermore, the maximum number of thumbnails may be determined based on, for example, the number of main images having good evaluations, which is performed on the main images stored in the proxy target file by the file control unit 43. In the evaluation of the main images, for example, by obtaining information on image quality such as S / N and clarity of the main images, and main images having S / N, clarity, etc. equal to or greater than a threshold value may be determined as main images having good evaluations.

[0533] Furthermore, the maximum number of thumbnails may be determined according to a number of factors such as the remaining capacity of the medium as described above.

[0534] In addition, in the case where a HEIF file stored in a predetermined directory or predetermined medium is used as a proxy target file, the maximum number of thumbnails can be determined based on, for example, a determined maximum number of main images that can be stored in one HEIF file, wherein the one HEIF file is stored in a predetermined directory or predetermined medium and used as a proxy target file.

[0535] In step S213, the file control unit 43 generates a common collection file as a proxy file, which includes a free box having the size calculated in step S212 and in which thumbnail images of the main images stored in the proxy target file are stored as proxy items in the mdat box, and the proxy file generation process ends.

[0536] Fig.53 is a flowchart for explaining an example of an adding process for adding a thumbnail image to a proxy file of the first mode.

[0537] In the case where a proxy file already exists, when a thumbnail image is to be added to the proxy file, an adding process is performed on the proxy file that is the target of the adding process.

[0538] In step S221, the file control unit 43 acquires a thumbnail image of a main image stored in the proxy target file as a thumbnail image of a proxy item to be added to the proxy file as a target of the adding process (hereinafter also referred to as an additional thumbnail image), and the process proceeds to step S222. Fig.52 The additional thumbnail image is acquired in step S221 in a similar manner to step S211 in FIG.

[0539] In step S222, the file control unit 43 adds the additional thumbnail image as a proxy item to the mdat box in the proxy file that is the target of the adding process, and the process proceeds to step S223.

[0540] In step S223 , the file control unit 43 acquires the meta box by reading the meta box from the proxy file that is the target of the adding process, and the process proceeds to step S224 .

[0541] In step S224, the file control unit 43 acquires the remaining capacity of the free box in the proxy file that is the target of the adding process, and confirms that the remaining capacity is sufficient to add metadata of the additional thumbnail image, and then the process proceeds to step S225.

[0542] Note that, in step S224, in the case where the remaining capacity of the free box is insufficient to add metadata of the additional thumbnail image, for example, information indicating such insufficiency is displayed on the liquid crystal panel 19, and the adding process ends. Also, in this case, a proxy file generation process for generating a new proxy file storing the additional thumbnail image may be performed.

[0543] In step S225, in accordance with adding the additional thumbnail image as a proxy item to the mdat box in the proxy file that is the target of the adding process, the file control unit 43 regenerates a new meta box by adding metadata of the additional thumbnail image to the meta box acquired in step S223. Furthermore, the file control unit 43 overwrites the meta box in the proxy file that is the target of the adding process with the new meta box, and the process proceeds from step S225 to step S226.

[0544] In step S226, the file control unit 43 generates a new free box whose size is reduced by the amount of increase in the size (data amount) of the new meta box relative to the meta box before rewriting. Furthermore, the file control unit 43 rewrites the free box in the proxy file that is the target of the adding process with the new free box, and the adding process ends.

[0545] <Proxy File Generation Processing for Generating Proxy File of Second or Third Mode and Addition Processing for Adding Thumbnail Image to Proxy File>

[0546] Fig.54 It is a view for explaining a proxy file generation process for generating a proxy file of the second or third mode and an adding process for adding a thumbnail image to the proxy file.

[0547] Fig.54 An example of the structure of the proxy file of the second or third mode is shown.

[0548] Fig.54 A shows an example of the structure of a generated proxy file. It should be noted here that it is assumed that the generated proxy file stores a thumbnail image of one main image stored in the proxy target file.

[0549] To generate a proxy file, the file control unit 43 generates, as the proxy file, an association type set file having, for example, a free box and having an mdat box in which a thumbnail image #1 of one main image stored in the proxy target file is stored as a proxy item.

[0550] In the association type set file as a proxy file, thumbnail image #1 is stored in association with (the designated information of) thumbnail image #1. That is, in the association type set file as a proxy file, relationship information related to the association between the thumbnail image in the proxy file and the designated information of the main image as external data corresponding to the thumbnail image is stored.

[0551] As relational information, for example, there is storage in Fig.10 The association information in the association information storage box of the first association type set file.

[0552] Furthermore, as relational information, for example, there is Fig.11 The item Item#201 in the mdat box of the second association type collection file, the infe box for storing the item Item#201 in the iinf box of the meta box, the offset of the storage position of the item Item#201 stored in the iloc box in the iprp box of the meta box, etc.

[0553] In addition, as relational information, for example, there is Fig.12 The third association type in the collection collects designation information of an item in the mdat box in the file, and association information in which the item ID of a main image in the cdsc box in the iref box in the meta box stored as an item is associated with the item ID as the designation information of the item.

[0554] Furthermore, as relational information, for example, there is stored Fig.14 Track #3 of the designated information in the mdat box of the associated type sequence file, and the trak box of track #3 which is stored in the moov box and manages the designated information.

[0555] exist Fig.54 In the example, a third association type set file is generated as a proxy file.

[0556] In addition, Fig.54 In A, the designation information #1 is the designation information of the main image corresponding to the thumbnail image #1 and is a part of the relationship information.

[0557] In the proxy file as the third association type set file, the designated information #1 is stored in the mdat box. Regarding the relationship information related to the association between the thumbnail image #1 and the designated information #1, information other than the designated information #1 (such as, for example, the item ID of the associated thumbnail image #1 and the reference Fig.12 The association information of the item ID of the designated information #1 stored in the cdsc box of the iref box of the meta box, and the metadata including the offset of the storage location of the designated information #1) is stored in the meta box in the proxy file which is a third association type collection file.

[0558] Fig.54 B shows an example of the structure of a proxy file to which a thumbnail image of another main image has been added.

[0559] In order to additionally store a thumbnail image #2 of another main image in an already generated proxy file (such as, for example Fig.54 In the adding process in the proxy file shown in A of the proxy file, the file control unit 43 stores thumbnail image #2 of another main image as a proxy item by adding thumbnail image #2 after thumbnail image #1 and designation information #1 already stored in the mdat box of the proxy file.

[0560] Furthermore, the file control unit 43 rewrites the meta box to a state in which the metadata of the thumbnail image #2 as a surrogate is added thereto.

[0561] Thumbnail image #2 of another main image is stored in association with designation information #2 of the main image corresponding to thumbnail image #2. That is, the file control unit 43 stores designation information #2 as a proxy item by adding designation information #2 to thumbnail image #2 in the mdat box in the proxy file.

[0562] Furthermore, the file control unit 43 rewrites the meta box to a state in which metadata of the thumbnail image #2 as a surrogate is added, and relationship information related to the association between the thumbnail image #2 and the designation information #2 is added in addition to the designation information #2.

[0563] The file control unit 43 reduces the capacity of the free box by the amount by which the capacity of the meta box is increased by rewriting the meta box. A thumbnail image of another main image is added to the proxy file in a similar manner.

[0564] Note that in the adding process, thumbnail image #2 and designation information #2 are written so as to be added after thumbnail image #1 and designation information #1 already stored in the mdat box. Therefore, adding thumbnail image #2 and designation information #2 to the mdat box itself neither consumes free boxes nor causes any offset shift of thumbnail image #1 and designation information #1 already stored in the mdat box.

[0565] Fig.55 is a flowchart for explaining an example of a proxy file generation process for generating a proxy file of the second or third mode.

[0566] like Fig.52 In step S211 in the above description, in step S241, the file control unit 43 acquires the thumbnail image of the main image stored in the proxy target file as the thumbnail image of the proxy item to be stored in the proxy file, and the processing proceeds to step S242.

[0567] In step S242 , the file control unit 43 acquires designation information of a main image corresponding to the thumbnail image acquired in step S241 , and the process proceeds to step S243 .

[0568] In the case where the HEIF file serving as a proxy target file storing a main image is a collection file, as the designated information of the main image, for example, uuid, a hash value calculated by using the main image as input, a collection of URLs assigned to the proxy target file storing the main image, and the item ID of the main image, etc. may be used.

[0569] In the case where the HEIF file serving as a proxy target file storing a main image is a sequence file, as the designated information of the main image, for example, uuid, a set of URLs assigned to the proxy target file storing the main image, the track ID of the track of the frame of the main image, the sequence number of the frame of the main image starting from the track header (frame offset), a set of URLs assigned to the proxy target file storing the main image, and time information of the frame of the main image on the timeline, etc. may be used.

[0570] Note that in the case where the designated information is capable of specifying data (main image) in the proxy target file when the designated information is written to the proxy target file, for example, in the case where the designated information is uuid, in step S242, after the designated information of the main image is generated, the designated information is written to the proxy target file storing the main image to be specified by the designated information.

[0571] In addition, in the case where the proxy target file is an associated type HEIF file and the designation information of the external data associated with the main image stored in the associated type HEIF file is stored in the associated type HEIF file as the proxy target file, in step S242, the designation information of the external data can be obtained as the designation information of the main image. In this case, since the designation information has been written to the associated type HEIF file as the proxy target file, it is not necessary to rewrite the designation information of the main image.

[0572] like Fig.52 As in step S212 in , in step S243, the file control unit 43 calculates the size of the free box to be set in the proxy file, and the process proceeds to step S244.

[0573] Note that, in the case where a proxy file of the second or third mode is subjected to adding processing as a target proxy file for adding processing, the meta box in the proxy file stores not only metadata of the thumbnail image to be added but also relationship information related to the association between the thumbnail image to be added and designated information of the main image corresponding to the thumbnail image (which is to be stored in the meta box).

[0574] Therefore, even if the maximum number of thumbnails of the thumbnail images stored in the proxy file of the second or third mode is the same as the maximum number of thumbnails of the thumbnail images stored in the proxy file of the first mode, Fig.55 The size of the free box calculated in step S243 is also larger than that in Fig.52 The size of the free box calculated in step S212.

[0575] In step S244, the file control unit 43 generates an associated type collection file as a proxy file, which includes a free box having the size calculated in step S243, and a thumbnail image of the main image stored in the proxy target file is stored in the associated type collection file as a proxy item in the mdat box, and the proxy file generation processing is completed.

[0576] In the generation of the proxy file in step S244, an association type set file is generated as a proxy file, in which the thumbnail image as the proxy item is stored in association with the designated information of the main image corresponding to the thumbnail image (i.e., the designated information acquired in step S242). For example, a proxy file is generated that stores, together with the thumbnail image as the proxy item, relationship information related to the association between the thumbnail image and the designated information of the main image corresponding to the thumbnail image.

[0577] Fig.56 is a flowchart for explaining an example of adding processing for adding a thumbnail image to a proxy file of the second or third mode.

[0578] In the case where a proxy file already exists, when a thumbnail image is to be added to the proxy file, an adding process is performed on the proxy file that is the target of the adding process. It is assumed here that the proxy file that is the target of the adding process is, for example, a third association type collection file ( Fig.12 ).

[0579] In step S251, the file control unit 43 acquires a thumbnail image of a main image stored in a proxy target file as an additional thumbnail image to be added to a proxy item of the proxy file that is a target of the adding process, and the process proceeds to step S252. Fig.55 An additional thumbnail image is acquired in step S251 in a similar manner to step S241 in FIG.

[0580] In step S252, the file control unit 43 executes Fig.55 Similar processing to that in step S242 in is performed, such as acquiring designation information of a main image corresponding to the additional thumbnail image acquired in step S241, and the processing proceeds to step S253.

[0581] In step S253, the file control unit 43 adds the additional thumbnail image and the designation information of the main image corresponding to the additional thumbnail image as a proxy item to the mdat box in the proxy file serving as the target of the adding process, and the process proceeds to step S254.

[0582] In step S254, the file control unit 43 acquires the meta box by reading the meta box from the proxy file serving as the target of the adding process, and the process proceeds to step S255.

[0583] In step S255, the file control unit 43 obtains the remaining capacity of the free box in the proxy file used as the target of the adding processing, and confirms that the remaining capacity is sufficient to add the metadata of the additional thumbnail image and to add the relationship information related to the association between the additional thumbnail image and the designated information of the main image corresponding to the additional thumbnail image (however, the designated information that has been added to the mdat box in step S253 is excluded here), and the processing proceeds to step S256.

[0584] Note that if it is determined in step S255 that the remaining capacity of the free box is insufficient for the addition, then the same process as reference may be performed. Fig.53 The process is similar to the process described in step S224 in .

[0585] In step S256 , the file control unit 43 regenerates a new meta box from the meta box acquired in step S254 .

[0586] A new meta box is generated by adding, to the meta box acquired in step S254, metadata of an additional thumbnail image that is a proxy item of the mdat box added to the proxy file serving as a target of the adding process, and relationship information related to the association between the additional thumbnail image and designated information of the main image corresponding to the additional thumbnail image (however, designated information that has already been added to the mdat box in step S253 is excluded here).

[0587] The file control unit 43 rewrites the meta box in the proxy file serving as the target of the adding process with the new meta box, and the process proceeds from step S256 to step S257.

[0588] In step S257, the file control unit 43 generates a new free box whose size is reduced by the amount of increase in the size (data amount) of the new meta box relative to the meta box before rewriting. Furthermore, the file control unit 43 rewrites the free box in the proxy file serving as the target of the adding process with the new free box, and the adding process ends.

[0589] <Detection Processing for Detecting a Main Image Corresponding to a Thumbnail Image Stored in a Proxy File of the Second Mode>

[0590] Fig.57 is a view for explaining an example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file in the second mode.

[0591] Fig.57 An example of a set of proxy files and proxy target files of the second mode is shown.

[0592] exist Fig.57 In the example, a proxy file is generated, and the set files #1 and #2 are used as proxy target files. As the proxy file, for example, a set file of the third association type is used.

[0593] In the set file #i, the main image #i and the thumbnail image #i of the main image #i are stored in the mdat box.

[0594] In the proxy file, thumbnail images #1 and #2 and designation information #1 and #2 of main images corresponding to the thumbnail images #1 and #2 are stored in such a manner that the thumbnail images #i and the designation information #i are associated with each other.

[0595] The set file #i used as the proxy target file of the second mode does not include the designation information #i of the main image #i. Therefore, in the second mode, as the designation information #i of the main image #i, information i that can designate the main image #i even if the designation information #i of the main image #i is not written in the set file #i (such as, for example, a hash value obtained by using the main image #i as an input) is adopted.

[0596] Now, it is assumed that, for example, a hash value obtained by using the main image #i as an input is adopted as the designation information #i of the main image #i.

[0597] In this case, among hash values ​​generated by using main images #1 and #2 in set files #1 and #2 respectively serving as proxy target files as input, main image #i providing a hash value matching a hash value as designated information #i is detected as a main image corresponding to thumbnail image #i in the proxy file, and designated information #i is associated with thumbnail image #i as a proxy item in the proxy file.

[0598] Fig.58 is a view for further explaining an example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the second mode.

[0599] Fig.58 An example of a set of proxy files and proxy target files of the second mode is shown.

[0600] exist Fig.58 In the example, a proxy file is generated, and set files #1 and #2 are used as proxy target files. As the proxy file, for example, a first association type set file is used. As set files #1 and #2 used as proxy target files, ordinary set files are used.

[0601] In set file #1, main images A and B and thumbnail images A and B of main images A and B are stored in the mdat box. In set file #2, main images C and D and thumbnail images C and D of main images C and D are stored in the mdat box.

[0602] In the proxy file, thumbnail images A to D and hash values ​​#A to #D as designation information of the main images A to D are stored in a manner that the thumbnail images A to D are associated with the hash values ​​#A to #D as designation information, respectively. That is, in the proxy file, the association information storage box stores association information that associates the item IDs of the thumbnail images A to D with the hash values ​​#A to #D as designation information, respectively. The hash values ​​#A to #D as designation information are hash values ​​obtained by using the main images A to D as inputs, respectively.

[0603] In this case, among hash values ​​generated by using main images A and D in set files #1 and #2 respectively serving as proxy target files as input, a main image providing a hash value matching a hash value serving as designated information is detected as a main image corresponding to thumbnail image X in the proxy file, wherein the designated information is associated with a certain thumbnail image X (where X is any one of A to D) serving as a proxy item in the proxy file.

[0604] Fig.59 is a flowchart for explaining an example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file in the second mode.

[0605] For example, the UI control unit 47 causes the file control unit 43 and the like to perform index display of generating a list screen using thumbnail images stored in the proxy file and displaying the list screen on the liquid crystal panel 19 .

[0606] For example, the file control unit 43 generates a list of handles for identifying the respective thumbnail images stored in the proxy file, which are main images for the proxy file. The UI control unit 47 requests the file control unit 43 to perform index display of the thumbnail images stored in the proxy file in response to, for example, a user's operation, etc.

[0607] In response to a request to perform index display made by the UI control unit 47 , the file control unit 43 reads (a frame of) a thumbnail image identified by a handle in the handle list from the proxy file, and supplies the thumbnail image to the display control unit 46 via the encoding control unit 42 .

[0608] The display control unit 46 generates a list screen of thumbnail images from the file control unit 43 and displays the list screen on the liquid crystal panel 19 .

[0609] Thereafter, for example, when the user selects a desired thumbnail image from the list screen, in step S271, the file control unit 43 obtains designated information associated with the thumbnail image selected from the list screen (hereinafter also referred to as the selected thumbnail image) by reading the designated information from the proxy file, and the processing proceeds to step S272.

[0610] That is, for example, the file control unit 43 specifies the handle of the thumbnail image selected in the handle list (the selected related image), and acquires the specified information associated with the selected thumbnail image identified by the handle by reading the specified information in the proxy file.

[0611] In step S272 , the file control unit 43 selects a HEIF file to be searched for detecting a main image, and the process proceeds to step S273 .

[0612] Here, as reference Fig.49 , 50 As described in the proxy file generation process (and addition process), one or more collection files, one or more sequence files, HEIF files stored in a specific directory, or HEIF files in a specific medium can be selected as proxy target files.

[0613] The file control unit 43 selects the HEIF file selected as the proxy target file for the proxy file as the search target. Information necessary for specifying the proxy target file (such as information for specifying a specific directory or medium in the case where a HEIF file stored in a specific directory or medium is selected as the proxy target file) can be stored in, for example, a meta box or an mdat box in the proxy file.

[0614] In step S273 , the file control unit 43 searches for a main image in the search target HEIF file to detect the main image specified by the specifying information associated with the selected thumbnail image, and the detection process ends.

[0615] For example, in the case of adopting a hash value calculated by sha-256 or the like using the main image as input as the designation information of the main image, the file control unit 43 obtains a hash value calculated by using a (single) main image in the HEIF file as input for each search target HEIF file.

[0616] That is, for example, in the case where the search target HEIF file is a collection file, the file control unit 43 acquires a list of item IDs of main images in the collection file (main image list) for each search target collection file.

[0617] Furthermore, for each item ID in the main image list, based on the information in the iloc box in the search target set file, the file control unit 43 reads (the data of) the main image from the mdat box as the item specified by the item ID. The file control unit 43 calculates a hash value using the main image read from the mdat box in the search target set file as an input.

[0618] Then, the file control unit 43 detects the main image #i providing the hash value matching the hash value as the specifying information associated with the selected thumbnail image as the main image corresponding to the selected thumbnail image.

[0619] <Detection Processing for Detecting a Main Image Corresponding to a Thumbnail Image Stored in a Proxy File in the Third Mode>

[0620] Fig.60 is a view for explaining an example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode.

[0621] Fig.60 An example of a collection of a proxy target file and a proxy file of the third mode is shown.

[0622] exist Fig.60 In , a proxy file is generated, with set files #1 and #2 as proxy target files.

[0623] As the proxy file, for example, the third association type set file is adopted. The third association type set file is also adopted as the set files #1 and #2 used as the proxy target files.

[0624] and Fig.57 As in the case of , in the set file #i, the main image #i and the thumbnail image #i of the main image #i are stored in the mdat box.

[0625] Furthermore, in the set file #i, the main image #i and the specifying information #i of the main image #i are stored in such a manner that the main image #i and the specifying information #i are associated with each other.

[0626] In the proxy file, with Fig.57 As in the case of , thumbnail images #1 and #2 of the main image and the designation information #1 and #2 are stored in such a manner that the thumbnail image #i and the designation information #i are associated with each other.

[0627] The set file #i used as the proxy target file of the third mode includes the designation information #i of the main image #i. Therefore, in the third mode, any information can be used as the designation information of the main image as long as the designation information of the main image can be written into the set file #i used as the proxy target file.

[0628] Now, it is assumed that, for example, uuid is adopted as the designation information #i of the main image #i.

[0629] In this case, primary image #i (which is any one of main images #1 and #2 in collection files #1 and #2 which are proxy target files, respectively, and is associated with a uuid that matches the uuid of designated information #i, which is associated with thumbnail image #i which is a proxy item in the proxy file) is detected as the primary image corresponding to thumbnail image #i in the proxy file.

[0630] Fig.61 is a view for further explaining an example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode.

[0631] Fig.61 An example of a collection of a proxy target file and a proxy file of the third mode is shown.

[0632] exist Fig.61 In , a proxy file is generated, with set files #1 and #2 as proxy target files.

[0633] As the proxy file, for example, a first association type set file is used. As the set files #1 and #2 used as the proxy target files, the first association type set files are also used.

[0634] In the set file #1, main images A and B and thumbnail images A and B of the main images A and B are stored in the mdat box.

[0635] Furthermore, in the set file #1, the association information storage box in the meta box stores, as the specifying information of the main images A and B, association information associating the item IDs of the main images A and B with the UUIDs #A and #B, respectively.

[0636] In the set file #2, main images C and D and thumbnail images C and D of the main images C and D are stored in the mdat box.

[0637] Furthermore, in the set file #2, the association information storage box in the meta box stores, as the specifying information of the main images C and D, association information associating the item IDs of the main images C and D with the UUIDs #C and #D, respectively.

[0638] In the proxy file, thumbnail images A to D and hash values ​​#A to #D as designation information of main images A to D are stored in such a manner that the thumbnail images A to D are associated with the hash values ​​#A to #D as designation information, respectively.

[0639] That is, in the proxy file, the association information storage box stores association information that associates the item IDs of the thumbnail images A to D with UUID#A to UUID#D as designation information.

[0640] In this case, a main image that is one of the main images A to D in the collection files #1 and #2 that are proxy target files and is associated with designated information associated with a certain thumbnail image X that is a proxy item in the proxy file (i.e., a main image associated with a uuid that matches the uuid associated with the item ID of thumbnail image X in the associated information storage box in the proxy file) is detected as a main image corresponding to thumbnail image X in the proxy file.

[0641] Fig.62 is a view for explaining a first example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode.

[0642] Fig.62 It shows that the first association type set file ( Fig.10 ) is an example of a proxy target file in the case of a proxy target file.

[0643] In the first example of the detection process, it is assumed that a first association type set file is adopted as a proxy target file.

[0644] In the first association type set file as a proxy target file, the association information storage box is stored in the meta box. Fig.62 In the associated information storage box in , information indicating that the number of main images is 1 is stored, and item ID=1 of the main image and UUID#1 which is the designation information of the main image are stored in association with each other.

[0645] In the case of adopting the first association type set file as the proxy target file, the file control unit 43 detects a proxy target file in which the association information storage box stores a uuid matching the uuid as the designated information associated with the thumbnail image selected in the proxy file in the detection process.

[0646] Then, in the proxy target file, the file control unit 43 detects a main image of an item ID associated with a uuid matching the uuid as the designation information associated with the selected thumbnail image as a main image corresponding to the selected thumbnail image.

[0647] Fig.63 is a flowchart for explaining a first example of a detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file in the third mode.

[0648] For example, as referenced Fig.59 As described, the UI control unit 47 causes execution of index display of generating a list screen using thumbnail images stored in a proxy file and displaying the list screen on the liquid crystal panel 19 .

[0649] Thereafter, for example, when the user selects a desired thumbnail image from the list screen, in step S281, as shown in FIG. Fig.59 As in step S271 in , the file control unit 43 acquires the designation information associated with the selected thumbnail image selected from the list screen from the proxy file by reading the designation information, and the process proceeds to step S282.

[0650] In step S282, if Fig.59 As in step S272 in , the file control unit 43 selects the HEIF file to be searched to detect the main image, and the processing proceeds to step S283.

[0651] In step S283, the file control unit 43 detects a HEIF file in which an associated information storage box stores specified information (hereinafter also referred to as matching specified information) that matches the specified information associated with the selected thumbnail image from the search target HEIF file as a detection target file for detecting the main image corresponding to the selected thumbnail image.

[0652] Moreover, the file control unit 43 obtains access information for accessing the main image associated with the matching specified information (the main image specified by the matching specified information) in the detection target file by reading the access information from the associated information storage box in the detection target file, and the processing proceeds from step S283 to step S284.

[0653] As reference Fig.62 As described above, in the first example of the detection process, the first association type set file is used as the proxy target file. Therefore, in the first example of the detection process, the detection target file is the first association type set file.

[0654] In the first associated type set file as the detection target file, as referenced Fig.62 As described above, the association information storage box stores the item ID of the main image and the specified information (uuid, etc.) of the main image in association with each other.

[0655] In the association information storage box in the first association type set file as the detection target file, the file control unit 43 acquires the item ID (of the main image) associated with the matched designation information as access information.

[0656] In step S284 , the file control unit 43 uses the access information to detect the main image specified by the specifying information associated with the selected thumbnail image from the detection target file, and the detection process ends.

[0657] That is, the file control unit 43 detects the main image specified by the item ID as the access information from the detection target file by reading the main image as the main image specified by the specifying information associated with the selected thumbnail image. Fig.18 The described method reads a main image specified by an item ID as access information from a detection target file.

[0658] Fig.64 is a view for explaining a second example of the detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode.

[0659] Fig.64 It shows that the second association type set file ( Fig.11 ) is an example of a proxy target file in the case of a proxy target file.

[0660] In the second example of the detection process, it is assumed that a second association type set file is used as the proxy target file.

[0661] In the second association type collection file as a proxy target file, association information stored in the association information storage box in the first association type collection file and including the number of main images and the item ID of the main image associated with the uuid as the designated information of the main image is stored as an item in the mdat box.

[0662] In the case of adopting the second association type set file as the proxy target file, the file control unit 43 detects in the detection process a proxy target file in which the mdat box stores association information including a uuid matching the uuid that is the designated information associated with the thumbnail image selected in the proxy file.

[0663] Then, in the proxy target file, the file control unit 43 detects a main image of an item ID associated with a uuid matching the uuid as the designation information associated with the selected thumbnail image as a main image corresponding to the selected thumbnail image.

[0664] In a second example of the detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode, Fig.63 The detection processing in the first example is similar to the processing.

[0665] Note, however, that in the second example of the detection process, Fig.63 In step S283, regarding the search target HEIF file (the second associated type set file ( Fig.11 )), the file control unit 43 acquires the item ID of the item type in the Infe box and including the IDIF in the infe box in the iinf box as the item ID which is the association information of the item.

[0666] Also, regarding the search target HEIF file, the file control unit 43 searches the iloc box for an item ID which is associated information of an item, and reads the offset and size associated with the item ID.

[0667] Then, based on the offset and size associated with the item ID as the associated information of the item, the file control unit 43 reads the associated information as the item stored in the mdat box in the search target HEIF file.

[0668] As described above, the file control unit 43 detects the associated information including the specifying information (matched specifying information) matching the specifying information associated with the selected thumbnail image from the associated information read from the search target HEIF file.

[0669] Thereafter, the file control unit 43 detects the search target HEIF file having the associated information including the matching designated information as an item as a detection target file. Then, among the associated information in the detection target file, the file control unit 43 obtains the item ID (of the main image) associated with the matching designated information as access information for accessing the main image.

[0670] Fig.65 is a view for explaining a third example of the detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode.

[0671] Fig.65 It shows that the third association type set file ( Fig.12 ) is an example of a proxy target file in the case of a proxy target file.

[0672] In the third example of the detection process, it is assumed that a third association type set file is adopted as the proxy target file.

[0673] In the third association type set file as the proxy target file, uuid as the designation information of the main image is stored as an item in the mdat box.

[0674] Furthermore, association information associating the item ID of the main image with the item ID of the designated information of the main image is stored in the cdsc box in the iref box in the meta box ( Fig.12 ).

[0675] In the case of adopting the third association type set file as the proxy target file, the file control unit 43 detects a proxy target file in which the mdat box stores a uuid matching the uuid as the designation information associated with the thumbnail image selected in the proxy file in the detection process.

[0676] Then, in the proxy target file, the file control unit 43 detects a main image of an item ID associated with a uuid matching the uuid that is the designation information associated with the selected thumbnail image as a main image corresponding to the selected thumbnail image.

[0677] In a third example of the detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode, the process is performed in accordance with Fig.63 The detection processing in the first example is similar to the processing.

[0678] However, it is noted that in the third example of the detection process, Fig.63 In step S283, regarding the search target HEIF file (the third associated type set file ( Fig.12 )), the file control unit 43 acquires the item ID of the item type in the Infe box and including the IDIF in the infe box in the iinf box as the item ID which is the designation information of the item.

[0679] Also, regarding the search target HEIF file, the file control unit 43 searches the iloc box for an item ID as designation information of an item, and reads the offset and size associated with the item ID.

[0680] Then, based on the offset and the size associated with the item ID as the specifying information of the item, the file control unit 43 reads the specifying information in the mdat box stored in the search target HEIF file as the item.

[0681] As described above, the file control unit 43 detects the specifying information (matching specifying information) that matches the specifying information associated with the selected thumbnail image from the specifying information read from the search target HEIF file.

[0682] Thereafter, the file control unit 43 detects the search target HEIF file having the matching specified information as an item as a detection target file.

[0683] Then, in the detection target file meta box ( Fig.12), the file control unit 43 obtains the item ID (master image) associated with the item ID of the designated information that is a match of the item, as access information for accessing the master image, that is, the item ID is the reference source associated with the reference destination, which is the item ID of the designated information that is a match of the item.

[0684] Fig.66 is a view for explaining a fourth example of the detection process for detecting a main image corresponding to a thumbnail image stored in the proxy file of the third mode.

[0685] Fig.66 It shows that the associated type sequence file ( Fig.14 ) is an example of a proxy target file in the case of a proxy target file.

[0686] In the fourth example of the detection process, it is assumed that a correlation type sequence file is adopted as a proxy target file.

[0687] In an associated type sequence file as a proxy target file, a track of uuid as designated information of a main image is stored in an mdat box. In an associated type sequence file as a proxy target file, (a frame of) a main image in a track of a main image (hereinafter also referred to as a main image track) and uuid in a track of designated information as designated information of the main image (hereinafter also referred to as a designated information track) are associated with each other and stored according to time information on a timeline (or a sequence number of arrangement on a track).

[0688] In the case of adopting an association type sequence file as a proxy target file, the file control unit 43 detects a proxy target file in which an mdat box stores a uuid matching the uuid as the designation information associated with the thumbnail image selected in the proxy file in the detection process.

[0689] Then, in the proxy target file, the file control unit 43 detects the main image specified by the uuid which is the specifying information associated with the selected thumbnail image as the main image corresponding to the selected thumbnail image.

[0690] In a fourth example of the detection process for detecting a main image corresponding to a thumbnail image stored in a proxy file of the third mode, the process is performed in accordance with Fig.63 The detection processing in the first example is similar to the processing.

[0691] However, it is noted that in the fourth example of the detection process, Fig.63In step S283, regarding the search target HEIF file (associated type sequence file ( Fig.14 )), the file control unit 43 searches the trak box in the moov box for a trak box in which the type in the tref box is cdsc, as the trak box that manages the designated information track.

[0692] Furthermore, the file control unit 43 reads the track ID of the designated information track from the tkhd box in the trak box that manages the designated information track, and reads the designated information in the track of the track ID, that is, the designated information in the designated information track. As described above, the file control unit 43 detects designated information (matched designated information) that matches the designated information associated with the selected thumbnail image from the designated information read from the search target HEIF file.

[0693] Thereafter, the file control unit 43 detects a search target HEIF file having a designated information track including the matching designated information as a detection target file.

[0694] Then, in the detection target file, the file control unit 43 obtains the trak box ( Fig.14 ) in the tref box (track ID of the main image track) as access information for accessing the main image, and also obtains the time information of the matching designated information on the timeline (or the serial number of the matching designated information arranged in the designated information track).

[0695] <Proxy File Generation Processing and Addition Processing Executed When Generating Thumbnail Images to be Stored in Proxy Files When Generating Proxy Target Files>

[0696] Fig.67 It is a view for explaining a proxy file generation process and an addition process which are performed in the case of generating a thumbnail image to be stored in a proxy file when a proxy target file is generated.

[0697] Fig.67 An example of the structure of a proxy file of the first mode is shown.

[0698] Fig.67 A shows an example of the structure of a proxy file generated by the proxy file generation process.

[0699] In a proxy file generation process that is performed in the case where a thumbnail image to be stored in a proxy file of a first mode is generated when a proxy target file is generated, the file control unit 43 generates a thumbnail image to be stored in the proxy file of the first mode (hereinafter also referred to as a proxy thumbnail image) by using an original image that was used to generate a thumbnail image stored in the proxy target file when the proxy target file was generated (hereinafter also referred to as a current thumbnail image).

[0700] As the proxy thumbnail image, an image identical to the current thumbnail image (a copy) may be used, or an image different from the current thumbnail image in terms of image quality, data volume, etc. (an image with lower image quality or smaller data volume) may be used.

[0701] For example, when the YUV signal format of the current thumbnail image is 4:2:2, 4:2:0 may be used as the signal format of the proxy thumbnail image. Also, for example, when the grayscale of the current thumbnail image is 10 bits, 8 bits may be used as the signal format of the proxy thumbnail image. Also, for example, as the proxy thumbnail image, an image having a smaller size (number of pixels) than the current thumbnail image may be used.

[0702] Furthermore, for example, for the proxy thumbnail image, encoding at a compression rate different from that used for the current thumbnail image, for example, encoding at a compression rate higher than that used for the current thumbnail image may be employed.

[0703] Moreover, for example, a different type of codec may be used for the proxy thumbnail image than that of the current thumbnail image, for example, HEVC may be used for the current thumbnail image, while JPEG may be used for the proxy thumbnail image.

[0704] The file control unit 43 generates, as a proxy file of the first mode, a general collection file having, for example, a free box and an mdat box storing, as a proxy item, a proxy thumbnail image which is a thumbnail image of a main image stored in a proxy target file.

[0705] exist Fig.67 In A, a proxy file storing two proxy thumbnail images A and B (as main images) has been generated through the proxy file generation process.

[0706] Furthermore, a free box is set in the meta box. Metadata of the proxy thumbnail images A and B are stored in the meta box in the proxy file generated by the proxy file generation process.

[0707] Fig.67B shows an example of the structure of a proxy file to which a proxy thumbnail image of another main image is added by adding processing.

[0708] For example, in an adding process performed in the case where a proxy thumbnail image to be stored in a proxy file of a first mode is generated when a proxy target file is generated, when a HEIF file generated after the proxy file is generated is used as a new proxy target file, a proxy thumbnail image of the main image stored in the new proxy target file is added to the already generated proxy file (hereinafter also referred to as an existing proxy file).

[0709] That is, the file control unit 43 generates a proxy thumbnail image to be added to the proxy file of the first mode using an original image used to generate a current thumbnail image stored in a new proxy target file when generating the new proxy target file.

[0710] exist Fig.67 In B, Fig.67 After the proxy thumbnail images A and B are already stored in the mdat box in the existing proxy file in A, two proxy thumbnail images C and D (as main images) have been added through the adding process.

[0711] When the proxy thumbnail images C and D are added, the file control unit 43 rewrites the meta box to a state in which the metadata of the proxy thumbnail images C and D are added, and reduces the capacity of the free box by the amount by which the capacity of the meta box is rewritten. In a similar manner, the proxy thumbnail images of the primary images stored in the still new proxy target file are added to the existing proxy file.

[0712] Fig.68 is a flowchart for explaining an example of a proxy file generation process performed in the case of generating a thumbnail image to be stored in a proxy file of the first mode when generating a proxy target file.

[0713] When a HEIF file is generated and a proxy file generation process execution condition for executing a proxy file generation process using the HEIF file as a proxy target file is satisfied, the file control unit 43 executes the following proxy file generation process.

[0714] As the proxy file generation processing execution condition, for example, the fact that there is no proxy file in the directory or medium storing the proxy target file can be adopted. In addition, as the proxy file generation processing execution condition, for example, in the case where there is a proxy file in the directory or medium storing the proxy target file, the fact that the number of thumbnail images stored in the proxy file has reached the upper limit can be adopted.

[0715] In step S311, the file control unit 43 generates a proxy thumbnail image to be stored in the proxy file of the first mode using, for example, an original image used to generate the current thumbnail image stored in the proxy target file, and the process proceeds to step S312.

[0716] Note that in step S311 , the current thumbnail image generated from the original image may be acquired (copied) as it is as the proxy thumbnail image, instead of generating the proxy thumbnail image from the original image.

[0717] like Fig.52 In step S312, the file control unit 43 calculates the size of the free box to be set in the proxy file, as in step S212 in the above, and the process proceeds to step S313.

[0718] In step S313, the file control unit 43 generates a general collection file as a proxy file, which includes a free box having the size calculated in step S312 and in which the proxy thumbnail image generated in step S311 (i.e., for example, a thumbnail image of the main image stored in the proxy target file) is stored as a proxy item in the mdat box, and the proxy file generation processing is completed.

[0719] Fig.69 is a flowchart for explaining an example of adding processing performed in the case of generating a thumbnail image to be stored in a proxy file of the first mode when generating a proxy target file.

[0720] In the case where a HEIF file is generated and an adding process execution condition for executing an adding process using the HEIF file as a proxy target file is satisfied, the file control unit 43 executes the following adding process.

[0721] As the adding processing execution condition, for example, the fact that a proxy file exists in the directory or medium storing the proxy target file and the number of thumbnail images stored in the proxy file has not reached the upper limit may be employed.

[0722] The adding process is performed on an already existing proxy file (existing proxy file) which is a target of the adding process.

[0723] like Fig.68 As in step S311 in , in step S321, the file control unit 43 generates a proxy thumbnail image as an additional thumbnail image, and the process proceeds to step S322.

[0724] In steps S322 to S326, respectively, Fig.53 Similar processing is performed to steps S222 to S226, and the adding processing ends.

[0725] Fig.70 It is a view for explaining the proxy file generation processing and the adding processing performed in the case of generating a thumbnail image to be stored in the proxy file of the third mode when generating a proxy target file.

[0726] Note that the description of the proxy file generation processing and addition processing performed when a thumbnail image to be stored in the proxy file of the second mode is generated when a proxy target file is generated is omitted because, except that the designated information is not written to the proxy target file and the information that does not need to be written to the proxy target file is used as the designated information, the processing is similar to the processing for the proxy file of the third mode.

[0727] Fig.70 An example of the structure of the proxy file of the third mode is shown.

[0728] Fig.70 A shows an example of the structure of a proxy file generated by the proxy file generation process.

[0729] In the proxy file generation process performed in the case of generating a thumbnail image to be stored in the proxy file of the third mode when generating a proxy target file, the proxy file generation process ( Fig.67 ), the file control unit 43 generates a proxy thumbnail image to be stored in the proxy file of the third mode by using the original image used to generate the current thumbnail image stored in the proxy target file.

[0730] The file control unit 43 generates, as a proxy file of the third mode, an association type set file having, for example, a free box and an mdat box storing, as a proxy item, a proxy thumbnail image which is a thumbnail image of a main image stored in the proxy target file.

[0731] exist Fig.70 In A, a proxy file storing (as main images) two proxy thumbnail images A and B has been generated by proxy file generation processing. In addition, a free box is set in the meta box. Metadata of the proxy thumbnail images A and B are stored in the meta box in the proxy file generated by the proxy file generation processing.

[0732] Moreover, in Fig.70In the example, a first association type set file is used as a proxy file, and the proxy file has an association information storage box in a meta box. The association information storage box stores association information associating item ID=1 of proxy thumbnail image A with UUID#A as designated information of a main image corresponding to proxy thumbnail image A, and associating item ID=2 of proxy thumbnail image B with UUID#B as designated information of a main image corresponding to proxy thumbnail image B.

[0733] Fig.70 B shows an example of the structure of a proxy file to which a proxy thumbnail image of another main image is added by adding processing.

[0734] For example, in an adding process performed in the case where a proxy thumbnail image to be stored in a proxy file in a third mode is generated when a proxy target file is generated, when a HEIF file generated after the proxy file is generated is used as a new proxy target file, a proxy thumbnail image of the main image stored in the new proxy target file is added to an existing proxy file that has been generated.

[0735] That is, the file control unit 43 generates a proxy thumbnail image to be added to the proxy file of the third mode using an original image used to generate a current thumbnail image stored in a new proxy target file when generating the new proxy target file.

[0736] exist Fig.70 In B, Fig.70 After the proxy thumbnail images A and B are already stored in the mdat box of the existing proxy file in A, two proxy thumbnail images C and D (as main images) are added through the adding process.

[0737] When the proxy thumbnail images C and D are added, the file control unit 43 rewrites the meta box to a state in which the metadata of the proxy thumbnail images C and D are added.

[0738] Moreover, the file control unit 43 rewrites the meta box (the associated information storage box in it) to a state in which associated information is added that associates item ID=3 of proxy thumbnail image C with UUID#C, which is the designated information of the main image corresponding to proxy thumbnail image C, and associated information that associates item ID=4 of proxy thumbnail image D with UUID#D, which is the designated information of the main image corresponding to proxy thumbnail image D.

[0739] Then, the file control unit 43 reduces the capacity of the free box by the amount by which the capacity of the meta box is increased by rewriting the meta box. In a similar manner, a proxy thumbnail image of the main image stored in a still new proxy target file is added to the existing proxy file.

[0740] Fig.71 is a flowchart for explaining an example of a proxy file generation process performed in the case of generating a thumbnail image to be stored in a proxy file of the third mode when generating a proxy target file.

[0741] When generating HEIF files, using HEIF files as proxy target files and meeting the reference Fig.68 Under the above-mentioned proxy file generation processing execution conditions, the file control unit 43 executes the following proxy file generation processing.

[0742] like Fig.68 As in step S311 in , in step S341, the file control unit 43 generates a proxy thumbnail image, and the processing proceeds to step S342.

[0743] In step S342 , the file control unit 43 acquires, for example, uuid by generating uuid as the designation information of the main image corresponding to the proxy thumbnail image generated in step S341 , and the process proceeds to step S343 .

[0744] like Fig.55 In step S343, the file control unit 43 calculates the size of the free box to be set in the proxy file, as in step S243 in the above, and the process proceeds to step S344.

[0745] In step S344, the file control unit 43 generates, for example, a first association type collection file as a proxy file, the first association type collection file including a free box having the size calculated in step S343 and in which the proxy thumbnail image generated in step S341 (i.e., the thumbnail image of the main image stored in the proxy target file) is stored as a proxy item in the mdat box, and the proxy file generation processing ends.

[0746] In the proxy file generation in step S344, a first association type set file is generated in which the proxy thumbnail image is stored in association with the designated information of the main image corresponding to the proxy thumbnail image (eg, the uuid acquired in step S342).

[0747] That is, a first association type set file is generated as a proxy file, in which the proxy thumbnail image is stored in the mdat box, and association information associating the item ID of the proxy thumbnail image with the uuid of the main image corresponding to the proxy thumbnail image is stored in the association information storage box.

[0748] Fig.72 is a flowchart for explaining an example of adding processing performed in the case of generating a thumbnail image to be stored in a proxy file of the third mode when generating a proxy target file.

[0749] When generating HEIF files, using HEIF files as proxy target files and meeting the reference Fig.69 In the case of the described adding process execution conditions, the file control unit 43 executes the following adding process.

[0750] The adding process is performed for the already existing proxy file (existing proxy file) as the target of the adding process. It is assumed here that the existing proxy file is a first association type collection file, such as Fig.70 and 71 The situation in.

[0751] like Fig.69 As in step S321 in , in step S351, the file control unit 43 generates a proxy thumbnail image as an additional thumbnail image, and the process proceeds to step S352.

[0752] In step S352 , the file control unit 43 acquires, for example, uuid which is the designation information of the main image corresponding to the additional thumbnail image generated in step S351 , and the process proceeds to step S353 .

[0753] In step S353, the file control unit 43 adds the additional thumbnail image as a proxy item to the mdat box in the proxy file that is the target of the adding process, and the process proceeds to step S354.

[0754] In step S354, the file control unit 43 acquires the meta box by reading the meta box from the proxy file that is the target of the adding process, and the process proceeds to step S355.

[0755] In step S355, the file control unit 43 obtains the remaining capacity of the free box in the proxy file that is the target of the adding processing, and confirms that the remaining capacity is sufficient to add the metadata of the additional thumbnail image and to add the association information that associates the item ID of the additional thumbnail image with the uuid that is the designated information of the main image corresponding to the additional thumbnail image, and the processing proceeds to step S356.

[0756] Note that if it is determined in step S355 that the remaining capacity of the free box is insufficient for the addition, then the same as in the reference may be performed. Fig.53 The processing is similar to the processing described in step S224 in .

[0757] In step S356 , the file control unit 43 regenerates a new meta box from the meta box acquired in step S354 .

[0758] A new meta box is generated by adding the metadata of the attached thumbnail image as a proxy item added to the mdat box in the proxy file which is the target of the adding processing to the meta box acquired in step S354, and by adding associated information which associates the item ID of the attached thumbnail image with the uuid which is the designated information of the main image corresponding to the attached thumbnail image to the associated information storage box in the meta box.

[0759] The file control unit 43 rewrites the meta box in the proxy file that is the target of the adding process with the new meta box, and the process proceeds from step S356 to step S357.

[0760] In step S357, the file control unit 43 generates a new free box whose size is reduced by the amount of increase in the size of the new meta box relative to the meta box before rewriting. Furthermore, the file control unit 43 overwrites the free box in the proxy file that is the target of the adding process with the new free box, and the adding process ends.

[0761] <Proxy file generation and addition processing in which an already existing HEIF file is used as a proxy target file>

[0762] Fig.73 is a flowchart for explaining an example of a proxy file generation process in which an already existing HEIF file is used as a proxy target file.

[0763] For example, when a user or application requests the generation of a proxy file and satisfies the reference Fig.68 Under the above-mentioned proxy file generation processing execution condition, the file control unit 43 executes the following proxy file generation processing: It is assumed that, in the proxy file generation processing, for example, a proxy file of the second or third mode as a first association type set file is generated.

[0764] In step S371, the file control unit 43 selects (all or some) HEIF files that already exist in a predetermined directory or medium as proxy target files. Furthermore, the file control unit 43 obtains a thumbnail image of the main image in the proxy target file as a proxy thumbnail image, and the process proceeds from step S371 to step S372.

[0765] For example, the file control unit 43 obtains a copy of the current thumbnail image of the main image stored in the proxy target file as the proxy thumbnail image. In addition, the file control unit 43 generates the proxy thumbnail image by, for example, thinning the main image stored in the proxy target file or the current thumbnail image, converting the signal format of the main image or the current thumbnail image, re-encoding the main image or the current thumbnail image, etc.

[0766] In step S372 , the file control unit 43 acquires, for example, uuid by generating uuid as the designation information of the main image corresponding to the proxy thumbnail image generated in step S371 , and the process proceeds to step S373 .

[0767] like Fig.55 In step S373, the file control unit 43 calculates the size of the free box to be set in the proxy file, as in step S243 in , and the process proceeds to step S374.

[0768] In step S374, the file control unit 43 generates, for example, a first association type collection file as a proxy file, which includes a free box having the size calculated in step S373 and in which the proxy thumbnail image generated in step S371 is stored as a proxy item in the mdat box, and the proxy file generation process ends.

[0769] In the proxy file generation in step S374, a first association type set file is generated in which the proxy thumbnail image is stored in association with the uuid acquired in step S372 as the designation information of the main image corresponding to the proxy thumbnail image.

[0770] That is, a first association type set file is generated as a proxy file, in which the proxy thumbnail image is stored in the mdat box, and association information associating the item ID of the proxy thumbnail image with the uuid of the main image corresponding to the proxy thumbnail image is stored in the association information storage box.

[0771] Fig.74 : is a flowchart for explaining an example of adding processing using an already existing HEIF file as a proxy target file.

[0772] For example, when a user or application requests the generation of a proxy file and satisfies the reference Fig.69 In the case of the described adding process execution condition, the file control unit 43 performs the following adding process. The adding process is performed for the already existing proxy file (existing proxy file) as the target of the adding process. It is assumed here that the existing proxy file is a proxy file of the second or third mode, which is a first association type collection file.

[0773] In step S381, for example, using (all or some) HEIF files already existing in a predetermined directory or medium as proxy target files, the file control unit 43 obtains a proxy thumbnail image of the main image stored in the proxy target file as an additional thumbnail image, and the processing proceeds to step S382.

[0774] For example, the file control unit 43 obtains a copy of the current thumbnail image of the main image stored in the proxy target file as the additional thumbnail image. In addition, the file control unit 43 generates the additional thumbnail image by, for example, thinning the main image stored in the proxy target file or the current thumbnail image, converting the signal format of the main image or the current thumbnail image, re-encoding the main image or the current thumbnail image, etc.

[0775] In step S382 , the file control unit 43 acquires, for example, uuid which is the designation information of the main image corresponding to the additional thumbnail image generated in step S381 , and the process proceeds to step S383 .

[0776] In step S383, the file control unit 43 adds the additional thumbnail image as a proxy item to the mdat box in the existing proxy file that is the target of the adding process, and the process proceeds to step S384.

[0777] In step S384, the file control unit 43 acquires the meta box by reading the meta box from the proxy file that is the target of the adding process, and the process proceeds to step S385.

[0778] In step S385, the file control unit 43 obtains the remaining capacity of the free box in the proxy file that is the target of the adding processing, and confirms that the remaining capacity is sufficient to add the metadata of the additional thumbnail image and to add the association information that associates the item ID of the additional thumbnail image with the uuid that is the designated information of the main image corresponding to the additional thumbnail image, and the processing proceeds to step S386.

[0779] Note that if it is determined in step S385 that the remaining capacity of the free box is insufficient for the addition, then the same process as reference may be performed. Fig.53 The processing is similar to the processing described in step S224 in .

[0780] In step S386, the file control unit 43 regenerates a new meta box from the meta box acquired in step S384. The new meta box is generated by adding the metadata of the additional thumbnail image to the meta box acquired in step S384 as a proxy item added to the mdat box in the existing proxy file that is the target of the adding process, and by adding association information that associates the item ID of the additional thumbnail image with the uuid that is the designation information of the main image corresponding to the additional thumbnail image to the association information storage box in the meta box.

[0781] The file control unit 43 rewrites the meta box in the proxy file that is the target of the adding process with the new meta box, and the process proceeds from step S386 to step S387.

[0782] In step S387, the file control unit 43 generates a new free box whose size is reduced by the amount of increase in the size of the new meta box relative to the meta box before rewriting. Furthermore, the file control unit 43 overwrites the free box in the proxy file that is the target of the adding process with the new free box, and the adding process ends.

[0783] <Description of Computer to Which the Present Technology is Applied>

[0784] Next, the above-mentioned file control unit 43 and another signal processing unit 13 ( Figure 1 ) can be executed by hardware or software. In the case where the series of processes are executed by software, a program forming the software is installed in a computer or the like.

[0785] Fig.75 : is a block diagram illustrating a configuration example of an embodiment of a computer to be installed with a program for executing the series of processes described above.

[0786] The program can be recorded in advance on the hard disk 905 or the ROM 903 which is a recording medium built in the computer.

[0787] Alternatively, the program may be stored (recorded) in a removable recording medium 911 driven by the drive 909. Such a removable recording medium 911 may be provided as so-called packaged software. Here, examples of the removable recording medium 911 include, for example, a floppy disk, a compact disk read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a magnetic disk, a semiconductor memory, and the like.

[0788] Note that, as described above, the program can be installed in the computer from the removable recording medium 911, or can also be downloaded to the computer via a communication network or a broadcasting network and installed in the built-in hard disk 905. That is, for example, the program can be wirelessly transmitted from a download site to the computer via an artificial satellite used for digital satellite broadcasting, or can be transmitted to the computer LAN by a cable via a network such as a local area network (LAN) and the Internet.

[0789] The computer incorporates a central processing unit (CPU) 902 , and an input / output interface 910 is connected to the CPU 902 via a bus 901 .

[0790] When a user inputs a command by operating the input unit 907 or the like via the input / output interface 910, in response thereto, the CPU 902 executes a program stored in a read only memory (ROM) 903. Alternatively, the CPU 902 loads a program stored in a hard disk 905 into a random access memory (RAM) 904 and executes the program.

[0791] Therefore, the CPU 902 performs processing according to the above-mentioned flowchart or the processing performed by the configuration of the above-mentioned block diagram. Then, when necessary, the CPU 902 outputs the processing result from the output unit 906 via the input / output interface 910 or transmits it from the communication unit 908, and further records it on the hard disk 905 or the like.

[0792] Note that the input unit 907 includes a keyboard, a mouse, a microphone, etc. Furthermore, the output unit 906 includes a liquid crystal display (LCD), a speaker, and the like.

[0793] Here, in this specification, the processing performed by the computer according to the program is not necessarily performed in chronological order according to the order described in the flowchart. That is, the processing performed by the computer according to the program includes processing performed in parallel or individually (for example, parallel processing or processing by target).

[0794] In addition, the program may be processed by one computer (processor), or may be distributed and processed by a plurality of computers. Also, the program may be transmitted to a remote computer for execution.

[0795] Moreover, in this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network and a single device with multiple modules housed in one housing are both systems.

[0796] Note that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications may be made without departing from the scope of the present technology.

[0797] For example, the present technology may have a cloud computing configuration in which one function is shared and collaboratively processed by multiple devices via a network.

[0798] In addition, each step described in the above flowchart may be performed by one device, and may also be shared and performed by a plurality of devices.

[0799] Furthermore, in the case where one step includes a plurality of processes, the plurality of processes included in one step may be executed by one device, and may also be shared and executed by a plurality of devices.

[0800] Furthermore, the effects described in this specification are merely examples and not limitations, and other effects may exist.

[0801] Note that the present technology can have the following configurations.

[0802] <1>

[0803] A file processing device, comprising:

[0804] The file control unit is configured to generate a proxy file, wherein the proxy file stores a related image related to an image in a High Efficiency Image File Format (HEIF) file, wherein the proxy file is another HEIF file.

[0805] <2>

[0806] according to <1> The file processing device, wherein:

[0807] The file control unit generates a collection file as the proxy file, the collection file is a HEIF file in an image item format and the related images are stored as items in the collection file.

[0808] <3>

[0809] according to <2> The file processing device, wherein:

[0810] The file control unit generates the proxy file, the proxy file storing association information that associates a related image with designation information that designates an image corresponding to the related image.

[0811] <4>

[0812] according to <3> The file processing device, wherein:

[0813] The file control unit generates the proxy file storing association information in which the item ID specifying the related image is associated with the specifying information.

[0814] <5>

[0815] according to <4> The file processing device, wherein:

[0816] The file control unit generates the proxy file, in which the associated information is stored in a meta box or an mdat box.

[0817] <6>

[0818] according to <3> The file processing device, wherein:

[0819] The file control unit generates the proxy file, in which:

[0820] Specifies that information is stored as entries in mdat boxes in the proxy file, and

[0821] The association information is stored in the meta box, and the association information associates the item ID specifying the related image with the item ID specifying the specified information stored in the mdat box.

[0822] <7>

[0823] according to <1> to <6> The file processing device according to any one of claims , wherein:

[0824] The related image is different from the image in image quality or data amount.

[0825] <8>

[0826] according to <1> to <6> The file processing device according to any one of claims , wherein:

[0827] The related image is different from the image in terms of the number of pixels or size and is the same as the image in terms of content, different from the image in terms of signal format and is the same as the image in terms of content, or different from the image in terms of grayscale and is the same as the image in terms of content.

[0828] <9>

[0829] according to <1> to <6> The file processing device according to any one of claims , wherein:

[0830] HEIF files store an image and a thumbnail image of that image, and

[0831] The related image is: a thumbnail image stored in a HEIF file, an image encoded with a compression rate different from that used for a thumbnail image, or an image encoded with a codec different from that used for a thumbnail image.

[0832] <10>

[0833] A file processing method, comprising:

[0834] A proxy file is generated, the proxy file storing related images related to an image in a High Efficiency Image File Format (HEIF) file, the proxy file being another HEIF file.

[0835] <11>

[0836] A program for causing a computer to function as:

[0837] The file control unit is configured to generate a proxy file, wherein the proxy file stores a related image related to an image in a High Efficiency Image File Format (HEIF) file, wherein the proxy file is another HEIF file.

[0838] <12>

[0839] A file processing device, comprising:

[0840] A file control unit is configured to detect an image in a High Efficiency Image File Format (HEIF) file, the image corresponding to a selected related image selected from related images related to the image and stored in a proxy file as another HEIF file.

[0841] <13>

[0842] according to <12> The file processing device, wherein:

[0843] The proxy file is a HEIF file in image item format, which stores related images as items.

[0844] <14>

[0845] according to <13> The file processing device, wherein:

[0846] The proxy file stores association information that associates a related image with specified information that specifies an image corresponding to the related image, and

[0847] The file control unit detects an image specified by the specifying information associated with the selected related image as an image corresponding to the selected related image.

[0848] <15>

[0849] according to <14> The file processing device, wherein:

[0850] The proxy file stores association information in which an item ID specifying a related image is associated with specified information, and

[0851] The file control unit detects the image specified by the specifying information associated with the item ID of the selected related image as the image corresponding to the selected related image.

[0852] <16>

[0853] according to <12> to <15> The file processing device according to any one of claims , wherein:

[0854] The related image is different from the image in image quality or data amount.

[0855] <17>

[0856] according to <12> to <15> The file processing device according to any one of claims , wherein:

[0857] The related image is different from the image in terms of the number of pixels or size and is the same as the image in terms of content, different from the image in terms of signal format and is the same as the image in terms of content, or different from the image in terms of grayscale and is the same as the image in terms of content.

[0858] <18>

[0859] according to <12> to <15> The file processing device according to any one of claims , wherein:

[0860] HEIF files store an image and a thumbnail image of that image, and

[0861] The related image is: a thumbnail image stored in a HEIF file, an image encoded with a compression rate different from that used for a thumbnail image, or an image encoded with a codec different from that used for a thumbnail image.

[0862] <19>

[0863] A file processing method, comprising:

[0864] An image in a High Efficiency Image File Format (HEIF) file is detected, the image corresponding to a selected related image selected from related images related to the image and stored in the proxy file which is another HEIF file.

[0865] <20>

[0866] A program for causing a computer to function as:

[0867] A file control unit is configured to detect an image in a High Efficiency Image File Format (HEIF) file, the image corresponding to a selected related image selected from related images related to the image and stored in a proxy file as another HEIF file.

[0868] Reference numerals list

[0869] 10 Digital Camera

[0870] 11 Optical System

[0871] 13 Signal Processing Unit

[0872] 14 Medium

[0873] 15, 16 Interface

[0874] 17 buttons / keys

[0875] 18 Touch Panel

[0876] 19 LCD Panel

[0877] 20 Viewfinder

[0878] 21 Interface

[0879] 41 Optical system / image sensor control unit

[0880] 42 Encoding control unit

[0881] 43 File Control Unit

[0882] 44 Media Control Unit

[0883] 45 Operation control unit

[0884] 46 Display control unit

[0885] 47 UI control unit

[0886] 901 Bus

[0887] 902 CPU

[0888] 903 ROM

[0889] 904 RAM

[0890] 905 Hard Drive

[0891] 906 Output Unit

[0892] 907 Input Unit

[0893] 908 Communication Unit

[0894] 909 Drive

[0895] 910 Input / Output Interface

[0896] 911 Removable recording media

Claims

1. A file processing device, comprising: A file control unit is configured to generate a proxy file, the proxy file storing a related image related to a main image in a High Efficiency Image File Format (HEIF) file, the proxy file being another HEIF file, wherein the proxy file stores association information and designation information, the association information associates the related image with the designation information, and the designation information designates a primary image corresponding to the related image, Wherein, the index display using the proxy file only requires opening / closing the proxy file without opening / closing the HEIF file storing the main image. wherein the HEIF file storing the main image stores the main image and a thumbnail image of the main image, and The related images are: thumbnail images of main images stored in the proxy file, or images encoded with a compression rate different from that used for thumbnail images, or images encoded with a codec different from that used for thumbnail images.

2. The file processing device according to claim 1, wherein: The file control unit generates a collection file as the proxy file, the collection file is a HEIF file in an image item format and the related images are stored as items in the collection file.

3. The file processing device according to claim 1, wherein: The file control unit generates the proxy file storing association information in which the item ID specifying the related image is associated with the specifying information.

4. The file processing device according to claim 3, wherein: The file control unit generates the proxy file, in which the associated information is stored in a meta box or an mdat box.

5. The file processing device according to claim 1, wherein: The file control unit generates the proxy file, in which: Specifies that information is stored as entries in mdat boxes in the proxy file, and The association information is stored in the meta box, and the association information associates the item ID specifying the related image with the item ID specifying the specified information stored in the mdat box.

6. The file processing device according to claim 1, wherein: The related image is different from the main image in image quality or data amount.

7. The file processing device according to claim 1, wherein: The related image is different from the main image in terms of the number of pixels or size and is the same as the main image in terms of content, or is different from the main image in terms of signal format and is the same as the main image in terms of content, or is different from the main image in terms of grayscale and is the same as the main image in terms of content.

8. A file processing method, comprising: generating a proxy file storing related images related to a main image in a High Efficiency Image File Format (HEIF) file, the proxy file being another HEIF file, wherein the proxy file stores association information and designation information, the association information associates the related image with the designation information, and the designation information designates a primary image corresponding to the related image, Wherein, the index display using the proxy file only requires opening / closing the proxy file without opening / closing the HEIF file storing the main image. wherein the HEIF file storing the main image stores the main image and a thumbnail image of the main image, and The related images are: thumbnail images of main images stored in the proxy file, or images encoded with a compression rate different from that used for thumbnail images, or images encoded with a codec different from that used for thumbnail images.

9. A medium storing a program for causing a computer to function as a file control unit, The file control unit is configured to generate a proxy file storing a related image related to a main image in a High Efficiency Image File Format (HEIF) file, the proxy file being another HEIF file, in, The proxy file stores association information and designation information, the association information associating a related image with the designation information, and the designation information designating a main image corresponding to the related image, Wherein, the index display using the proxy file only requires opening / closing the proxy file without opening / closing the HEIF file storing the main image. wherein the HEIF file storing the main image stores the main image and a thumbnail image of the main image, and The related images are: thumbnail images of main images stored in the proxy file, or images encoded with a compression rate different from that used for thumbnail images, or images encoded with a codec different from that used for thumbnail images.

10. A file processing device, comprising: A file control unit is configured to detect a main image in a High Efficiency Image File Format (HEIF) file, the image corresponding to a related image selected from related images related to the image and stored in a proxy file as another HEIF file, wherein the proxy file stores association information and designation information, the association information associates the related image with the designation information, and the designation information designates a primary image corresponding to the related image, Wherein, the index display using the proxy file only requires opening / closing the proxy file without opening / closing the HEIF file storing the main image. wherein the HEIF file storing the main image stores the main image and a thumbnail image of the main image, and The related images are: thumbnail images of main images stored in the proxy file, or images encoded with a compression rate different from that used for thumbnail images, or images encoded with a codec different from that used for thumbnail images.

11. The file processing device according to claim 10, wherein: The proxy file is a HEIF file in image item format, which stores related images as items.

12. The file processing device according to claim 11, wherein: The proxy file stores association information that associates a related image with specified information that specifies an image corresponding to the related image, and The file control unit detects an image specified by the specifying information associated with the selected related image as an image corresponding to the selected related image.

13. The file processing device according to claim 12, wherein: The proxy file stores association information in which an item ID specifying a related image is associated with specified information, and The file control unit detects the image specified by the specifying information associated with the item ID of the selected related image as the image corresponding to the selected related image.

14. The file processing device according to claim 10, wherein: The related image is different from the main image in image quality or data amount.

15. The file processing device according to claim 10, wherein: The related image is different from the main image in terms of the number of pixels or size and is the same as the main image in terms of content, or is different from the main image in terms of signal format and is the same as the main image in terms of content, or is different from the main image in terms of grayscale and is the same as the main image in terms of content.

16. A file processing method, comprising: detecting a primary image in a High Efficiency Image File Format (HEIF) file, the image corresponding to a related image selected from related images related to the image and stored in a proxy file that is another HEIF file, wherein the proxy file stores association information and designation information, the association information associates the related image with the designation information, and the designation information designates a primary image corresponding to the related image, and Wherein, the index display using the proxy file only requires opening / closing the proxy file without opening / closing the HEIF file storing the main image. wherein the HEIF file storing the main image stores the main image and a thumbnail image of the main image, and The related images are: thumbnail images of main images stored in the proxy file, or images encoded with a compression rate different from that used for thumbnail images, or images encoded with a codec different from that used for thumbnail images.

17. A medium storing a program for causing a computer to function as a file control unit, The file control unit is configured to detect a main image in a High Efficiency Image File Format (HEIF) file, the image corresponding to a related image selected from related images related to the image and stored in a proxy file as another HEIF file, in, The proxy file stores association information and designation information, the association information associating a related image with the designation information, and the designation information designating a main image corresponding to the related image, and Wherein, the index display using the proxy file only requires opening / closing the proxy file without opening / closing the HEIF file storing the main image. wherein the HEIF file storing the main image stores the main image and a thumbnail image of the main image, and The related images are: thumbnail images of main images stored in the proxy file, or images encoded with a compression rate different from that used for thumbnail images, or images encoded with a codec different from that used for thumbnail images.