Data processing device, data processing method, and program

Abstract

The present technology relates to a data processing device, a data processing method, and a program that make it possible to easily perform synchronized generation of a plurality of files in which file streams of different data formats are stored. File data, which is obtained by encoding images and stored in each of a plurality of files, is generated from identical images. Between first file data stored in one file among a plurality of files and second file data stored in another file among the plurality of files, at least one of the codec, the chroma format, and the bit depth is different. The present technology can be applied to the generation of a plurality of files having identical image content.

Description

[Technical field]

[0001] The present technology relates to a data processing device, a data processing method, and a program, and in particular to a data processing device, a data processing method, and a program that enable, for example, easy simultaneous generation of multiple files in which file streams with different data formats are stored. [Background technology]

[0002] High Efficiency Image File Format (HEIF) is a file format for efficiently storing images (see Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] ISO / IEC 23008-12:2017, Information technology -- High efficiency coding and media delivery in heterogeneous environments -- Part 12: Image File Format Summary of the Invention [Problem to be solved by the invention]

[0004] A stream of images stored in a file is called a file stream.

[0005] For HEIF files and other files that store images, it would be useful to be able to simultaneously generate multiple files for the same image content, each containing file streams with different data formats, such as image size.

[0006] The present technology has been made in view of such circumstances, and makes it possible to easily simultaneously generate a plurality of files in which file streams with different data formats are stored. [Means for solving the problem]

[0007] A first data processing device or a first program of the present technology is a data processing device that includes an encoding control unit that generates file data encoding an image from the same image, which is stored in each of a plurality of files, and in which the first file data stored in one of the plurality of files and the second file data stored in the other file are data differing in at least one of codec, chroma format, and bit depth, or a program for causing a computer to function as such a data processing device.

[0008] A first data processing method of the present technology includes generating file data encoding an image from the same image, the file data being stored in each of a plurality of files, wherein the first file data stored in one of the plurality of files and the second file data stored in the other file are data differing in at least one of codec, chroma format, and bit depth.

[0009] In the first data processing device, the first data processing method, and the first program of the present technology, file data that encodes an image and is stored in a plurality of files is generated from the same image. The first file data stored in one of the plurality of files and the second file data stored in the other file are data that differ in at least one of codec, chroma format, and bit depth.

[0010] A second data processing device or a second program of the present technology is a data processing device that includes an encoding control unit that generates file data encoding an image from the same image, which is stored in each of a plurality of files including at least a HEIF (High Efficiency Image File Format) file, and in which first file data stored in the HEIF file of the plurality of files and second file data stored in another file other than the HEIF file are data that differ in at least one of codec, chroma format, bit depth, and image size, or a program for causing a computer to function as such a data processing device.

[0011] A second data processing method of the present technology includes generating, from the same image, file data encoding the image, which is stored in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file, wherein first file data stored in the HEIF file among the plurality of files and second file data stored in another file other than the HEIF file are data differing in at least one of codec, chroma format, bit depth, and image size.

[0012] In a second data processing device, a second data processing method, and a second program of the present technology, file data encoding an image is generated from the same image and stored in a plurality of files including at least a High Efficiency Image File Format (HEIF) file. The first file data stored in the HEIF file among the plurality of files and the second file data stored in another file other than the HEIF file are data that differ in at least one of codec, chroma format, bit depth, and image size.

[0013] The data processing device may be an independent device or an internal block constituting a single device.

[0014] The program can also be provided by being recorded on a recording medium or transmitted via a transmission medium. [Brief description of the drawings]

[0015] [Figure 1] 1 is a block diagram showing an example of the configuration of an embodiment of a digital camera to which the present technology is applied. [Diagram 2] FIG. 1 is a diagram showing an example of a JPEG file format that complies with JPEG (Joint Photographic Experts Group). [Diagram 3] FIG. 1 is a diagram illustrating an example of an ISO base media file format. [Figure 4] FIG. 2 is a diagram showing an example of a format of a HEIF file that complies with HEIF. [Diagram 5] FIG. 1 is a diagram showing an example of the format of a HEIF file in image item format. [Figure 6] FIG. 13 is a diagram showing an example of an iprp box. [Figure 7] FIG. 1 is a diagram showing an example of the format of a HEIF file in image sequence format. [Figure 8] FIG. 1 is a diagram showing an example of a trak box. [Figure 9] FIG. 13 is a diagram showing an example of a collection file in which main images and thumbnail images are stored. [Figure 10] FIG. 13 is a diagram showing an example of a sequence file in which a track of a main image and a track of thumbnail images of the main image are stored. [Figure 11] FIG. 1 is a diagram for explaining an overview of the generation of a file stream to be stored in a file in a digital camera 10. [Figure 12] FIG. 2 is a diagram illustrating a first example of the generation of multiple files with the same image content in the digital camera 10. [Figure 13] FIG. 13 is a diagram illustrating a second example of the generation of multiple files with the same image content in the digital camera 10. [Figure 14] 10 is a flowchart illustrating the process of generating multiple files of the same image content in digital camera 10. [Figure 15] 2 is a block diagram showing a first exemplary configuration of an encoding control unit 42. FIG. [Figure 16] 13 is a diagram illustrating an example of generation of an HEIF file and a JPEG file of the same image content when the encoding control unit 42 is configured as a single semiconductor chip 110. [Figure 17] 13 is a diagram illustrating another example of generation of an HEIF file and a JPEG file of the same image content when the encoding control unit 42 is configured as a single semiconductor chip 110. FIG. [Figure 18] 13 is a diagram illustrating an example of generating an HEIF file and an ARW file of the same image content when the encoding control unit 42 is configured as a single semiconductor chip 110. [Figure 19] 13 is a diagram illustrating an example of generating two HEIF files with the same image content when the encoding control unit 42 is configured as a single semiconductor chip 110. FIG. [Figure 20] 13 is a diagram illustrating an example of generating two JPEG files having the same image content when the encoding control unit 42 is configured as a single semiconductor chip 110. FIG. [Figure 21] 13 is a block diagram showing a second exemplary configuration of the encoding control unit 42. FIG. [Figure 22] 13 is a diagram illustrating an example of generation of an HEIF file and a JPEG file of the same image content when the encoding control unit 42 is configured with two semiconductor chips 210 and 220. [Figure 23] 13 is a diagram illustrating another example of generation of an HEIF file and a JPEG file of the same image content when the encoding control unit 42 is configured from two semiconductor chips 210 and 220. [Figure 24] 1 is a block diagram showing an example of the configuration of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] <One embodiment of a digital camera to which the present technology is applied>

[0017] FIG. 1 is a block diagram showing an example of the configuration of an embodiment of a digital camera to which a data processing device of the present technology is applied.

[0018] Digital camera 10 has an optical system 11, an image sensor 12, a signal processing unit 13, a media 14, I / Fs (Interfaces) 15 and 16, buttons / keys 17, a touch panel 18, an LCD panel 19, a viewfinder 20, and an I / F 21, etc.

[0019] The optical system 11 collects light from a subject onto the image sensor 12 .

[0020] The image sensor 12 receives light from the optical system 11 and performs photoelectric conversion to capture an image, thereby generating image data as an electrical signal, and supplies the data to a signal processing unit 13 .

[0021] The signal processing unit 13 has an optical system / image sensor control unit 41, an encoding control unit 42, a file control unit 43, a media control unit 44, an operation control unit 45, a display control unit 46, and a UI control unit 47.

[0022] The optical system / image sensor control unit 41 controls the optical system 11 and the image sensor 12, and supplies the image (data) obtained by imaging performed under that control to the encoding control unit .

[0023] The encoding control unit 42 supplies the image from the optical system / image sensor control unit 41 to the display control unit 46, and also encodes the image as necessary and supplies it to the file control unit 43. The encoding control unit 42 also decodes the image supplied from the file control unit 43 as necessary and supplies it to the display control unit 46.

[0024] The file control unit 43 generates a file storing the image supplied from the encoding control unit 42, and supplies the file to the media control unit 44. The file control unit 43 also plays the file supplied from the media control unit 44, i.e., reads data such as images stored in the file. For example, an image read from a file is supplied from the file control unit 43 to the encoding control unit 42.

[0025] The media control unit 44 controls the exchange of files between the medium 14 and the I / Fs 15 and 16. For example, the media control unit 44 causes a file from the file control unit 43 to be recorded on the medium 14 or transmitted from the I / Fs 15 and 16. The media control unit 44 also reads a file from the medium 14 or causes the I / Fs 15 and 16 to receive the file and supply it to the file control unit 43.

[0026] In response to a user's operation of the buttons / keys 17 or the touch panel 18, the operation control unit 45 supplies an operation signal corresponding to the operation to a necessary block.

[0027] The display control unit 46 performs display control and the like to supply the images and the like supplied from the encoding control unit 42 to the liquid crystal panel 19, the viewfinder 20, and the I / F 21 for display.

[0028] The UI control unit 47 is in charge of UI (User Interface) control.

[0029] The medium 14 is, for example, a storage medium such as an SD card. The I / F 15 is, for example, an I / F of a LAN (Local Area Network) such as WiFi (registered trademark) or Ethernet (registered trademark). The I / F 16 is, for example, an I / F of a USB (Universal Serial Bus). The buttons / keys 17 and the touch panel 18 are operated by a user when inputting commands or other information to the digital camera 10. The touch panel 18 can be configured integrally with a liquid crystal panel 19. The liquid crystal panel 19 and the viewfinder 20 display images and the like supplied from the display control unit 46. The I / F 21 is an I / F that transmits at least images, such as an HDMI (registered trademark) (High-Definition Multimedia Interface) or a DP (Display Port).

[0030] In the digital camera 10 configured as above, the optical system / image sensor control unit 41 generates, for example, a YUV image having the same resolution (number of pixels) (size) as the RAW image from an image of RAW data obtained by imaging with the image sensor 12 (hereinafter also referred to as a RAW image), and supplies this together with the RAW image to the encoding control unit 42. The encoding control unit 42 generates a main image (master image) of the HEIF file from the YUV image from the optical system / image sensor control unit 41. For example, the YUV image from the optical system / image sensor control unit 41 can be used as the main image of the HEIF file as is.

[0031] The encoding control unit 42 generates, from the YUV main image, a YUV image (hereinafter also referred to as a screen nail image) having a lower resolution than the main image as a first other image based on the main image for display on the liquid crystal panel 19 or an external display, and generates, for example, a YUV image (hereinafter also referred to as a thumbnail image) having a lower resolution than the screen nail image as a second other image based on the main image for index display (list display). The encoding control unit 42 supplies, for example, the screen nail image to the liquid crystal panel 19 via the display control unit 46, and displays it as a so-called through image. As the thumbnail image, for example, an image with a long side of 320 pixels or less can be used. The ratio of the size (number of pixels) between the main image and the screen nail image as the first other image based on the main image, or the thumbnail image as the second other image based on the main image can be, for example, 200 times or less. Similarly, the ratio of the size between the screen nail image as the first other image based on the main image and the thumbnail image as the second other image based on the main image can be 200 times or less. As the screen nail image, for example, an image with a resolution of 4K or more can be adopted. Furthermore, as the screen nail image, for example, an image with a resolution of 4K (QFHD) or FHD can be adopted depending on the user's selection. Furthermore, images with the same resolution can be adopted as the main image and the screen nail image. When images with the same resolution are adopted as the main image and the screen nail image, the HEIF file can store both the main image and the screen nail image, or the main image can be stored without storing the screen nail image. When the main image is stored without storing the screen nail image in the HEIF file, the main image can be resized and used as the screen nail image.

[0032] In addition, the encoding control unit 42 encodes the main image, screennail image, and thumbnail image corresponding to the RAW image (the main image, screennail image, and thumbnail image generated from the same RAW image) as necessary, and supplies them to the file control unit 43 together with the RAW image.

[0033] The file control unit 43 generates, as necessary, a RAW file in which a RAW image is stored, a HEIF file in which the corresponding main image, screennail image, and thumbnail image (the main image, screennail image, and thumbnail image generated from the same RAW image) are stored, and / or a JPEG file, and supplies them to the media control unit 44. A HEIF file is a file that complies with HEIF (High Efficiency Image File Format), and a JPEG file is a file that complies with JPEG (Joint Photographic Experts Group).

[0034] The media control unit 44 records the RAW file, HEIF file, or JPEG file from the file control unit 43 on the media 14 or transmits it from the I / F 15 or 16.

[0035] The type of file (e.g., RAW file, HEIF file, JPEG file, etc.) generated by the file control unit 43 can be selected, for example, according to a user operation (designation). Also, as described below, HEIF files come in an image item format and an image sequence format, and whether the image item format or the image sequence format is to be adopted can be selected, for example, according to a user operation. Furthermore, the file control unit 43 can convert between HEIF files and JPEG files according to a user operation.

[0036] Furthermore, the file control unit 43 can generate multiple files of the same image content, but with different codecs, image sizes (resolutions), color formats, and bit depths.

[0037] When the file control unit 43 generates multiple files with the same image content, the encoding control unit 42 generates a stream of images (file stream) to be stored in each of the multiple files from the YUV image from the optical system / image sensor control unit 41.

[0038] The encoding control unit 42 can generate image streams with different codecs, image sizes (resolutions), color formats, and bit depths.

[0039] For example, the encoding control unit 42 can generate an image of a predetermined size, a predetermined color format, and a predetermined bit depth from a YUV image supplied from the optical system / image sensor control unit 41, and generate a first stream by encoding the image with a predetermined codec (encoding method). Furthermore, the encoding control unit 42 can generate an image of a different size, a different color format, and a different bit depth from the same YUV image supplied from the optical system / image sensor control unit 41, and generate a second stream by encoding the image with a different codec.

[0040] Then, the file control unit 43 can generate a file storing the first stream and a file storing the second stream.

[0041] <JPEGファイル>

[0042] FIG. 2 is a diagram showing an example of a JPEG file format that complies with JPEG (Joint Photographic Experts Group).

[0043] A JPEG file is configured by storing, for example, Exif metadata, a thumbnail image, XMP (Extensible Metadata Platform) (registered trademark) metadata, an MPF ​​indicating the storage location (position) of the main image and the simplified display image, etc., a main image, and an image for simplified display. For example, a screen nail image can be used as the image for simplified display.

[0044] <ISOベースメディアファイルフォーマット>

[0045] FIG. 3 is a diagram showing an example of the ISO base media file format.

[0046] HEIF (ISO / IEC 23008-12) is a file format that conforms to the ISO Base Media File Format (ISO / IEC 14496-12), and therefore HEIF files conform to the ISO Base Media File Format.

[0047] The ISO base media file format is composed of units called boxes, which serve as containers for storing data, and has a structure called a box structure.

[0048] A box has a type (box type) and actual data (data). The type indicates the type of actual data in the box. Actual data can be playable media data such as images (still images, videos), audio, subtitles, etc., attribute names (field names) and attribute values ​​(field values) of the attribute names (variables represented by the attribute names), and various other data.

[0049] Furthermore, a box can be used as actual data, that is, a box can have another box as actual data, which allows a hierarchical structure to be created.

[0050] A base media file that complies with the ISO base media file format can have an ftyp box, a moov box (MovieBox), a meta box (MetaBox), an mdat box (MediaDataBox), and the like. The ftyp box stores identification information that identifies the file format. The moov box can store a trak box, and the like. The meta box can store an iinf box, an iprp box, an iref box, an iloc box, and the like. The mdat box can store media data (AV data) and any other data.

[0051] HEIF complies with the above ISO Base Media File Formats.

[0052] <HEIFファイル>

[0053] FIG. 4 is a diagram showing an example of a format of a HEIF file that complies with HEIF.

[0054] HEIF files are broadly divided into an image item format and an image sequence format, and the image item format is further divided into a single image format that has only one item, which will be described later, and an image collection format that has multiple items.

[0055] A HEIF file in the image item format has an ftyp box, a meta box, and an mdat box.

[0056] A HEIF file in image sequence format has an ftyp box, a moov box, and an mdat box.

[0057] Note that a HEIF file can have both a meta box and a moov box, not just one of them.

[0058] The ftyp box stores identification information for identifying the file format, for example, that the file is a HEIF file in image item format or image sequence format.

[0059] The meta box and moov box store metadata necessary for playback and management of the media data stored in the mdat box, such as the storage location of the media data.

[0060] The mdat box stores media data (AV data) and the like.

[0061] In the digital camera 10, which HEIF file to generate, the image item format or the image sequence format, can be selected according to, for example, a user's operation. Also, when encoding and storing an image in the mdat box of the HEIF file, only intra-encoding is permitted for the image item format, and intra-encoding and inter-encoding are permitted for the image sequence format. Therefore, for example, when high-speed access to data stored in the HEIF file is prioritized, generation of a HEIF file in the image item format can be selected, and when reducing the size (amount of data) of the HEIF file is prioritized, generation of a HEIF file in the image sequence format can be selected.

[0062] FIG. 5 is a diagram showing an example of the format of a HEIF file in image item format.

[0063] In a HEIF file in the image item format, information indicating that it is a HEIF file in the image item format, such as mif1, is stored (as an attribute value) in the ftyp box.

[0064] The meta box stores an iinf box, an iref box, an iprp box, and an iloc box.

[0065] The iinf box stores the number of items (attribute names and attribute values ​​representing) that are media data (AV data) stored in the mdat box. An item is a piece of data stored in the mdat box of a HEIF file in image item format; for example, one image (screen) is an item. In this specification, one image, whether a still image or a video, is also called a frame. One frame is one item.

[0066] The iref box stores information indicating the relationship between items. For example, the mdat box can store the corresponding main image, screennail image, and thumbnail image as items. When the mdat box stores item I1 as the main image, item I2 as the screennail image, and item I3 as the thumbnail image, the iref box stores information indicating that item I2 is the screennail image of the main image as item I1, and information indicating that item I3 is the thumbnail image of the main image as item I1.

[0067] The iprp box stores information about the properties of an item.

[0068] The iloc box stores information about the storage location of the items stored in the mdat box.

[0069] In the mdat box of the image item format (HEIF file), for example, a frame of an image is stored as an item. One or more items can be stored in the mdat box. Also, a frame as an item can be encoded and stored in the mdat box. However, encoding of a frame as an item stored in the mdat box of the image item format is limited to intra encoding. As an encoding method (codec) for encoding a frame as an item, for example, HEVC or the like can be adopted.

[0070] FIG. 6 is a diagram showing an example of the iprp box of FIG.

[0071] The iprp box stores an ipco box and an ipma box related to item properties. The ipco box stores the properties of the item stored in the mdat box, such as codec information related to the codec of the image as the item and image size information related to the size. The ipma box stores an index (pointer) to the properties stored in the ipco box of the item stored in the mdat box. 。

[0072] FIG. 7 is a diagram showing an example of the format of a HEIF file in image sequence format.

[0073] In a HEIF file in the image sequence format, information indicating that it is a HEIF file in the image sequence format, such as msf1, is stored in the ftyp box.

[0074] The moov box stores a trak box, which stores information about the track stored in the mdat box.

[0075] A track consists of one independent piece of media data, such as images and audio, that is played according to a timeline. For example, a track consists of one or more frames of images that form an elementary stream. For tracks stored in an mdat box, multiple tracks, for example, image and audio tracks recorded at the same time, can be played simultaneously.

[0076] The media data of a track is composed of units called samples. A sample is the smallest unit (access unit) for accessing media data in a HEIF file. Therefore, it is not possible to access media data in a HEIF file in units smaller than a sample.

[0077] For image media data, one sample is, for example, one frame, etc. For audio media data, one sample is, for example, one audio frame or the like defined by the standard of the audio media data.

[0078] In the mdat box of the image sequence format (HEIF file), the media data of a track is arranged in units called chunks. A chunk is a collection of one or more samples arranged at logically consecutive addresses.

[0079] When multiple tracks of media data are stored in the mdat box, the multiple tracks are arranged in an interleaved manner in chunk units.

[0080] As described above, an mdat box in the image sequence format stores one or more tracks made up of media data such as images and audio.

[0081] In the mdat box, the frames of the images that make up the track can be encoded and stored. To encode the frames that make up the track stored in the mdat box in the image sequence format, long GOP can be used as the GOP (Group of Picture), and both intra-encoding and inter-encoding can be used. As a codec for encoding the frames that make up the track, for example, HEVC can be used.

[0082] FIG. 8 is a diagram showing an example of a trak box.

[0083] The trak box can store a tkhd box and an mdia box. The tkhd box stores header information of the track managed by the trak box, such as the creation date and time of the track. The mdia box stores the minf box and the like. The minf box stores the stbl box. The stbl box stores the stsd box, stsc box, stsz box, and stco box, which store the track samples and, in turn, information for accessing the chunks. The stsd box stores codec information related to the codec of the track. The stsc box stores the chunk size (the number of samples in one chunk). The stsz box stores the sample size. The stco box stores the chunk offset, i.e., the offset of the placement position of each chunk of the track stored in the mdat box.

[0084] Here, a HEIF file in the image item format is also called a collection file, and a HEIF file in the image sequence format is also called a sequence file.

[0085] The digital camera 10 can generate a HEIF file that stores the main image and also any required screennail and / or thumbnail images.

[0086] <Collection file>

[0087] FIG. 9 is a diagram showing an example of a collection file in which main images and thumbnail images are stored.

[0088] Now, it is assumed that frames (items) are encoded in HEVC and stored in the mdat box of the collection file.

[0089] In the ftyp box, heic, which indicates that the file format is an image item format and that the codec is HEVC, is stored as identification information for identifying the file format.

[0090] The number of items (item count) stored in the mdat box is stored in the iinf box. In Fig. 9, a total of four items (frames) are stored in the mdat box: the main image (hereinafter also referred to as main image Item#1) identified by item ID#1, the main image Item#2, the thumbnail image (hereinafter also referred to as thumbnail image Item#101) identified by item ID#101, and the thumbnail image Item#102. Therefore, the number of items is four. Note that the thumbnail image Item#101 is the thumbnail image of the main image Item#1, and the thumbnail image Item#102 is the thumbnail image of the main image Item#2.

[0091] The iinf box further stores an infe box for each item stored in the mdat box. The infe box stores an item ID that identifies the item and an item type. In FIG. 9, there are infe boxes for the main images Item#1 and Item#2, and the thumbnail images Item#101 and Item#102.

[0092] In the iref box, for example, a thmb box is stored as information relating items stored in the mdat box. In the thmb box, a reference source and a reference destination are stored in association with each other as information relating a main image and a thumbnail image of the main image. In the thmb box, the reference source represents the item ID of the main image, and the reference destination represents the item ID of the thumbnail image of the main image specified by the item ID of the reference source. Therefore, according to the reference destination associated with the reference source, it is possible to recognize the item ID of the thumbnail image of the main image specified by the item ID represented by the reference source. Also, according to the reference source associated with the reference destination, it is possible to recognize the item ID of the main image of the thumbnail image specified by the item ID represented by the reference destination.

[0093] The iprp box stores an ipco box and an ipma box, as described in Fig. 6. The ipco box stores properties of a frame as an item stored in the mdat box, such as codec information related to the codec and image size information related to the size, as described in Fig. 6. The ipma box stores an index to the properties stored in the ipco box of the item stored in the mdat box, as described in Fig. 6.

[0094] The iloc box stores information about the storage location of an item in the mdat box, as described in Fig. 6. In Fig. 9, the iloc box stores that the number of items is 4. Furthermore, the iloc box stores the offsets to the storage locations and sizes of the main images Item#1 and Item#2 and the thumbnail images Item#101 and Item#102 stored in the mdat box, in association with the item IDs.

[0095] <Sequence file>

[0096] FIG. 10 is a diagram showing an example of a sequence file in which a track of a main image and a track of thumbnail images of the main image are stored.

[0097] Now, let us assume that frames are encoded in HEVC and stored in the mdat box of the sequence file.

[0098] In the ftyp box, hevc, which indicates that the file format is an image sequence format and that the codec is HEVC, is stored as identification information for identifying the file format.

[0099] As described in Fig. 7, the moov box stores a trak box that manages each track stored in the mdat box. In Fig. 10, the track of the main image identified by track ID #1 (hereinafter also referred to as track #1) and track #2 of the thumbnail image of the main image in track #1 are stored in the mdat box. Therefore, the moov box stores a trak box that manages track #1 and a trak box that manages track #2. The (frame of) n-th thumbnail image (from the beginning) of track #2 is the thumbnail image of the n-th main image in track #1.

[0100] A sequence file is useful, for example, when continuous shooting is performed with the digital camera 10, in which case the main images and thumbnail images of a plurality of frames obtained by the continuous shooting are each recorded as one track.

[0101] The tkhd box of the trak box that manages track #1 of the main images stores track ID #1 that identifies track #1, the image size of the main images that make up track #1, rotation information that indicates the orientation of the digital camera 10 when the main images were captured, and the creation date and time of track #1. The tkhd box of the trak box that manages track #2 of the thumbnail images stores track ID #2 that identifies track #2 and the creation date and time of track #2.

[0102] In addition to the tkhd box and mdia box described in Fig. 7, the trak box can store a tref box. The tref box stores a track ID that identifies other tracks related to the track managed by the trak box in which the tref box is stored, and information that indicates the contents of the track. In Fig. 10, a tref box is provided in a trak box that manages track #2. The tref box stores information that indicates that the other track related to track #2 is track #1 (track_ID=1) and that the data that constitutes track #2 is a thumbnail image (track #2 is a thumbnail image track) (type=thmb).

[0103] In addition to the MINF box described in Fig. 8, the mdia box of the trak box can store an hdlr box. The hdlr box stores information indicating the type of data that constitutes the track managed by the trak box in which the hdlr box is stored. The hdlr box stored in the trak box that manages track #1 of the main image (in the mdia box in which it is stored) stores information (pict) indicating that the data that constitutes track #1 is a picture (frame), and the hdlr box stored in the trak box that manages track #2 of the thumbnail images stores information indicating that the data that constitutes track #2 is a picture.

[0104] The minf box is as explained in FIG.

[0105] <Create file stream>

[0106] FIG. 11 is a diagram for explaining an overview of how the digital camera 10 generates a file stream to be stored in a file.

[0107] In the digital camera 10, the encoding control unit 42 generates a stream (file data) (hereinafter also referred to as a file stream) to be stored in a file generated by the file control unit 43 from the RAW images and YUV images (YUV (YCbCr) (YPbPr) images) supplied from the optical system / image sensor control unit 41.

[0108] The encoding control unit 42, for example, as necessary, supplies (a data stream of) a RAW image from the optical system / image sensor control unit 41 as it is as a RAW stream (ES (Elementary Stream)) to the file control unit 43. Also, for example, the encoding control unit 42 JPEG-encodes a YUV image (YUV image) from the optical system / image sensor control unit 41 as necessary, and supplies the resulting JPEG stream (ES) to the file control unit 43. Also, for example, the encoding control unit 42 HEVC-encodes the YUV image from the optical system / image sensor control unit 41 as necessary, and supplies the resulting HEVC stream (ES) to the file control unit 43.

[0109] When generating a RAW file, the file control unit 43 generates a RAW file that stores the RAW stream and the JPEG stream from the encoding control unit 42. It is sufficient to store at least the RAW stream in the RAW file, and it is not necessary to store the JPEG stream. However, when a JPEG stream is stored in the RAW file, an image can be easily displayed using the JPEG stream. That is, an image may be displayed in order to confirm what kind of image stream is stored in the RAW file. In this case, when a RAW stream is stored in the RAW file but a JPEG stream is not stored, it is necessary to display an image using the RAW stream stored in the RAW file. When an image is displayed using the RAW stream, a process of developing the RAW stream is required. On the other hand, when a JPEG stream is stored in the RAW file, an image can be easily displayed using the JPEG stream without performing a process of development. Hereinafter, a RAW file in which a JPEG stream is stored in addition to a RAW stream is also referred to as an ARW file.

[0110] When generating a JPEG file, the file control unit 43 generates a JPEG file that stores the JPEG stream from the encoding control unit .

[0111] Furthermore, when generating an HEIF file, the file control unit 43 generates an HEIF file that stores the HEVC stream from the encoding control unit 42.

[0112] <File creation>

[0113] FIG. 12 is a diagram illustrating a first example of the generation of multiple files of the same image content in the digital camera 10. In FIG.

[0114] FIG. 12 shows an example in which an ARW file and a JPEG file are generated as multiple files.

[0115] In the digital camera 10, when an ARW file and a JPEG file are generated, the encoding control unit 42 supplies the RAW image supplied from the optical system / image sensor control unit 41 to the file control unit 43 as a RAW stream, which is a file stream stored in the ARW file. Furthermore, the encoding control unit 42 generates a JPEG stream, which is a file stream stored in the ARW file and the JPEG file, from the YUV image supplied from the optical system / image sensor control unit 41, and supplies it to the file control unit 43.

[0116] The file control unit 43 generates an ARW file that stores the RAW stream and the JPEG stream from the encoding control unit 42. Furthermore, the file control unit 43 generates a JPEG file that stores the JPEG stream from the encoding control unit 42. The file control unit 43 supplies the ARW file and the JPEG file to the media control unit 44.

[0117] In addition, the file generated by the file control unit 43 can store not only the original image to be stored in the file (an image equivalent to the main image stored in the HEIF file, hereinafter also referred to as the main image, as in the case of the HEIF file), but also a screennail image and a thumbnail image.

[0118] For example, in an ARW file and a JPEG file, along with a JPEG stream of a main image, a JPEG stream of a screennail image having a smaller size (resolution) than the main image, and a JPEG stream of a thumbnail image having a smaller size than the screennail image can be stored.

[0119] In the following, it is assumed that a file generated by file control unit 43 stores, for example, file streams of a main image, a screennail image, and a thumbnail image.

[0120] For a JPEG stream, if the size of the main image is a predetermined size larger than, for example, FHD, the size of the screennail image is, for example, FHD, and the size of the thumbnail image is, for example, 160 x 120 (horizontal x vertical). Furthermore, for a JPEG stream, the chroma format is, for example, 422 (4:2:2), and the bit depth (the number of bits expressing one pixel component) is, for example, 8 bits.

[0121] Here, the size of the main image, the size of the screennail image, and the size of the thumbnail image are also referred to as the main size, the screennail size, and the thumbnail size, respectively.

[0122] Also, for an image file stream, the data format of the file stream includes the codec, size, chroma format, and bit depth of the image.

[0123] When the file control unit 43 generates an ARW file and a JPEG file, the encoding control unit 42 generates JPEG streams of the main image, screennail image, and thumbnail image from the YUV image supplied from the optical system / image sensor control unit 41.

[0124] The image (here, a YUV image) supplied from the optical system / image sensor control unit 41 to the file control unit 43 and used to generate a file stream stored in the file generated by the file control unit 43 is also referred to as the original image.

[0125] For example, if the JPEG streams of the main image, screennail image, and thumbnail image stored in an ARW file and the JPEG streams of the main image, screennail image, and thumbnail image stored in a JPEG file are each file streams of the same data format, the encoding control unit 42 generates file streams of three types of data formats from the original image, namely, the JPEG streams of the main image, screennail image, and thumbnail image, which differ only in size of the data format.

[0126] In this case, there is not a large number of YUV converters required for the encoding control unit 42. The YUV converter is a converter that converts the size, chroma format, or bit depth of a YUV image as an original image.

[0127] 12, the file control unit 43 generates an ARW file and a JPEG file as multiple files, but the multiple files generated by the file control unit 43 are not limited to this. For example, the file control unit 43 can generate two JPEG files with different main image sizes. Also, for example, the file control unit 43 can generate four files: two JPEG files with different main image sizes, and two ARW files with different main image sizes.

[0128] FIG. 13 is a diagram illustrating a second example of the generation of multiple files of the same image content in the digital camera 10. In FIG.

[0129] FIG. 13 shows an example in which a HEIF file and a JPEG file are generated as multiple files.

[0130] 12 from the YUV image as the original image, and supplies the JPEG stream to the file control unit 43. Furthermore, the encoding control unit 42 generates an HEVC stream as a file stream to be stored in the HEIF file from the YUV image as the original image, and supplies the HEVC stream to the file control unit 43.

[0131] The file control unit 43 generates a JPEG file that stores the JPEG stream from the encoding control unit 42. Furthermore, the file control unit 43 generates a HEIF file that stores the HEVC stream from the encoding control unit 42. The file control unit 43 supplies the JPEG file and the HEIF file to the media control unit 44.

[0132] As described in FIG. 12, when file streams of a main image, a screennail image, and a thumbnail image are stored in an HEIF file, the encoding control unit 42 needs to generate HEVC streams of the main image, the screennail image, and the thumbnail image.

[0133] For the HEVC stream stored in the HEIF file, the main size may be, for example, a size larger than OFHD or FHD, the screen nail size may be, for example, OFHD or FHD, and the thumbnail size may be, for example, 320x240.

[0134] Furthermore, for the HEVC stream, 422 or 420 can be adopted as the chroma format, and 10 bits or 8 bits can be adopted as the bit depth.

[0135] Therefore, when generating HEIF files and JPEG files, it may be necessary to generate images in various data formats having different sizes, chroma formats, and / or bit depths from a YUV image as the original image.

[0136] Therefore, the number of YUV converters required for the encoding control unit 42 increases compared to the case where the ARW files and JPEG files are generated as described with reference to FIG. 12, and the control of the processing performed by the encoding control unit 42 becomes more complicated.

[0137] In addition, when generating HEIF files and JPEG files, the HEVC stream stored in the HEIF file and the JPEG stream stored in the JPEG file have different codecs, so the encoding control unit 42 requires separate codecs for generating the HEVC stream and the JPEG stream.

[0138] Note that the increase in the number of YUV converters required for the encoding control unit 42 is the same not only when the file control unit 43 generates HEIF files and JPEG files as multiple files, but also when, for example, instead of JPEG files, it generates ARW files as shown by dotted lines in the figure, or when it generates two HEIF files with different main image sizes, or when it generates four files: two HEIF files with different main sizes, and two JPEG files or two ARW files with different main sizes.

[0139] Below, a method will be described for simultaneously generating multiple files with the same image content, in which file streams with different data formats are stored, with as simple a configuration as possible.

[0140] In the following, we will use FHD as the screen nail size of the JPEG stream (image), 160x120 as the thumbnail size, 422 as the chroma format, and 8 bits as the bit depth.

[0141] In addition, the HEVC stream screen nail size will be OFHD or FHD, the thumbnail size will be 320x240, the chroma format will be 422 or 420, and the bit depth will be 10 bits or 8 bits.

[0142] However, the screennail size, thumbnail size, chroma format, and bit depth of the JPEG stream and the HEVC stream are not limited to these.

[0143] For example, if JPEG is extended in the future, a data format that complies with the extended JPEG can be adopted. For example, a screen nail size of the JPEG stream that is larger or smaller than FHD, a thumbnail size that is larger or smaller than 160×120, a chroma format of 444, and a bit depth of 12 bits can be adopted.

[0144] Also, for example, the screen nail size of the HEVC stream may be a size other than OFHD or FHD, the thumbnail size may be a size other than 320x240, the chroma format may be 444, and the bit depth may be 12 bits.

[0145] Further, although JPEG and HEVC are adopted as codecs here, the codec is not limited to these. As the codec, a codec other than JPEG and HEVC, for example, VVC (Versatile Video Coding) or the like can be adopted.

[0146] <Handling multiple file generation>

[0147] FIG. 14 is a flow chart illustrating the process of generating multiple files of the same image content in digital camera 10.

[0148] In step S211, the encoding control unit 42 accepts (acquires) a YUV image as an original image from the optical system / image sensor control unit 41, and the process proceeds to step S212.

[0149] In step S212, the encoding control unit 42 generates, from the original image, a plurality of file streams having different data formats to be stored in a plurality of files.

[0150] That is, the encoding control unit 42 generates, from the original image, a plurality of file streams that differ in at least one of the size, the chroma format, the bit depth, and the codec.

[0151] Specifically, the encoding control unit 42 converts part or all of the size, chroma format, and bit depth of the original image as necessary to generate one or more images of the required size, chroma format, and bit depth. Also, the encoding control unit 42 converts the original image as it is into an image of the required size, chroma format, and bit depth as necessary.

[0152] Furthermore, the encoding control unit 42 encodes an image of a required size, chroma format, and bit depth by a required encoding method, and generates a plurality of file streams having at least one different size, chroma format, bit depth, and codec. Note that the encoding control unit 42 can accept a RAW image from the optical system / image sensor control unit 41 as necessary, and make the RAW image into a file stream as it is.

[0153] The encoding control unit 42 supplies the file stream to the file control unit 43, and the process proceeds from step S212 to step S213.

[0154] In step S213, the file control unit 43 stores multiple file streams from the encoding control unit 42, which differ in at least one of size, chroma format, bit depth, and codec, in separate files to generate multiple files.

[0155] <First Configuration Example of the Encoding Control Unit 42>

[0156] FIG. 15 is a block diagram showing a first exemplary configuration of the encoding control unit 42. As shown in FIG.

[0157] In FIG. 15, the encoding control unit 42 is configured as one semiconductor chip 110.

[0158] The semiconductor chip 110 functionally includes an input I / F 111, generation units 112 and 113, and an output I / F 114.

[0159] The input I / F 111 receives a signal from the outside and supplies it to a necessary block. For example, the input I / F 111 receives a YUV image as an original image from the optical system / image sensor control unit 41 and supplies it to the generation units 112 and 113. Also, for example, the input I / F 111 receives a RAW image from the optical system / image sensor control unit 41 and supplies it to the output I / F 114.

[0160] The generating unit 112 functions as a first generating unit that generates a first file stream from the original image from the input I / F 111 and supplies the first file stream to the output I / F 114. For example, the generating unit 112 encodes the original image using a first encoding method, and supplies the resulting stream to the output I / F 114 as a first file stream. Also, for example, the generating unit 112 converts one or more of the size, chroma format, and bit depth of the original image to generate a first converted image. Then, the generating unit 112 encodes the first converted image using a first encoding method, and supplies the resulting stream to the output I / F 114 as a first file stream.

[0161] The generating unit 113 functions as a second generating unit that generates a second file stream from the original image from the input I / F 111 and supplies the second file stream to the output I / F 114. For example, the generating unit 113 encodes the original image using a second encoding method, and supplies the resulting stream to the output I / F 114 as a second file stream. Also, for example, the generating unit 112 converts one or more of the size, chroma format, and bit depth of the original image to generate a second converted image. Then, the generating unit 112 encodes the second converted image using a second encoding method, and supplies the resulting stream to the output I / F 114 as a second file stream.

[0162] The output I / F 114 outputs (supplies) a signal to the outside. For example, the output I / F 114 supplies the first file stream supplied from the generation unit 112 and the second file stream supplied from the generation unit 113 to the file control unit 43. Also, for example, the output I / F 114 supplies the RAW image from the input I / F 111 to the file control unit 43 as (a part of) the first file stream or the second file stream.

[0163] In the file control unit 43, a file in which the first file stream is stored and a file in which the second file stream is stored are generated.

[0164] The first file stream and the second file stream may be file streams of the same or different data formats. When the first file stream and the second file stream are file streams of different data formats, the first file stream and the second file stream are different in one or more of size, chroma format, bit depth, and codec.

[0165] For example, when the file in which the first file stream is stored and the file in which the second file stream is stored, which are generated in the file control unit 43, are arbitrary files, the first file stream and the second file stream may be file streams having different one or more of the chroma format, bit depth, and codec.

[0166] Furthermore, for example, when a HEIF file is included in the file in which the first file stream is stored and the file in which the second file stream is stored, which are generated in the file control unit 43, i.e., when one of the file in which the first file stream is stored and the file in which the second file stream is stored is an HEIF file, the first file stream and the second file stream can be file streams that differ in one or more of size, chroma format, bit depth, and codec.

[0167] FIG. 16 is a diagram illustrating an example of generation of an HEIF file and a JPEG file of the same image content when the encoding control unit 42 is configured as a single semiconductor chip 110.

[0168] Here, there are three sizes of main images (main sizes) that become HEVC streams stored in HEIF files. Similarly, there are three sizes of main images that become JPEG streams stored in JPEG files and ARW files. The three sizes are represented by S, M, and L, in ascending order.

[0169] In addition, the three sizes S, M, and L of the main image that becomes the HEVC stream (hereinafter also referred to as the HEVC main image) and the three sizes S, M, and L of the main image that becomes the JPEG stream (hereinafter also referred to as the JPEG main image) may be the same or different.

[0170] 16, as the original image sig11, for example, a YUV image having a size S, M, or L (S / M / L) of the HEVC main image, a chroma format of 422 or 420 (422 / 420), and a bit depth of 10 bits or 8 bits (10bit / 8bit) is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. The encoding control unit 42 accepts such an original image sig11 as the HEVC main image sig11.

[0171] In addition, in FIG. 16, the sizes S, M, and L of the HEVC main image are larger than the sizes S, M, and L of the JPEG main image, respectively.

[0172] In the encoding control unit 42 configured by a single semiconductor chip 110, a YUV image serving as an original image sig11 from the optical system / image sensor control unit 41 is branched and supplied to generation units 112 and 113.

[0173] Then, in the encoding control unit 42, the generation unit 112 generates a YUV image having a size of QFHD or FHD (QFHD / FHD), a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits from the YUV image as the original image sig11 from the optical system / image sensor control unit 41, as a screennail image (hereinafter also referred to as HEVC screennail image) sig12 that becomes an HEVC stream.

[0174] Furthermore, in the encoding control unit 42, the generation unit 112 generates a YUV image having a size of 320 x 240, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits from the YUV image as the original image sig11, as a thumbnail image (hereinafter also referred to as HEVC thumbnail image) sig13 that becomes an HEVC stream.

[0175] In addition, in the encoding control unit 42, the generation unit 113 generates a YUV image having a size of S, M, or L, a chroma format of 422, and a bit depth of 8 bits from the YUV image as the original image sig11, as a main image (hereinafter also referred to as JPEG main image) sig31 that becomes a JPEG stream.

[0176] Furthermore, in the encoding control unit 42, the generation unit 113 generates a YUV image having a size of FHD, a chroma format of 422, and a bit depth of 8 bits from the YUV image as the original image sig11 as a screennail image (hereinafter also referred to as a JPEG screennail image) sig32 that becomes a JPEG stream.

[0177] In addition, in the encoding control unit 42, the generation unit 113 generates a YUV image having a size of 160 x 120, a chroma format of 422, and a bit depth of 8 bits from the YUV image as the original image sig11, as a thumbnail image (hereinafter also referred to as a JPEG thumbnail image) sig33 that becomes a JPEG stream.

[0178] As described above, the HEVC main image sig11, HEVC screennail image sig12, and HEVC thumbnail image sig13, as well as the JPEG main image sig31, JPEG screennail image sig32, and JPEG thumbnail image sig33, are all generated from the common (same) (single) original image sig11. This makes it easy to simultaneously generate multiple files with the same image content, in which file streams with different data formats are stored, with as simple a configuration as possible.

[0179] The data format (excluding the codec) of the original image is to be equal to or higher than the data format of the image that will become the file stream stored in the file generated by the file control unit 43. In other words, the size, chroma format, and bit depth of the original image are to be equal to or higher than the size, chroma format, and bit depth of the image that will become the file stream.

[0180] For example, in FIG. 16, when size L is adopted as the size of the JPEG main image, if the size L of the JPEG main image is larger than the size S of the HEVC main image but smaller than the size M of the HEVC main image, the size of the original image sig11 is adopted as the M size or L size of the HEVC main image, which is equal to or larger than the L size of the JPEG main image.

[0181] 16, for example, 422 or 420 can be adopted as the chroma format of the original image sig11. However, 422 is adopted as the color format of the JPEG main image sig31, the JPEG screennail image sig32, and the JPEG thumbnail image sig33. Therefore, of 422 and 420, 422, which is equal to or greater than the color format 422 (resolution) of the JPEG main image sig31, the JPEG screennail image sig32, and the JPEG thumbnail image sig33, is adopted as the chroma format of the original image sig11.

[0182] As described above, the data format of the original image can be a data format that is equal to or better than the data format (specifications) of the image that will become the file stream. In this case, for example, it is possible to prevent the image quality of the main image from being substantially the same as the image quality of the screennail image or thumbnail image.

[0183] The data format of the original image may be a data format lower than the data format of the image that will become the file stream. In other words, the data format of the original image may be a data format with lower specifications than the data format of the image that will become the file stream. In this case, the image that will become the file stream is generated by interpolation using the original image.

[0184] Thereafter, in the encoding control unit 42, the generation unit 112 HEVC-encodes the HEVC main image sig11, the HEVC screennail image sig12, and the HEVC thumbnail image sig13. As a result, HEVC streams st11, st12, and st13 of the main image, the screennail image, and the thumbnail image are generated (as first file streams, respectively). Similarly, in the encoding control unit 42, the generation unit 113 JPEG-encodes the JPEG main image sig31, the JPEG screennail image sig32, and the JPEG thumbnail image sig33. As a result, JPEG streams st21, st22, and st23 of the main image, the screennail image, and the thumbnail image are generated (as second file streams, respectively).

[0185] The encoding control unit 42 supplies the HEVC streams st11, st12, and st13 of the main image, screennail image, and thumbnail image, and the JPEG streams st21, st22, and st23 to the file control unit 43.

[0186] The file control unit 43 generates an HEIF file f11 that stores HEVC streams st11, st12, and st13 of the main image, screennail image, and thumbnail image from the encoding control unit 42. Furthermore, the file control unit 43 generates a JPEG file f12 that stores JPEG streams st21, st22, and st23 of the main image, screennail image, and thumbnail image from the encoding control unit 42.

[0187] In addition, in FIG. 16, the HEVC thumbnail image is generated directly using the original image, but the HEVC thumbnail image can be generated in the encoding control unit 42 using images generated in the process of generating images of various data formats.

[0188] For example, the HEVC thumbnail image sig13 can be generated using the HEVC screennail image sig12, as shown by the hollow arrow in the figure. Similarly, for example, the JPEG screennail image sig32 can be generated using the JPEG main image sig31, and the JPEG thumbnail image sig33 can be generated using the JPEG screennail image sig32.

[0189] Furthermore, for example, if the data format (excluding codec) of the HEVC screennail image sig12 and the JPEG screennail image sig32 are the same, i.e., if the HEVC screennail image sig12 and the JPEG screennail image sig32 are YUV images with a size of FHD, a chroma format of 422, and a bit depth of 8 bits, then, for example, the HEVC screennail image sig12 generated from the original image sig11 can be used as the JPEG screennail image sig32 as is.

[0190] The above also applies to the generation of multiple files with the same image content, which will be described below.

[0191] FIG. 17 is a diagram illustrating another example of generation of an HEIF file and a JPEG file of the same image content when the encoding control unit 42 is configured as a single semiconductor chip 110.

[0192] 17, as the original image sig41, for example, a YUV image having a size S, M, or L of the JPEG main image, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. The encoding control unit 42 accepts such an original image sig41 as the JPEG main image sig41.

[0193] In addition, in FIG. 17, the sizes S, M, and L of the JPEG main image are larger than the sizes S, M, and L of the HEVC main image, respectively.

[0194] In the encoding control unit 42, the generation unit 112 generates a YUV image as the HEVC main image sig51 from the YUV image as the original image sig41 from the optical system / image sensor control unit 41, the YUV image having the sizes S, M, and L of the HEVC main image, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits.

[0195] In addition, in the encoding control unit 42, the generation unit 112 generates a YUV image having a size of QFHD or FHD, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as an HEVC screennail image sig52 from the YUV image as the original image sig41.

[0196] Furthermore, in the encoding control unit 42, the generation unit 112 generates a YUV image having a size of 320 x 240, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits from the YUV image as the original image sig41, as an HEVC thumbnail image sig53.

[0197] Furthermore, in the encoding control unit 42, the generation unit 113 generates a YUV image having a size of FHD, a chroma format of 422, and a bit depth of 8 bits as a JPEG screennail image sig42 from the YUV image as the original image sig41.

[0198] Furthermore, in the encoding control unit 42, the generation unit 113 generates a YUV image having a size of 160×120, a chroma format of 422, and a bit depth of 8 bits as a JPEG thumbnail image sig43 from the YUV image as the original image sig41.

[0199] As described above, the HEVC main image sig51, HEVC screennail image sig52, and HEVC thumbnail image sig53, as well as the JPEG main image sig41, JPEG screennail image sig42, and JPEG thumbnail image sig43 are all generated from the same original image sig41.

[0200] 16, in the encoding control unit 42, the generation unit 112 HEVC-encodes the HEVC main image sig51, the HEVC screennail image sig52, and the HEVC thumbnail image sig53. This generates HEVC streams st31, st32, and st33 of the main image, the screennail image, and the thumbnail image. Similarly, in the encoding control unit 42, the generation unit 113 JPEG-encodes the JPEG main image sig41, the JPEG screennail image sig42, and the JPEG thumbnail image sig43. This generates JPEG streams st41, st42, and st43 of the main image, the screennail image, and the thumbnail image.

[0201] The encoding control unit 42 supplies the HEVC streams st31, st32, and st33 of the main image, screennail image, and thumbnail image, and the JPEG streams st41, st42, and st43 to the file control unit 43.

[0202] The file control unit 43 generates an HEIF file f31 that stores HEVC streams st31, st32, and st33 of the main image, screennail image, and thumbnail image from the encoding control unit 42. Furthermore, the file control unit 43 generates a JPEG file f32 that stores JPEG streams st41, st42, and st43 of the main image, screennail image, and thumbnail image from the encoding control unit 42.

[0203] FIG. 18 is a diagram illustrating an example of generating an HEIF file and an ARW file of the same image content when the encoding control unit 42 is configured as a single semiconductor chip 110.

[0204] 18, as in Fig. 16, for example, a YUV image having a size S, M, or L of the HEVC main image, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits is supplied as the original image sig11 from the optical system / image sensor control unit 41 to the encoding control unit 42. The encoding control unit 42 accepts such an original image sig11 as the HEVC main image sig11.

[0205] In addition, in FIG. 18, similarly to FIG. 16, the sizes S, M, and L of the HEVC primary image are larger than the sizes S, M, and L of the JPEG primary image, respectively.

[0206] In the encoding control unit 42, the generation unit 112 generates an HEVC screennail image sig12 and an HEVC thumbnail image sig13 similar to those in FIG. 16 from the YUV image serving as the original image sig11 from the optical system / image sensor control unit 41.

[0207] Furthermore, in the encoding control unit 42, the generating unit 113 generates a JPEG main image sig31, a JPEG screennail image sig32, and a JPEG thumbnail image sig33, similar to the case of FIG. 16, from the YUV image serving as the original image sig11.

[0208] 16, in the encoding control unit 42, the generation unit 112 HEVC-encodes the HEVC main image sig11, the HEVC screennail image sig12, and the HEVC thumbnail image sig13. This generates HEVC streams st11, st12, and st13 of the main image, the screennail image, and the thumbnail image. Similarly, in the encoding control unit 42, the generation unit 113 JPEG-encodes the JPEG main image sig31, the JPEG screennail image sig32, and the JPEG thumbnail image sig33. This generates JPEG streams st21, st22, and st23 of the main image, the screennail image, and the thumbnail image.

[0209] The encoding control unit 42 supplies the HEVC streams st11, st12, and st13 of the main image, screennail image, and thumbnail image, and the JPEG streams st21, st22, and st23 to the file control unit 43.

[0210] 18, the encoding control unit 42 supplies the RAW image sig71 from the optical system / image sensor control unit 41 to the file control unit 43 as a RAW stream sig71. The file control unit 43 generates an HEIF file f11 storing the HEVC streams st11, st12, and st13 of the main image, screennail image, and thumbnail image from the encoding control unit 42. The file control unit 43 also generates an ARW file f51 storing the RAW stream sig71 from the encoding control unit 42, and the JPEG streams st21, st22, and st23 of the main image, screennail image, and thumbnail image.

[0211] FIG. 19 is a diagram illustrating an example of generating two HEIF files with the same image content when the encoding control unit 42 is configured from a single semiconductor chip 110.

[0212] Here, the two HEIF files generated in Fig. 19 store (HEVC streams of) HEVC main images having different sizes, for example, in a data format. For example, one of the two HEIF files stores an HEVC main image of size L, and the other HEIF file stores an HEVC main image of size S.

[0213] The HEIF file in which the HEVC main image of size L is stored is also referred to as the first HEIF file, and the HEIF file in which the HEVC main image of size S is stored is also referred to as the second HEIF file.

[0214] In addition, the HEVC main image (HEVC stream) stored in the first HEIF file is also referred to as the first HEVC main image, and the HEVC main image stored in the second HEIF file is also referred to as the second HEVC main image.

[0215] 19, as the original image sig101, for example, a YUV image having a size L of the HEVC main image, a chroma format of 422, and a bit depth of 10 bits is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. The encoding control unit 42 accepts such an original image sig101 as the first HEVC main image sig101.

[0216] In the encoding control unit 42, the generation unit 112 generates a YUV image having a size of QFHD, a chroma format of 422, and a bit depth of 10 bits as an HEVC screennail image sig102 from the YUV image serving as the original image sig101 from the optical system / image sensor control unit 41.

[0217] Furthermore, in the encoding control unit 42, the generation unit 112 generates a YUV image having a size of 320×240, a chroma format of 422, and a bit depth of 10 bits as an HEVC thumbnail image sig103 from the YUV image as the original image sig101.

[0218] In addition, in the encoding control unit 42, the generation unit 113 generates a YUV image having a size S of the HEVC main image, a chroma format of 422, and a bit depth of 10 bits as the second HEVC main image sig111 from the YUV image as the original image sig101.

[0219] Thereafter, in the encoding control unit 42, the generation unit 112 HEVC-encodes the first HEVC main image sig101, the HEVC screennail image sig102, and the HEVC thumbnail image sig103. As a result, HEVC streams st101, st102, and st103 of the first HEVC main image, the HEVC screennail image, and the HEVC thumbnail image are generated.

[0220] Moreover, in the encoding control unit 42, the generation unit 113 HEVC-encodes the second HEVC primary image sig111 to generate an HEVC stream st111 of the second HEVC primary image.

[0221] The encoding control unit 42 supplies the HEVC streams st101, st111, st102, and st103 of the first HEVC main image, the second HEVC main image, the HEVC screennail image, and the HEVC thumbnail image to the file control unit 43.

[0222] The file control unit 43 generates a first HEIF file f101 storing HEVC streams st101, st102, and st103 of the first HEVC main image, HEVC screennail image, and HEVC thumbnail image from the encoding control unit 42. Furthermore, the file control unit 43 generates a second HEIF file f102 storing HEVC streams st111, st102, and st103 of the second HEVC main image, HEVC screennail image, and HEVC thumbnail image from the encoding control unit 42.

[0223] FIG. 20 is a diagram for explaining an example of generating two JPEG files having the same image content when the encoding control unit 42 is configured from a single semiconductor chip 110. In FIG.

[0224] Here, the two JPEG files generated in Fig. 20 store (JPEG streams of) JPEG main images with different bit depths in the data format, for example. For example, one of the two JPEG files stores a JPEG main image with a bit depth of 10 bits, and the other JPEG file stores a JPEG main image with a bit depth of 8 bits.

[0225] A JPEG file in which a JPEG main image with a bit depth of 10 bits is stored is also called a first JPEG file, and a JPEG file in which a JPEG main image with a bit depth of 8 bits is also called a second JPEG file.

[0226] In addition, the JPEG primary image stored in the first JPEG file (JPEG stream) is also referred to as the first JPEG primary image, and the JPEG primary image stored in the second JPEG file is also referred to as the second JPEG primary image.

[0227] It should be noted that although current JPEG cannot handle a bit depth of 10 bits, it is assumed here that JPEG will be extended in the future to be able to handle a bit depth of 10 bits.

[0228] 20, as the original image sig131, for example, a YUV image having a size L of the JPEG main image, a chroma format of 422, and a bit depth of 10 bits is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. The encoding control unit 42 accepts such an original image sig131 as the first JPEG main image sig131.

[0229] In the encoding control unit 42, the generation unit 112 generates a YUV image having a size of FHD, a chroma format of 422, and a bit depth of 8 bits as a JPEG screennail image sig132 from the YUV image serving as the original image sig131 from the optical system / image sensor control unit 41.

[0230] Furthermore, in the encoding control unit 42, the generation unit 112 generates a YUV image having a size of 160×120, a chroma format of 422, and a bit depth of 8 bits as a JPEG thumbnail image sig133 from the YUV image as the original image sig131.

[0231] In addition, in the encoding control unit 42, the generation unit 113 generates a YUV image having a size L of the JPEG main image, a chroma format of 422, and a bit depth of 8 bits as the second JPEG main image sig141 from the YUV image as the original image sig131.

[0232] Thereafter, in the encoding control unit 42, the generation unit 112 JPEG-encodes the first JPEG main image sig131, the JPEG screennail image sig132, and the JPEG thumbnail image sig133. As a result, JPEG streams st131, st132, and st133 of the first JPEG main image, the JPEG screennail image, and the JPEG thumbnail image are generated.

[0233] Moreover, in the encoding control unit 42, the generating unit 113 JPEG-encodes the second JPEG primary image sig141 to generate a JPEG stream st141 of the second JPEG primary image.

[0234] The encoding control unit 42 supplies the file control unit 43 with the JPEG streams st131, st141, st132, and st133 of the first JPEG main image, the second JPEG main image, the JPEG screennail image, and the JPEG thumbnail image.

[0235] The file control unit 43 generates a first JPEG file f111 storing the JPEG streams st131, st132, and st133 of the first JPEG main image, JPEG screennail image, and JPEG thumbnail image from the encoding control unit 42. Furthermore, the file control unit 43 generates a second JPEG file f112 storing the JPEG streams st141, st132, and st133 of the second JPEG main image, JPEG screennail image, and JPEG thumbnail image from the encoding control unit 42.

[0236] <Second Configuration Example of the Encoding Control Unit 42>

[0237] FIG. 21 is a block diagram showing a second exemplary configuration of the encoding control unit 42. As shown in FIG.

[0238] 21, the encoding control unit 42 is composed of two semiconductor chips 210 and 220. The semiconductor chips 210 and 220 are connected in series, with the semiconductor chip 210 being disposed in the front stage and the semiconductor chip 220 being disposed in the rear stage.

[0239] The semiconductor chip 210 functionally includes an input I / F 211, a generation unit 212, and an output I / F 213.

[0240] The input I / F 211 receives a signal from the outside and supplies it to a necessary block. For example, the input I / F 211 receives a YUV image as an original image from the optical system / image sensor control unit 41 and supplies it to the generation unit 212 and the output I / F 213. Also, for example, the input I / F 211 receives a RAW image from the optical system / image sensor control unit 41 and supplies it to the output I / F 213.

[0241] The generation unit 212 generates a first file stream from the original image from the input I / F 211, and supplies the first file stream to the output I / F 213. For example, the generation unit 212 encodes the original image using a first encoding method, and supplies the resulting stream to the output I / F 213 as a first file stream. Also, for example, the generation unit 212 converts one or more of the size, chroma format, and bit depth of the original image to generate a first converted image. Then, the generation unit 212 encodes the first converted image using a first encoding method, and supplies the resulting stream to the output I / F 213 as a first file stream.

[0242] The output I / F 213 outputs (supplies) a signal to the outside. For example, the output I / F 213 supplies the original image and the RAW image supplied from the input I / F 211 and the first file stream supplied from the generation unit 212 to the semiconductor chip 220.

[0243] The semiconductor chip 220 functionally includes an input I / F 221, a generation unit 222, and an output I / F 223.

[0244] The input I / F 221 receives a signal from the outside and supplies it to a necessary block. For example, the input I / F 221 receives a YUV image as an original image from (the output I / F 213 of) the semiconductor chip 210 and supplies it to the generation unit 222. Also, for example, the input I / F 221 receives a first file stream and a RAW image from the semiconductor chip 210 and supplies them to the output I / F 223.

[0245] The generation unit 222 generates a second file stream from the original image from the input I / F 221, and supplies the second file stream to the output I / F 223. For example, the generation unit 222 encodes the original image using a second encoding method, and supplies the resulting stream to the output I / F 223 as a second file stream. Also, for example, the generation unit 222 converts one or more of the size, chroma format, and bit depth of the original image to generate a second converted image. Then, the generation unit 222 encodes the second converted image using a second encoding method, and supplies the resulting stream to the output I / F 223 as a second file stream.

[0246] The output I / F 223 outputs (supplies) a signal to the outside. For example, the output I / F 223 supplies the first file stream and the RAW image supplied from the input I / F 221, and the second file stream supplied from the generation unit 222 to the file control unit 43.

[0247] In the file control unit 43, a file in which the first file stream is stored and a file in which the second file stream is stored are generated.

[0248] As described above, the encoding control unit 42, which is composed of the two semiconductor chips 210 and 220, is useful when multiple files with the same image content generated by the digital camera 10 have different file formats. For example, one of the semiconductor chips 210 and 220 can generate a file stream (first file stream) to be stored in a file of one file format. Furthermore, the other semiconductor chip can generate a file stream (second file stream) to be stored in a file of another file format.

[0249] FIG. 22 is a diagram illustrating an example of generation of an HEIF file and a JPEG file of the same image content when the encoding control unit 42 is configured from two semiconductor chips 210 and 220.

[0250] 22, as the original image sig11, for example, a YUV image having a size S, M, or L of the HEVC main image, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. In the encoding control unit 42, the front-stage semiconductor chip 210 accepts such an original image sig11 as the HEVC main image sig11.

[0251] In addition, in FIG. 22, the sizes S, M, and L of the HEVC main image are larger than the sizes S, M, and L of the JPEG main image, respectively.

[0252] In the encoding control unit 42, the upstream semiconductor chip 210 generates a YUV image having a size of QFHD or FHD, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as an HEVC screennail image sig12 from the YUV image as the original image sig11 from the optical system / image sensor control unit 41.

[0253] Furthermore, in the encoding control unit 42, the semiconductor chip 210 generates a YUV image having a size of 320 x 240, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as an HEVC thumbnail image sig13 from the YUV image as the original image sig11.

[0254] Thereafter, in the encoding control unit 42, the semiconductor chip 210 HEVC-encodes the HEVC main image sig11, the HEVC screennail image sig12, and the HEVC thumbnail image sig13. As a result, HEVC streams st11, st12, and st13 of the main image, the screennail image, and the thumbnail image are generated (as first file streams, respectively).

[0255] The semiconductor chip 210 supplies the original image sig11, and the HEVC streams st11, st12, and st13 of the main image, screennail image, and thumbnail image to the semiconductor chip 220 in the subsequent stage.

[0256] Therefore, in the encoding control unit 42 configured with the two semiconductor chips 210 and 220, the original image sig11 is transferred from the semiconductor chip 210 at the front stage to the semiconductor chip 220 at the rear stage.

[0257] In the encoding control unit 42, the subsequent-stage semiconductor chip 220 receives the original image sig11 from the preceding-stage semiconductor chip 210, as well as the HEVC streams st11, st12, and st13 of the main image, screennail image, and thumbnail image.

[0258] Then, the semiconductor chip 220 generates a YUV image having a size of S, M, or L, a chroma format of 422, and a bit depth of 8 bits as a JPEG main image sig31 from the YUV image serving as the original image sig11 from the preceding semiconductor chip 210.

[0259] Furthermore, the semiconductor chip 220 generates a YUV image having a size of FHD, a chroma format of 422, and a bit depth of 8 bits as a JPEG screennail image sig32 from the YUV image as the original image sig11.

[0260] Furthermore, the semiconductor chip 220 generates a YUV image having a size of 160×120, a chroma format of 422, and a bit depth of 8 bits as a JPEG thumbnail image sig33 from the YUV image as the original image sig11.

[0261] After that, the semiconductor chip 220 JPEG-encodes the JPEG main image sig31, the JPEG screennail image sig32, and the JPEG thumbnail image sig33, thereby generating JPEG streams st21, st22, and st23 of the main image, the screennail image, and the thumbnail image (respectively as second file streams).

[0262] Then, the semiconductor chip 220 supplies the file control unit 43 with the JPEG streams st21, st22, and st23 of the main image, the screennail image, and the thumbnail image.

[0263] Furthermore, the semiconductor chip 220 supplies the HEVC streams st11, st12, and st13 of the main image, screennail image, and thumbnail image from the semiconductor chip 210 in the preceding stage to the file control unit 43.

[0264] The file control unit 43 generates an HEIF file f11 that stores HEVC streams st11, st12, and st13 of the main image, screennail image, and thumbnail image from the semiconductor chip 220. Furthermore, the file control unit 43 generates a JPEG file f12 that stores JPEG streams st21, st22, and st23 of the main image, screennail image, and thumbnail image from the semiconductor chip 220.

[0265] FIG. 23 is a diagram illustrating another example of generation of an HEIF file and a JPEG file of the same image content when the encoding control unit 42 is configured from two semiconductor chips 210 and 220.

[0266] 22, HEVC streams st11, st12, and st13 are generated in the front-stage semiconductor chip 210, and JPEG streams st21, st22, and st23 are generated in the rear-stage semiconductor chip 220. On the other hand, in Fig. 23, conversely, JPEG streams st21, st22, and st23 are generated in the front-stage semiconductor chip 210, and HEVC streams st11, st12, and st13 are generated in the rear-stage semiconductor chip 220, which is different from the case of Fig. 22.

[0267] 23, for example, a YUV image having a size S, M, or L of the HEVC main image, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits is supplied as the original image sig11 from the optical system / image sensor control unit 41 to the encoding control unit 42. In the encoding control unit 42, the front-stage semiconductor chip 210 accepts such an original image sig11 as the HEVC main image sig11.

[0268] Then, the semiconductor chip 210 generates a YUV image having a size of S, M, or L, a chroma format of 422, and a bit depth of 8 bits as a JPEG main image sig31 from the YUV image as the original image sig11 from the optical system / image sensor control unit 41.

[0269] Furthermore, the semiconductor chip 210 generates a YUV image having a size of FHD, a chroma format of 422, and a bit depth of 8 bits as a JPEG screennail image sig32 from the YUV image as the original image sig11.

[0270] Furthermore, the semiconductor chip 210 generates a YUV image having a size of 160×120, a chroma format of 422, and a bit depth of 8 bits as a JPEG thumbnail image sig33 from the YUV image as the original image sig11.

[0271] After that, the semiconductor chip 210 JPEG-encodes the JPEG main image sig31, the JPEG screennail image sig32, and the JPEG thumbnail image sig33, thereby generating JPEG streams st21, st22, and st23 of the main image, the screennail image, and the thumbnail image (respectively as a first file stream).

[0272] The semiconductor chip 210 supplies the original image sig11, and JPEG streams st21, st22, and st23 of the main image, screennail image, and thumbnail image to the semiconductor chip 220 in the subsequent stage.

[0273] In the encoding control unit 42, the subsequent-stage semiconductor chip 220 accepts the original image sig11 from the preceding-stage semiconductor chip 210 as the HEVC main image sig11. Furthermore, the semiconductor chip 220 accepts JPEG streams st21, st22, and st23 of the main image, screennail image, and thumbnail image from the preceding-stage semiconductor chip 210.

[0274] Then, the subsequent semiconductor chip 220 generates a YUV image having a size of QFHD or FHD, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as an HEVC screennail image sig12 from the YUV image serving as the original image sig11 from the previous semiconductor chip 210.

[0275] Furthermore, the semiconductor chip 220 generates a YUV image having a size of 320×240, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as an HEVC thumbnail image sig13 from the YUV image as the original image sig11.

[0276] Then, the semiconductor chip 220 HEVC-encodes the HEVC main image sig11, the HEVC screennail image sig12, and the HEVC thumbnail image sig13. This generates HEVC streams st11, st12, and st13 of the main image, the screennail image, and the thumbnail image (respectively as second file streams).

[0277] Then, the semiconductor chip 220 supplies the HEVC streams st11, st12, and st13 of the main image, the screennail image, and the thumbnail image to the file control unit 43.

[0278] Furthermore, the semiconductor chip 220 supplies the file control unit 43 with JPEG streams st21, st22, and st23 of the main image, screennail image, and thumbnail image from the semiconductor chip 210 in the preceding stage.

[0279] The file control unit 43 generates an HEIF file f11 that stores HEVC streams st11, st12, and st13 of the main image, screennail image, and thumbnail image from the semiconductor chip 220. Furthermore, the file control unit 43 generates a JPEG file f12 that stores JPEG streams st21, st22, and st23 of the main image, screennail image, and thumbnail image from the semiconductor chip 220.

[0280] The above describes the generation of multiple files with the same image content in the encoding control unit 42 (hereinafter also referred to as the encoding control unit 42 having a two-chip configuration) consisting of two semiconductor chips 210 and 220, taking the generation of an HEIF file and a JPEG file as an example. In the encoding control unit 42 having a two-chip configuration, in addition to the generation of an HEIF file and a JPEG file, for example, the generation of an HEIF file and an ARW file as described in FIG. 18 can be performed. In addition, in the encoding control unit 42 having a two-chip configuration, for example, the generation of two HEIF files as described in FIG. 19 and the generation of two JPEG files as described in FIG. 20 can be performed. Furthermore, in the encoding control unit 42 having a two-chip configuration, for example, the generation of three or more files and the generation of multiple files including files other than an HEIF file, a JPEG file, and an ARW file can be performed.

[0281] In addition, in digital camera 10, for example, one or more of the data formats (codec, size, chroma format, and bit depth) of file streams stored in multiple files with the same image content can be set in response to user operation (specification).

[0282] <Description of the computer to which this technology is applied>

[0283] Next, the series of processes of each block constituting the above-mentioned encoding control unit 42 and other signal processing unit 13 (FIG. 1) can be performed by hardware or software. When the series of processes are performed by software, a program constituting the software is installed in a computer or the like.

[0284] FIG. 24 is a block diagram showing an example of the configuration of an embodiment of a computer in which a program for executing the series of processes described above is installed.

[0285] The program can be recorded in advance on the hard disk 905 or ROM 903 as a recording medium built into the computer.

[0286] Alternatively, the program can be stored (recorded) in a removable recording medium 911 driven by the drive 909. Such a removable recording medium 911 can be provided as a so-called package software. Here, examples of the removable recording medium 911 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

[0287] The program can be installed in the computer from the removable recording medium 911 as described above, or can be downloaded to the computer via a communication network or a broadcasting network and installed in the built-in hard disk 905. That is, the program can be transferred to the computer wirelessly from a download site via an artificial satellite for digital satellite broadcasting, or transferred to the computer by wire via a network such as a LAN (Local Area Network) or the Internet.

[0288] The computer includes a built-in CPU (Central Processing Unit) 902 , to which an input / output I / F 910 is connected via a bus 901 .

[0289] When a command is input by a user operating an input unit 907 via an input / output I / F 910, the CPU 902 executes a program stored in a read only memory (ROM) 903 in accordance with the command. Alternatively, the CPU 902 loads a program stored in a hard disk 905 into a random access memory (RAM) 904 and executes the program.

[0290] As a result, the CPU 902 performs processing according to the above-mentioned flowchart or processing performed by the configuration of the above-mentioned block diagram. Then, the CPU 902 outputs the processing result from the output unit 906 via the input / output I / F 910, or transmits it from the communication unit 908, or further records it on the hard disk 905, as necessary.

[0291] The input unit 907 is composed of a keyboard, a mouse, a microphone, etc. The output unit 906 is composed of an LCD (Liquid Crystal Display), a speaker, etc.

[0292] Here, in this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order according to the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing executed in parallel or individually (for example, parallel processing or processing by objects).

[0293] The program may be processed by one computer (processor), or may be distributed among multiple computers. Furthermore, the program may be transferred to a remote computer for execution.

[0294] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device in which multiple modules are housed in a single housing, are both systems.

[0295] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0296] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0297] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0298] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices.

[0299] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0300] The present technology can have the following configurations.

[0301] <1> an encoding control unit that generates file data obtained by encoding an image from a single image and the file data is stored in each of a plurality of files; The first file data stored in one of the plurality of files and the second file data stored in the other file are data that differ in at least one of codec, chroma format, and bit depth. Data processing device. <2> The same image has a size, chroma format, and bit depth equal to or greater than the image whose file data is stored in the file. <1> A data processing device according to the present invention. <3> Set one or more of the codec, chroma format, and bit depth of the file data according to a user's designation. <1> or <2> A data processing device according to the present invention. <4> The one file or the other file is a RAW file in which a RAW image is stored. <1> Or <3> 2. A data processing device according to claim 1 , <5> the encoding control unit is composed of a single semiconductor chip, The one semiconductor chip is a first generation unit that generates one of the first file data and the second file data; a second generating unit for generating the other file data; have <1> Or <4> 2. A data processing device according to claim 1 , <6> The encoding control unit is composed of two semiconductor chips, one of the two semiconductor chips generates one of the first file data and the second file data; The other semiconductor chip generates the other file data. <1> Or <4> 2. A data processing device according to claim 1 , <7> the one semiconductor chip receives the same image, generates file data of the one semiconductor chip from the same image, and transfers the same image to the other semiconductor chip; The other semiconductor chip generates the other file data from the same image from the one semiconductor chip. <6> A data processing device according to the present invention. <8> The first file data is data obtained by encoding an image using HEVC, The second file data is JPEG-encoded image data. <1> Or <7> 2. A data processing device according to claim 1 , <9> generating file data encoding the image from the same image, the file data being stored in each of a plurality of files; The first file data stored in one of the plurality of files and the second file data stored in the other file are data that differ in at least one of codec, chroma format, and bit depth. Data processing methods. <10> A program for causing a computer to function as an encoding control unit that generates file data obtained by encoding an image from a single image and storing the file data in each of a plurality of files, The first file data stored in one of the plurality of files and the second file data stored in the other file are data that differ in at least one of codec, chroma format, and bit depth. program. <11> An encoding control unit that generates file data by encoding an image from a single image, the file data being stored in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file; The first file data stored in the HEIF file among the plurality of files and the second file data stored in another file other than the HEIF file are data that differ in at least one of codec, chroma format, bit depth, and image size. Data processing device. <12> The same image has a size, chroma format, and bit depth equal to or greater than the image whose file data is stored in the file. <11> A data processing device according to the present invention. <13> Set one or more of the codec, chroma format, bit depth, and image size of the file data according to a user's designation. <11> or <12> A data processing device according to the present invention. <14> The other file is a RAW file in which a RAW image is stored. <11> Or <13> 2. A data processing device according to claim 1 , <15> the encoding control unit is composed of a single semiconductor chip, The one semiconductor chip is a first generation unit that generates one of the first file data and the second file data; a second generating unit for generating the other file data; have <11> Or <14> 2. A data processing device according to claim 1 , <16> The encoding control unit is composed of two semiconductor chips, one of the two semiconductor chips generates one of the first file data and the second file data; The other semiconductor chip generates the other file data. <11> Or <14> 2. A data processing device according to claim 1 , <17> the one semiconductor chip receives the same image, generates file data of the one semiconductor chip from the same image, and transfers the same image to the other semiconductor chip; The other semiconductor chip generates the other file data from the same image from the one semiconductor chip. <16> A data processing device according to the present invention. <18> The first file data is data obtained by encoding an image using HEVC, The second file data is JPEG-encoded image data. <11> Or <17> 2. A data processing device according to claim 1 , <19> generating file data encoding the image from the same image, the file data being stored in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file; The first file data stored in the HEIF file among the plurality of files and the second file data stored in another file other than the HEIF file are data that differ in at least one of codec, chroma format, bit depth, and image size. Data processing methods. <20> A program for causing a computer to function as an encoding control unit that generates file data of encoded images from a single image, the file data being stored in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file; The first file data stored in the HEIF file among the plurality of files and the second file data stored in another file other than the HEIF file are data that differ in at least one of codec, chroma format, bit depth, and image size. program. [Explanation of symbols]

[0302] 10 digital camera, 11 optical system, 13 signal processing section, 14 media, 15,16 I / F, 17 button / key, 18 touch panel, 19 LCD panel, 20 viewfinder, 21 I / F, 41 optical system / image sensor control section, 42 encoding control section, 43 file control section, 44 media control section, 45 operation control section, 46 display control section, 47 UI control section, 110 semiconductor chip, 111 input I / F, 112,113 generator, 114 output I / F, 210 semiconductor chip, 211 input I / F, 212 generator, 213 output I / F, 220 semiconductor chip, 221 input I / F, 222 generator, 223 output I / F, 901 bus, 902 CPU, 903 ROM, 904 RAM, 905 hard disk, 906 output section, 907 input section, 908 communication section, 909 drive, 910 input / output interface, 911 removable recording medium

Claims

1. an encoding control unit that generates file data obtained by encoding an image from a single image and the file data is stored in each of a plurality of files; The first file data stored in one of the plurality of files and the second file data stored in the other file are data having at least one of different chroma formats and bit depths. Data processing device.

2. The same image has a size, chroma format, and bit depth equal to or greater than the image whose file data is stored in the file.

2. A data processing apparatus according to claim 1.

3. Set one of the chroma format and the bit depth of the file data according to a user's designation.

2. A data processing apparatus according to claim 1.

4. The one file or the other file is a RAW file in which a RAW image is stored.

2. A data processing apparatus according to claim 1.

5. the encoding control unit is configured with one semiconductor chip, The one semiconductor chip includes: a first generating unit configured to generate one of the first file data and the second file data; a second generating unit for generating the other file data; have 2. A data processing apparatus according to claim 1.

6. The encoding control unit is composed of two semiconductor chips, one of the two semiconductor chips generates one of the first file data and the second file data; The other semiconductor chip generates the other file data.

2. A data processing apparatus according to claim 1.

7. the one semiconductor chip receives the same image, generates file data of the one semiconductor chip from the same image, and transfers the same image to the other semiconductor chip; The other semiconductor chip generates the other file data from the same image from the one semiconductor chip.

7. A data processing apparatus according to claim 6.

8. The first file data is data obtained by encoding an image using HEVC, The second file data is JPEG-encoded image data.

2. A data processing apparatus according to claim 1.

9. generating file data encoding the image from the same image, the file data being stored in each of a plurality of files; The first file data stored in one of the plurality of files and the second file data stored in the other file are data having at least one of different chroma formats and bit depths. Data processing methods.

10. A program for causing a computer to function as an encoding control unit that generates file data of an encoded image, the file data being stored in each of a plurality of files from a single image; The first file data stored in one of the plurality of files and the second file data stored in the other file are data having at least one of different chroma formats and bit depths. program.

11. an encoding control unit that generates file data obtained by encoding an image from a single image, the file data being stored in a plurality of files including at least a High Efficiency Image File Format (HEIF) file; The first file data stored in the HEIF file among the plurality of files and the second file data stored in another file other than the HEIF file are data that differ in at least one of chroma format, bit depth, and image size. Data processing device.

12. The same image has a size, chroma format, and bit depth equal to or greater than the image whose file data is stored in the file. A data processing apparatus according to claim 11.

13. Set one or more of the chroma format, bit depth, and image size of the file data according to a user's designation. A data processing apparatus according to claim 11.

14. The other file is a RAW file in which a RAW image is stored. A data processing apparatus according to claim 11.

15. the encoding control unit is configured with one semiconductor chip, The one semiconductor chip includes: a first generating unit configured to generate one of the first file data and the second file data; a second generating unit for generating the other file data; have A data processing apparatus according to claim 11.

16. The encoding control unit is composed of two semiconductor chips, one of the two semiconductor chips generates one of the first file data and the second file data; The other semiconductor chip generates the other file data. A data processing apparatus according to claim 11.

17. the one semiconductor chip receives the same image, generates file data of the one semiconductor chip from the same image, and transfers the same image to the other semiconductor chip; The other semiconductor chip generates the other file data from the same image from the one semiconductor chip.

17. A data processing apparatus according to claim 16.

18. The first file data is data obtained by encoding an image using HEVC, The second file data is JPEG-encoded image data. A data processing apparatus according to claim 11.

19. generating file data encoding the image from the same image, the file data being stored in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file; The first file data stored in the HEIF file among the plurality of files and the second file data stored in another file other than the HEIF file are data that differ in at least one of chroma format, bit depth, and image size. Data processing methods.

20. A program for causing a computer to function as an encoding control unit that generates file data of encoded images from a single image, the file data being stored in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file; The first file data stored in the HEIF file among the plurality of files and the second file data stored in another file other than the HEIF file are data that differ in at least one of chroma format, bit depth, and image size. program.

Images