Data processing device, data processing method and program

Abstract

The present technology relates to a data processing device, a data processing method, and a program that can easily perform the simultaneous generation of multiple files containing file streams having different data formats. File data is generated based on the same image, obtained by encoding the image and stored in each of the multiple files. First file data stored in one of the multiple files and second file data stored in another of the multiple files differ in at least one of codec, chroma format, or bit depth. This technology can be applied to the generation of multiple files containing the same image content.

Description

Technical Field

[0001] The present technology relates to a data processing device, a data processing method, and a program, and more particularly to a data processing device, a data processing method, and a program capable of easily performing simultaneous generation of a plurality of files in which, for example, file streams having different data formats are stored. Background Art

[0002] As a file format for efficiently storing images, there is High Efficiency Image File Format (HEIF) (see Non-Patent Literature 1).

[0003] Reference List

[0004] Non-patent literature

[0005] Non-Patent Document 1: ISO / IEC 23008-12:2017, Information technology — Efficient coding and media delivery in heterogeneous environments — Part 12: Image file formats Summary of the Invention

[0006] Problems to be solved by the present invention

[0007] A stream of images stored in a file is called a file stream.

[0008] For HEIF files and files storing other images, it is convenient if multiple files having file streams with different data formats (such as image dimensions) stored therein can be generated simultaneously for images with the same content.

[0009] The present technology has been made in view of such circumstances, and can easily perform simultaneous generation of a plurality of files in which file streams having different data formats are stored.

[0010] Solution to the problem

[0011] A first data processing device or a first program of the present technology is: a data processing device including an encoding control unit that generates, based on the same image, file data that is obtained by encoding an image and is to be stored in each of a plurality of files, wherein first file data to be stored in one of the plurality of files and second file data to be stored in another file are data that are different from each other in at least one of a codec, a chroma format, or a bit depth; or a program for causing a computer to function as such a data processing device.

[0012] A first data processing method of the present technology is a data processing method including generating file data obtained by encoding an image and to be stored in each of a plurality of files based on the same image, wherein first file data to be stored in one file of the plurality of files and second file data to be stored in another file are data different from each other in at least one of a codec, a chroma format, or a bit depth.

[0013] In a first data processing device, a first data processing method, and a first program of the present technology, file data obtained by encoding an image and to be stored in each of a plurality of files is generated based on the same image. First file data stored in one file of the plurality of files and second file data stored in another file are data that differ from each other in at least one of codec, chroma format, or bit depth.

[0014] A second data processing device or a second program of the present technology is: a data processing device, an encoding control unit, which generates file data obtained by encoding an image and to be stored in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file based on the same image, wherein first file data to be stored in a HEIF file among the plurality of files and second file data to be stored in another file other than the HEIF file are data different from each other in at least one of a codec, a chroma format, a bit depth, or an image size; or a program for causing a computer to function as such a data processing device.

[0015] A second data processing method of the present technology is a data processing method, comprising generating file data in each of a plurality of files obtained by encoding an image and to be stored in at least a High Efficiency Image File Format (HEIF) file based on the same image, wherein first file data to be stored in a HEIF file among the plurality of files and second file data to be stored in another file other than the HEIF file are data that are different from each other in at least one of a codec, a chroma format, a bit depth, or an image size.

[0016] In a second data processing device, a second data processing method, and a second program of the present technology, file data is generated in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file, obtained by encoding an image, based on the same image. First file data stored in a HEIF file among the plurality of files and second file data stored in another file other than the HEIF file are data that differ from each other in at least one of a codec, a chroma format, a bit depth, or an image size.

[0017] Note that the data processing device may be an independent device or an internal block included in a device.

[0018] In addition, the program can be provided by recording the program on a recording medium or by transmitting the program via a transmission medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a block diagram showing a configuration example of an embodiment of a digital camera to which the present technology is applied.

[0020] Figure 2 is a diagram showing a format example of a JPEG file conforming to the Joint Photographic Experts Group (JPEG).

[0021] Figure 3 is a diagram showing an example of the ISO base media file format.

[0022] Figure 4 is a diagram showing a format example of a HEIF file conforming to HEIF.

[0023] Figure 5 FIG2 is a diagram showing a format example of a HEIF file in the image project format.

[0024] Figure 6 is a diagram showing an example of an iprp box.

[0025] Figure 7 FIG2 is a diagram showing a format example of a HEIF file in the image sequence format.

[0026] Figure 8 is a diagram showing an example of a trak box.

[0027] Figure 9 is a diagram showing an example of a collection file in which a main image and thumbnail images are stored.

[0028] Figure 10 is a diagram showing an example of a sequence file in which a track of a main image and a track of thumbnail images of the main image are stored.

[0029] Figure 11 This is a diagram explaining the outline of generating a file stream to be stored in a file in the digital camera 10 .

[0030] Figure 12 1 is a diagram illustrating a first example of generating a plurality of files of the same image content in the digital camera 10 .

[0031] Figure 13 FIG. 1 is a diagram illustrating a second example of generating a plurality of files of the same image content in the digital camera 10 .

[0032] Figure 14This is a flowchart illustrating a process of generating a plurality of files of the same image content in the digital camera 10 .

[0033] Figure 15 : is a block diagram showing a first configuration example of the encoding control unit 42 .

[0034] Figure 16 1 is a diagram illustrating an example of generating a HEIF file and a JPEG file having the same image content when the encoding control unit 42 includes a single chip, the semiconductor chip 110 .

[0035] Figure 17 1 is a diagram illustrating another example of generating a HEIF file and a JPEG file having the same image content when the encoding control unit 42 includes a single chip, the semiconductor chip 110 .

[0036] Figure 18 This is a diagram illustrating an example of generating a HEIF file and an ARW file having the same image content when the encoding control unit 42 includes a single chip, the semiconductor chip 110 .

[0037] Figure 19 1 is a diagram illustrating an example of generating two HEIF files having the same image content when the encoding control unit 42 includes a single chip, the semiconductor chip 110 .

[0038] Figure 20 1 is a diagram illustrating an example of generating two JPEG files having the same image content when the encoding control unit 42 includes a single chip, the semiconductor chip 110 .

[0039] Figure 21 : is a block diagram showing a second configuration example of the encoding control unit 42 .

[0040] Figure 22 1 is a diagram illustrating an example of generating a HEIF file and a JPEG file having the same image content when the encoding control unit 42 includes two semiconductor chips 210 and 220 .

[0041] Figure 23 1 is a diagram illustrating another example of generating a HEIF file and a JPEG file having the same image content when the encoding control unit 42 includes two semiconductor chips 210 and 220 .

[0042] Figure 24 is a block diagram illustrating a configuration example of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION

[0043] <Embodiment of a digital camera to which the present technology is applied>

[0044] Figure 1 is a block diagram showing a configuration example of an embodiment of a digital camera to which the data processing apparatus of the present technology is applied.

[0045] The digital camera 10 includes an optical system 11 , an image sensor 12 , a signal processing unit 13 , a medium 14 , interfaces (I / F) 15 and 16 , buttons / keys 17 , a touch panel 18 , a liquid crystal panel 19 , a viewfinder 20 , an I / F 21 , and the like.

[0046] The optical system 11 focuses light from a subject onto the image sensor 12 .

[0047] The image sensor 12 generates image data as an electric signal by receiving light from the optical system 11 and performing imaging using photoelectric conversion, and supplies the image data to the signal processing unit 13 .

[0048] The signal processing unit 13 includes an optical system / image sensor control unit 41 , an encoding control unit 42 , a file control unit 43 , a medium control unit 44 , an operation control unit 45 , a display control unit 46 , and a UI control unit 47 .

[0049] The optical system / image sensor control unit 41 controls the optical system 11 and the image sensor 12 , and supplies (data of) an image obtained by imaging according to the control to the encoding control unit 42 .

[0050] The encoding control unit 42 supplies the image from the optical system / image sensor control unit 41 to the display control unit 46, encodes the image as needed, and supplies the encoded image to the file control unit 43. In addition, the encoding control unit 42 decodes the image supplied from the file control unit 43 as needed, and supplies the decoded image to the display control unit 46.

[0051] The file control unit 43 generates a file storing the image supplied from the encoding control unit 42, and supplies the file to the medium control unit 44. In addition, the file control unit 43 reproduces the file supplied from the medium control unit 44, that is, reads data such as an image stored in the file. For example, the image read from the file is supplied from the file control unit 43 to the encoding control unit 42.

[0052] The medium control unit 44 controls file exchange between the medium 14 and the I / Fs 15 and 16. For example, the medium control unit 44 causes a file from the file control unit 43 to be recorded on the medium 14 or transferred from the I / Fs 15 and 16. In addition, the medium control unit 44 reads a file from the medium 14 or causes the I / Fs 15 and 16 to receive a file and supply the file to the file control unit 43.

[0053] The operation control unit 45 supplies an operation signal corresponding to the operation to a necessary block according to the user's operation of the button / key 17 or the touch panel 18 .

[0054] The display control unit 46 performs display control and the like to supply the image and the like supplied from the encoding control unit 42 to the liquid crystal panel 19 , the viewfinder 20 , and the I / F 21 to display them.

[0055] The UI control unit 47 manages user interface (UI) control.

[0056] The medium 14 is a storage medium such as an SD card. The I / F 15 is an I / F for a local area network (LAN) (for example, WiFi (registered trademark), Ethernet (registered trademark), etc.). The I / F 16 is an I / F such as a universal serial bus (USB). When a command or other information is input to the digital camera 10, the button / key 17 and the touch panel 18 are operated by the user. The touch panel 18 can be formed integrally with the liquid crystal panel 19. The liquid crystal panel 19 and the viewfinder 20 display images supplied from the display control unit 46, etc. The I / F 21 is an I / F for transmitting at least an image, such as a high-definition multimedia interface (HDMI (registered trademark)) or a display port (DP).

[0057] In the digital camera 10 configured as described above, the optical system / image sensor control unit 41 generates a YUV image having, for example, the same resolution (number of pixels) (size) as that of the RAW image from a RAW data image (hereinafter, also referred to as a RAW image) obtained by imaging by the image sensor 12, and supplies the YUV image together with the RAW image to the encoding control unit 42. The encoding control unit 42 generates a main image of a HEIF file or the like from the YUV image from the optical system / image sensor control unit 41. For example, the YUV image from the optical system / image sensor control unit 41 can be used as the main image of the HEIF file as it is.

[0058] The encoding control unit 42 generates, from the YUV main image, a YUV image (hereinafter also referred to as a screen thumbnail) having, for example, a lower resolution than the main image as a first other image based on the main image, for display on the liquid crystal panel 19 or an external display. It also generates, for example, a YUV image (hereinafter also referred to as a thumbnail) having, for example, a lower resolution than the screen thumbnail as a second other image based on the main image, for use in an index display (list display). For example, the encoding control unit 42 supplies the screen thumbnail to the liquid crystal panel 19 via the display control unit 46 to display the screen thumbnail as a so-called through image. For example, an image with a long side dimension of 320 pixels or less can be used as the thumbnail. The size (pixel count) ratio between the main image and the screen thumbnail serving as the first other image based on the main image, or the thumbnail serving as the second other image based on the main image, can be, for example, 200 times or less. Similarly, the size ratio between the screen thumbnail serving as the first other image based on the main image and the thumbnail serving as the second other image based on the main image can also be 200 times or less. For example, an image with a resolution of 4K or greater can be used as the thumbnail. In addition, as a screen thumbnail, for example, a 4K (QFHD) or FHD image can be used according to the user's selection. In addition, an image with the same resolution can be used as the main image and the screen thumbnail. In the case of using an image with the same resolution as the main image and the screen thumbnail, both the main image and the screen thumbnail can be stored in the HEIF file, or the main image can be stored without storing the screen thumbnail. In the case of storing the main image in the HEIF file without storing the screen thumbnail, the size of the main image can be adjusted and used as the screen thumbnail.

[0059] In addition, the encoding control unit 42 encodes the main image, screen thumbnail, and thumbnail corresponding to the RAW image (main image, screen thumbnail, and thumbnail generated from the same RAW image) as needed, and supplies them to the file control unit 43 together with the RAW image.

[0060] The file control unit 43 generates a RAW file in which a RAW image is stored, a HEIF file and / or a JPEG file in which corresponding main images, screen thumbnails, and thumbnails (main images, screen thumbnails, and thumbnails generated from the same RAW image) are stored, and the like as needed, and supplies the generated files to the media control unit 44. The HEIF file is a file compliant with the High Efficiency Image File Format (HEIF), and the JPEG file is a file compliant with the Joint Photographic Experts Group (JPEG).

[0061] The media control unit 44 records the RAW file, HEIF file, or JPEG file from the file control unit 43 on the medium 14, or transmits the RAW file, HEIF file, or JPEG file from the I / F 15 or 16.

[0062] The type of file to be generated in the file control unit 43 (e.g., RAW file, HEIF file, JPEG file, etc.) can be selected according to, for example, the operation (specification) of the user. In addition, as will be described later, the HEIF file includes an image item format and an image sequence format, and for example, which of the image item format and the image sequence format to adopt can be selected according to the operation of the user. Furthermore, the file control unit 43 can perform mutual conversion between the HEIF file and the JPEG file according to the operation of the user.

[0063] In addition, the file control unit 43 can generate multiple files having the same image content and different from each other in terms of codec, image size (resolution), color format, or bit depth.

[0064] When the file control unit 43 generates multiple files having the same image content, the encoding control unit 42 generates an image stream (file stream) to be stored in the corresponding multiple files based on the YUV image from the optical system / image sensor control unit 41.

[0065] The encoding control unit 42 can generate image streams different from each other in terms of codec, image size (resolution), color format, or bit depth.

[0066] For example, the encoding control unit 42 can generate an image of a predetermined size, a predetermined color format, and a predetermined bit depth based on the YUV image supplied from the optical system / image sensor control unit 41, and generate a first stream obtained by encoding the image using a predetermined codec (encoding method). In addition, the encoding control unit 42 can generate an image of another size, another color format, and another bit depth based on the same YUV image supplied from the optical system / image sensor control unit 41, and generate a second stream obtained by encoding the image using another codec.

[0067] Then, the file control unit 43 can generate a file storing the first stream and a file storing the second stream.

[0068] <JPEG file>

[0069] Figure 2 is a diagram showing a format example of a JPEG file conforming to the Joint Photographic Experts Group (JPEG).

[0070] The JPEG file is configured to store, for example, Exif metadata, a thumbnail, Extensible Metadata Platform (XMP) (registered trademark) metadata, MPF indicating the storage location (position) of the main image and the simple display image, the main image, and the simple display image. As the simple display image, for example, a screen thumbnail can be adopted.

[0071] <ISO Base Media File Format>

[0072] Figure 3 is a diagram showing an example of the ISO Base Media File Format.

[0073] HEIF (ISO / IEC 23008-12) is a file format compliant with the ISO Base Media File Format (ISO / IEC 14496-12), and thus, the HEIF file complies with the ISO Base Media File Format.

[0074] The ISO Base Media File Format includes units called boxes as containers for storing data and has a structure called a box structure.

[0075] A box includes a type (box type), actual data (data), etc. The type indicates the type of the actual data in the box. As the actual data, reproducible media data such as images (still images, moving images), audio, and subtitles, an attribute name (field name) and an attribute value (field value) of the attribute name (the represented variable), and various other data can be adopted.

[0076] In addition, a box can be adopted as the actual data. That is, a box can have a box as the actual data, and thus can have a hierarchical structure.

[0077] A basic media file compliant with the ISO Base Media File Format can include a ftyp box, a moov box (MovieBox), a meta box (MetaBox), a mdat box (MediaDataBox), etc. In the ftyp box, identification information for identifying the file format is stored. The moov box can store trak boxes, etc. The meta box can store iinf boxes, iprp boxes, iref boxes, iloc boxes, etc. The mdat box can store media data (AV data) and any other data.

[0078] HEIF complies with the ISO Base Media File Format as described above.

[0079] <HEIF File>

[0080] Figure 4 is a diagram showing an example of the format of a HEIF file compliant with HEIF.

[0081] HEIF files are roughly divided into image item formats and image sequence formats. In addition, image item formats include single image formats with only one item described later and image collection formats with multiple items.

[0082] A HEIF file in the image project format includes an ftyp box, a meta box, and an mdat box.

[0083] The HEIF file in the image sequence format includes an ftyp box, a moov box, and an mdat box.

[0084] Note that a HEIF file may include not only one of the meta box and the moov box, but both.

[0085] The ftyp box stores identification information for identifying the file format, for example, whether the file is a HEIF file in an image project format or an image sequence format.

[0086] In the meta box and the moov box, metadata required for reproduction, management, etc. of the media data stored in the mdat box, for example, metadata such as the storage location of the media data, is stored.

[0087] In the mdat box, media data (AV data) and the like are stored.

[0088] In the digital camera 10, for example, it is possible to select which HEIF file to generate between a HEIF file in an image project format and a HEIF file in an image sequence format according to a user operation. In addition, in the case of encoding an image and storing it in the mdat box of a HEIF file, the image project format only allows intra-frame coding, and the image sequence format allows intra-frame coding and inter-frame coding. Therefore, for example, in the case of giving priority to high-speed access to data stored in a HEIF file, it is possible to select the generation of a HEIF file in an image project format, and in the case of giving priority to reducing the size (data amount) of a HEIF file, it is possible to select the generation of a HEIF file in an image sequence format.

[0089] Figure 5 FIG2 is a diagram showing a format example of a HEIF file in the image project format.

[0090] In a HEIF file in the image item format, information indicating that the HEIF file is in the image item format (for example, mif1, etc.) is stored (as an attribute value) in the ftyp box.

[0091] In the meta box, the iinf box, iref box, iprp box, and iloc box are stored.

[0092] The iinf box stores the number of items (attribute names and values ​​indicating the number) of media data (AV data) stored in the mdat box. An item is a single piece of data stored in the mdat box of a HEIF file in the image item format. For example, a single (screen) image is an item. In this manual, regardless of whether it is a still image or a moving image, a single image is also referred to as a frame. A single frame is an item.

[0093] The iref box stores information indicating the relationship between items. For example, in the mdat box, each of the corresponding main image, screen thumbnail, and thumbnail image can be stored as an item. If item I1 is the main image, item I2 is the screen thumbnail, and item I3 is the thumbnail, the iref box stores information indicating that item I2 is the screen thumbnail of item I1, the main image, and the iref box stores information indicating that item I3 is the thumbnail of item I1.

[0094] In the iprp box, information related to the attributes of the item is stored.

[0095] In the iloc box, information on the storage location of the item stored in the mdat box is stored.

[0096] In the mdat box of the image project format (HEIF file), for example, frames of images as projects are stored. One or more projects can be stored in the mdat box. In addition, the frames as projects can be encoded and stored in the mdat box. However, the encoding of the frames as projects stored in the mdat box of the image project format is limited to intra-frame encoding. As an encoding method (codec) for encoding the frames as projects, for example, HEVC etc. can be used.

[0097] Figure 6 It shows Figure 5 Figure 1 shows an example of an iprp box in Figure 1.

[0098] The iprp box stores the ipco box and ipma box, which are related to the attributes of the item. The ipco box stores the attributes of the item stored in the mdat box, such as codec information related to the codec of the image being the item and image size information regarding the size. The ipma box stores an index (pointer) from the item stored in the mdat box to the attributes stored in the ipco box.

[0099] Figure 7 FIG2 is a diagram showing a format example of a HEIF file in the image sequence format.

[0100] In a HEIF file in an image sequence format, information indicating that the HEIF file is in an image sequence format, such as msf1, is stored in the ftyp box.

[0101] The moov box stores a trak box, and the trak box stores information related to the track stored in the mdat box.

[0102] A track includes independent media data, such as images or audio, that is reproduced according to a timeline. For example, a track includes one or more image frames that will become elementary streams. For tracks stored in an mdat box, multiple tracks can be reproduced simultaneously, for example, tracks for images and audio that are recorded simultaneously.

[0103] The media data of a track is divided into units called samples. A sample is the smallest unit (access unit) for accessing media data in a HEIF file. Therefore, media data in a HEIF file cannot be accessed in units finer than a sample.

[0104] For image media data, for example, one frame is one sample. Also, for audio media data, for example, one audio frame defined in the standard of audio media data is one sample.

[0105] In the mdat box of the image sequence format (HEIF file), the media data of the track is arranged in units called chunks. A chunk is a group of one or more samples arranged at logically consecutive addresses.

[0106] In the case where a plurality of tracks as media data are stored in the mdat box, the plurality of tracks are interleaved and arranged in units of blocks.

[0107] As described above, in the mdat box of the image sequence format, one or more tracks including media data such as images or audio are stored.

[0108] In the mdat box, the image frames constituting the track can be encoded and stored. When encoding the frames constituting the track stored in the mdat box in the image sequence format, a long GOP can be used as the group of pictures (GOP), and both intra-frame coding and inter-frame coding can be used. As the codec used to encode the frames constituting the track, for example, HEVC can be used.

[0109] Figure 8 is a diagram showing an example of a trak box.

[0110] In the trak box, a tkhd box and an mdia box can be stored. In the tkhd box, header information of the track, such as the creation date and time of the track managed by the trak box, is stored. In the mdia box, a minf box and the like are stored. In the minf box, an stbl box is stored. In the stbl box, a stsd box, a stsc box, a stsz box, and a stco box are stored, in which samples of the track are stored, and thus information for accessing blocks is stored. In the stsd box, codec information related to the codec of the track is stored. In the stsc box, a block size (the number of samples in one block) is stored. In the stsz box, a sample size is stored. In the stco box, a block offset, that is, an offset of the arrangement position of each block of the track stored in the mdat box, is stored.

[0111] Here, the HEIF file in the image project format is also referred to as a collection file, and the HEIF file in the image sequence format is also referred to as a sequence file.

[0112] In the digital camera 10 , a HEIF file can be generated in which a main image and one or both of additional necessary screen thumbnails and thumbnails are stored.

[0113] <Collection File>

[0114] Figure 9 is a diagram showing an example of a collection file in which a main image and thumbnail images are stored.

[0115] Now, assume that the frames (items) are encoded by HEVC and stored in the mdat box of the collection file.

[0116] In the ftyp box, as identification information for identifying the file format, heic is stored, indicating that the file format is the image item format and the codec is HEVC.

[0117] In the iinf box, the number of items stored in the mdat box is stored. Figure 9 In the mdat box, a total of four items (frames) are stored: a main image identified by item ID #1 (hereinafter, also described as main image item #1), main image item #2, a thumbnail identified by item ID #101 (hereinafter, also referred to as thumbnail item #101), and thumbnail item #102. Therefore, the number of items is four. Note that thumbnail item #101 is a thumbnail of main image item #1, and thumbnail item #102 is a thumbnail of main image item #2.

[0118] In the iinf box, for example, for each item stored in the mdat box, an infe box is also stored. In the infe box, an item ID and an item type for identifying an item are registered. Figure 9, each of the main image item #1 and item #2 and the thumbnail item #101 and item #102 has an info box.

[0119] In the iref box, for example, the thmb box is stored as information for associating the items stored in the mdat box. In the thmb box, a reference source and a reference destination, which are information for associating a main image with a thumbnail of the main image, are stored in association with each other. In the thmb box, the reference source represents the item ID of the main image, and the reference destination represents the item ID of the thumbnail of the main image identified by the item ID of the reference source. Therefore, using the reference destination associated with the reference source, the item ID of the thumbnail of the main image identified by the item ID represented by the reference source can be identified. In addition, using the reference source associated with the reference destination, the item ID of the main image of the thumbnail identified by the item ID represented by the reference destination can be identified.

[0120] In the iprp box, such as Figure 6 As described above, store the ipco box and ipma box. In the ipco box, such as Figure 6 As described above, attributes of the frame as items stored in the mdat box, such as codec information about codec and image size information about size, are stored. In the ipma box, as Figure 6 As described above, the indexes of the items stored in the mdat box to the attributes stored in the ipco box are stored.

[0121] In the iloc box, such as Figure 6 As described above, information related to the storage location of items in the mdat box is stored. Figure 9 In the iloc box, the number of items is stored as 4. In addition, in the iloc box, the offset and size of the storage location of each of the main image item #1 and item #2 and the thumbnail item #101 and item #102 stored in the mdat box are stored in association with the item ID.

[0122] <sequence file>

[0123] Figure 10 is a diagram showing an example of a sequence file in which a track of a main image and a track of thumbnails of the main image are stored.

[0124] Now, assume that the frames are encoded by HEVC and stored in the mdat box of the sequence file.

[0125] In the ftyp box, as identification information for identifying the file format, hevc is stored, indicating that the file format is an image sequence format and the codec is HEVC.

[0126] In the moov box, such as Figure 7As described above, a trak box for managing each track stored in the mdat box is stored. Figure 10 In the .mdat box, a track of the main image identified by track ID #1 (hereinafter also referred to as track #1) and track #2, which is a thumbnail of the main image of track #1, are stored. Therefore, the moov box stores a trak box for managing track #1 and a trak box for managing track #2. The nth thumbnail (frame) of track #2 (from the beginning) is a thumbnail of the nth main image of track #1.

[0127] For example, in the case of continuous shooting by the digital camera 10 , a sequence file is useful in the case where main images and thumbnails of a plurality of frames obtained by the continuous shooting are each recorded as one track or the like.

[0128] The tkhd box of the trak box for managing track #1 for main images stores track ID #1 for identifying track #1, the image size of the main image constituting track #1, rotation information indicating the orientation of the digital camera 10 when capturing the main image, and the creation date and time of track #1. The tkhd box of the trak box for managing track #2 for thumbnail images stores track ID #2 for identifying track #2, and the creation date and time of track #2.

[0129] In the trak box, in addition to Figure 7 In addition to the tkhd box and mdia box described in

[15] , a tref box may be stored. In the tref box, a track ID for identifying another track associated with the track managed by the trak box in which the tref box is stored, information indicating the track content, and the like are stored. Figure 10 In the example, the tref box is set in the trak box for managing track #2. Then, in the tref box, information indicating that another track associated with track #2 is track #1 (track_ID=1) and that data constituting track #2 is a thumbnail (track #2 is a track of thumbnails) (type=thmb) is stored.

[0130] In the mdia box of the trak box, except Figure 8 In addition to the minf box described in [ ], an hdlr box can also be stored. In the hdlr box, information indicating the type of data constituting the track managed by the trak box in which the hdlr box is stored is stored. In the hdlr box stored in the trak box for managing track #1 for main images (stored in the mdia box), information (pict) indicating that the data constituting track #1 is a picture (frame) is stored, and in the hdlr box stored in the trak box for managing track #2 for thumbnail images, information indicating that the data constituting track #2 is a picture is stored.

[0131] minf box Figure 8 describe.

[0132] <File Stream Generation>

[0133] Figure 11 This is a diagram explaining the outline of generating a file stream to be stored in a file in the digital camera 10 .

[0134] In the digital camera 10, the encoding control unit 42 generates a stream (file data) (hereinafter also referred to as a file stream) to be stored in a file to be generated by the file control unit 43 based on the RAW image and YUV image (YUV (YCbCr) (YPbPr) image) supplied from the optical system / image sensor control unit 41.

[0135] The encoding control unit 42 supplies the RAW image (data stream) from the optical system / image sensor control unit 41 as a RAW stream (elementary stream (ES)) as it is, for example, as needed, to the file control unit 43. Furthermore, the encoding control unit 42 performs JPEG encoding on the YUV image (YUV image) from the optical system / image sensor control unit 41 as needed, for example, and supplies the resulting JPEG stream (ES) to the file control unit 43. Furthermore, the encoding control unit 42 performs HEVC encoding on the YUV image from the optical system / image sensor control unit 41 as needed, for example, and supplies the resulting HEVC stream (ES) to the file control unit 43.

[0136] In the case of generating a RAW file, the file control unit 43 generates a RAW file that stores the RAW stream and the JPEG stream from the encoding control unit 42. Note that at least the RAW stream needs to be stored in the RAW file, and the JPEG stream does not necessarily need to be stored in the RAW file. However, in the case where the JPEG stream is stored in the RAW file, the image can be easily displayed by using the JPEG stream. That is, there is a case where you want to display an image to confirm which image stream is stored in the RAW file. In this case, when the RAW stream is stored but the JPEG stream is not stored in the RAW file, it is necessary to display the image by using the RAW stream stored in the RAW file. In the case of displaying the image by using the RAW stream, development processing of the RAW stream is required. On the other hand, when the JPEG stream is stored in the RAW file, the image can be easily displayed by using the JPEG stream without performing development processing. Hereinafter, a RAW file in which a JPEG stream is stored in addition to the RAW stream is also referred to as an ARW file.

[0137] When generating a JPEG file, the file control unit 43 generates a JPEG file storing the JPEG stream from the encoding control unit 42 .

[0138] In addition, when generating a HEIF file, the file control unit 43 generates a HEIF file storing the HEVC stream from the encoding control unit 42 .

[0139] <File Generation>

[0140] Figure 12 1 is a diagram illustrating a first example of generating a plurality of files of the same image content in the digital camera 10 .

[0141] Figure 12 An example is shown in which an ARW file and a JPEG file are generated as a plurality of files.

[0142] When the digital camera 10 generates an ARW file and a JPEG file, the encoding control unit 42 supplies the RAW image supplied from the optical system / image sensor control unit 41 to the file control unit 43 as a RAW stream to be stored in the ARW file. Furthermore, the encoding control unit 42 generates a JPEG stream as a file stream to be stored in the ARW file and the JPEG file based on the YUV image supplied from the optical system / image sensor control unit 41, and supplies the generated JPEG stream to the file control unit 43.

[0143] The file control unit 43 generates an ARW file storing the RAW stream and JPEG stream from the encoding control unit 42. The file control unit 43 also generates a JPEG file storing the JPEG stream from the encoding control unit 42. The file control unit 43 supplies the ARW file and JPEG file to the medium control unit 44.

[0144] Note that in addition to the original image to be stored in the file (i.e., the image corresponding to the main image to be stored in the HEIF file, hereinafter also referred to as the main image, similar to the case of the HEIF file), the file to be generated by the file control unit 43 can also store screen thumbnails and thumbnails.

[0145] For example, in an ARW file and a JPEG file, a JPEG stream of a screen thumbnail smaller in size (resolution) than the main image and a JPEG stream of a thumbnail smaller in size than the screen thumbnail can be stored together with the JPEG stream of the main image.

[0146] Hereinafter, it is assumed that file streams such as a main image, a screen thumbnail, and a thumbnail are stored in a file to be generated by the file control unit 43 .

[0147] For a JPEG stream, when the main image size is a predetermined size larger than FHD, for example, the screen thumbnail size is FHD, and the thumbnail size is, for example, 160×120 (horizontal×vertical). In addition, for a JPEG stream, the chroma format is, for example, 422 (4:2:2), and the bit depth (the number of bits expressing a single pixel component) is, for example, 8 bits.

[0148] Here, the size of the main image, the size of the screen thumbnail, and the size of the thumbnail are also referred to as the main size, the screen thumbnail size, and the thumbnail size, respectively.

[0149] In addition, assuming that for a file stream of an image, the data format of the file stream includes the codec, size, chroma format, and bit depth of the image.

[0150] When the ARW file and the JPEG file are generated in the file control unit 43 , the encoding control unit 42 generates a main image, a screen thumbnail, and a JPEG stream of thumbnails from the YUV image supplied from the optical system / image sensor control unit 41 .

[0151] The image (here, a YUV image) supplied from the optical system / image sensor control unit 41 to the file control unit 43 (which is used to generate a file stream to be stored in a file to be generated by the file control unit 43) is also called a raw image.

[0152] For example, in the case where the main image, screen thumbnail and JPEG stream of thumbnails to be stored in the ARW file and the main image, screen thumbnail and JPEG stream of thumbnails to be stored in the JPEG file are respectively file streams of the same data format, the encoding control unit 42 generates three data format file streams of the main image, screen thumbnail and JPEG stream of thumbnails that differ from each other only in the size of the data format based on the original image.

[0153] In this case, the number of YUV converters required by the encoding control unit 42 is not so large. The YUV converter is a converter that converts the size, chroma format, or bit depth of a YUV image as an original image.

[0154] Note that in Figure 12 In the example above, the file control unit 43 generates an ARW file and a JPEG file as multiple files, but the multiple files generated by the file control unit 43 are not limited to this. For example, the file control unit 43 may generate two JPEG files with different main image sizes. Alternatively, for example, the file control unit 43 may generate four files: two JPEG files with different main image sizes and two ARW files with different main image sizes.

[0155] Figure 13FIG. 1 is a diagram illustrating a second example of generating a plurality of files of the same image content in the digital camera 10 .

[0156] Figure 13 An example is shown in which a HEIF file and a JPEG file are generated as a plurality of files.

[0157] In this case, the encoding control unit 42 generates a YUV image similar to the original image. Figure 12 , and supplies the JPEG stream to the file control unit 43. In addition, the encoding control unit 42 generates an HEVC stream as a file stream to be stored in the HEIF file from the YUV image as the original image, and supplies the HEVC stream to the file control unit 43.

[0158] The file control unit 43 generates a JPEG file storing the JPEG stream from the encoding control unit 42. Furthermore, the file control unit 43 generates a HEIF file storing the HEVC stream from the encoding control unit 42. The file control unit 43 supplies the JPEG file and the HEIF file to the media control unit 44.

[0159] like Figure 12 As described above, in the case where the file streams of the main image, screen thumbnail, and thumbnail are stored in the HEIF file, the encoding control unit 42 needs to generate the HEVC streams of the main image, screen thumbnail, and thumbnail.

[0160] For HEVC streams to be stored in HEIF files, for example, a size larger than OFHD or FHD can be used as the main size. In addition, for example, OFHD or FHD can be used as the screen thumbnail size, and for example, 320×240 can be used as the thumbnail size.

[0161] In addition, for HEVC streams, 422 or 420 may be adopted as the chroma format, and 10 bits or 8 bits may be adopted as the bit depth.

[0162] Therefore, when generating HEIF files and JPEG files, it may be necessary to generate images of various data formats that are different from each other in at least one or more aspects of size, chroma format, or bit depth based on the YUV image as the original image.

[0163] Therefore, with Figure 12 Compared with the case of generating an ARW file and a JPEG file described in , the number of YUV converters required for the encoding control unit 42 increases, and control of the processing performed by the encoding control unit 42 becomes complicated.

[0164] In addition, when generating a HEIF file and a JPEG file, the HEVC stream to be stored in the HEIF file and the JPEG stream to be stored in the JPEG file are different from each other in terms of codecs, so the encoding control unit 42 requires separate codecs to generate the HEVC stream and generate the JPEG stream.

[0165] Note that the increase in the number of YUV converters required for the encoding control unit 42 is similar also in the case where a HEIF file and a JPEG file are generated as multiple files in the file control unit 43, and in addition, for example, in the case where an ARW file indicated by the dotted line in the figure is generated instead of a JPEG file, in the case where two HEIF files having main images having different sizes from each other are generated, in the case where four files—two HEIF files having different main sizes from each other and two JPEG files or two ARW files having different main sizes from each other—are generated, and so on.

[0166] Hereinafter, a method of simultaneously generating a plurality of files having the same image content will be described, in which file streams having different data formats are stored with a configuration as simple as possible.

[0167] Note that in the following, it is assumed that FHD is adopted as the screen thumbnail size of a JPEG stream (an image set as this JPEG stream), 160×120 is adopted as the thumbnail size, 422 is adopted as the chroma format, and 8 bits is adopted as the bit depth.

[0168] Furthermore, it is assumed that OFHD or FHD is adopted as the screen thumbnail size of the HEVC stream, 320×240 is adopted as the thumbnail size, 422 or 420 is adopted as the chroma format, and 10 bits or 8 bits is adopted as the bit depth, respectively.

[0169] However, the screen thumbnail size, thumbnail size, chroma format, and bit depth of the JPEG stream and the HEVC stream are not limited thereto.

[0170] For example, if JPEG is extended in the future, a data format that complies with the extended JPEG may be used. For example, the screen thumbnail size of the JPEG stream may be larger or smaller than FHD, the thumbnail size may be larger or smaller than 160×120, the chroma format may be 444, and the bit depth may be 12 bits.

[0171] In addition, for example, a size different from OFHD or FHD as the screen thumbnail size of the HEVC stream, a size different from 320×240 as the thumbnail size, 444 as the chroma format, and 12 bits as the bit depth may be adopted respectively.

[0172] In addition, here, JPEG and HEVC are used as codecs, but the codec is not limited to them. As the codec, codecs other than JPEG and HEVC, for example, Versatile Video Coding (VVC) and the like can be used.

[0173] <Processing of Generating Multiple Files>

[0174] Figure 14 This is a flowchart illustrating a process of generating a plurality of files of the same image content in the digital camera 10 .

[0175] In step S211 , the encoding control unit 42 receives (acquires) a YUV image as a raw image from the optical system / image sensor control unit 41 , and the process proceeds to step S212 .

[0176] In step S212 , the encoding control unit 42 generates a plurality of file streams having different data formats to be stored in a plurality of files based on the original image.

[0177] That is, the encoding control unit 42 generates a plurality of file streams that are different from each other in at least one of size, chroma format, bit depth, or codec from the original image.

[0178] Specifically, the encoding control unit 42 converts part or all of the size, chroma format, and bit depth of the original image as needed, and generates one or more images of the desired size, chroma format, and bit depth. Alternatively, the encoding control unit 42 may use the original image as is as needed as an image of the desired size, chroma format, and bit depth.

[0179] The encoding control unit 42 also encodes an image of a desired size, color format, and bit depth using a desired encoding method, and generates a plurality of file streams that differ from one another in at least one of size, color format, bit depth, or codec. Note that the encoding control unit 42 can receive a RAW image from the optical system / image sensor control unit 41 as needed and use the RAW image as is as a file stream.

[0180] The encoding control unit 42 supplies the file stream to the file control unit 43, and the process proceeds from step S212 to step S213.

[0181] In step S213 , the file control unit 43 stores the plurality of file streams from the encoding control unit 42 that are different from each other in at least one of size, chroma format, bit depth, or codec in separate files to generate a plurality of files.

[0182] <First Configuration Example of the Encoding Control Unit 42>

[0183] Figure 15: is a block diagram showing a first configuration example of the encoding control unit 42 .

[0184] exist Figure 15 In the embodiment, the encoding control unit 42 includes a one-chip semiconductor chip 110 .

[0185] The semiconductor chip 110 (functionally) includes an input I / F 111 , generation units 112 and 113 , and an output I / F 114 .

[0186] The input I / F 111 receives a signal from the outside and supplies it to a necessary block. For example, the input I / F 111 receives a YUV image as a raw image from the optical system / image sensor control unit 41 and supplies the YUV image to the generation units 112 and 113. Furthermore, for example, the input I / F 111 receives a RAW image from the optical system / image sensor control unit 41 and supplies the RAW image to the output I / F 114.

[0187] The generation unit 112 functions as a first generation unit that generates a first file stream from the original image from the input I / F 111 and supplies the first file stream to the output I / F 114. For example, the generation unit 112 encodes the original image using a first encoding method and supplies the resulting stream as the first file stream to the output I / F 114. Furthermore, for example, the generation unit 112 converts one or more of the size, chroma format, or bit depth of the original image to generate a first converted image. The generation unit 112 then encodes the first converted image using the first encoding method and supplies the resulting stream as the first file stream to the output I / F 114.

[0188] The generation unit 113 functions as a second generation unit that generates a second file stream from the original image from the input I / F 111 and supplies the second file stream to the output I / F 114. For example, the generation unit 113 encodes the original image using the second encoding method and supplies the resulting stream as the second file stream to the output I / F 114. Alternatively, for example, the generation unit 112 converts one or more of the size, chroma format, or bit depth of the original image to generate a second converted image. The generation unit 112 then encodes the second converted image using the second encoding method and supplies the resulting stream as the second file stream to the output I / F 114.

[0189] The output I / F 114 outputs (supplies) signals to the outside. For example, the output I / F 114 supplies the first file stream supplied from the generation unit 112 and the second file stream supplied from the generation unit 113 to the file control unit 43. Furthermore, for example, the output I / F 114 supplies the RAW image from the input I / F 111 to the file control unit 43 as (a portion of) the first file stream or the second file stream.

[0190] In the file control unit 43 , a file in which the first file stream is stored and a file in which the second file stream is stored are generated.

[0191] As the first file stream and the second file stream, file streams of the same or different data formats can be used. When file streams with different data formats are used as the first file stream and the second file stream, the first file stream and the second file stream differ from each other in one or two or more aspects of size, chroma format, bit depth or codec.

[0192] For example, in the case where a file in which a first file stream is stored and a file in which a second file stream is stored generated in the file control unit 43 are arbitrary files, file streams that are different from each other in at least one aspect or two or more aspects of chroma format, bit depth or codec can be used as the first file stream and the second file stream.

[0193] In addition, for example, in the case where the file in which the first file stream is stored and the file in which the second file stream is stored generated in the file control unit 43 include a HEIF file, that is, in the case where one of the file in which the first file stream is stored or the file in which the second file stream is stored is a HEIF file, file streams that are different from each other in at least one aspect or two or more aspects of size, chroma format, bit depth or codec can be used as the first file stream and the second file stream.

[0194] Figure 16 1 is a diagram illustrating an example of generating a HEIF file and a JPEG file having the same image content when the encoding control unit 42 includes a one-chip semiconductor chip 110 .

[0195] Here, it is assumed that there are three sizes of main images (primary sizes) to be stored in HEVC streams in HEIF files. Similarly, it is assumed that there are three sizes of main images to be stored in JPEG streams in JPEG and ARW files. These three sizes are represented by S, M, and L in ascending order.

[0196] Note that the three sizes S, M and L of the main image to be the HEVC stream (hereinafter also referred to as the HEVC main image) and the three sizes S, M and L of the main image to be the JPEG stream (hereinafter also referred to as the JPEG main image) can be the same or different.

[0197] exist Figure 16 In the present invention, as the original image sig11, for example, a YUV image having a size of S, M, or L (S / M / L) of the HEVC main image size, a chroma format of 422 or 420 (422 / 420), and a bit depth of 10 bits or 8 bits (10 bits / 8 bits) is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. The encoding control unit 42 receives such an original image sig11 as the HEVC main image sig11.

[0198] Note that in Figure 16 In , it is assumed that the sizes S, M, and L of the HEVC main image are larger than the sizes S, M, and L of the JPEG main image, respectively.

[0199] In the encoding control unit 42 including one chip of the semiconductor chip 110 , the YUV image as the original image sig11 from the optical system / image sensor control unit 41 is branched and supplied to the generation units 112 and 113 .

[0200] Then, in the encoding control unit 42, the generation unit 112 generates a YUV image having a size of QFHD or FHD (QFHD / FHD), a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits based on the YUV image as the original image sig11 from the optical system / image sensor control unit 41, as a screen thumbnail (hereinafter, also referred to as HEVC screen thumbnail) sig12 to be an HEVC stream.

[0201] In addition, in the encoding control unit 42, the generation unit 112 generates a YUV image with a size of 320×240, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits based on the YUV image as the original image sig11, as a thumbnail (hereinafter also referred to as HEVC thumbnail) sig13 to become an HEVC stream.

[0202] In addition, in the encoding control unit 42, the generation unit 113 generates a YUV image with a size of S, M or L, a chroma format of 422 and a bit depth of 8 bits based on the YUV image as the original image sig11, as the main image to be the JPEG stream (hereinafter also referred to as the JPEG main image) sig31.

[0203] In addition, in the encoding control unit 42, the generation unit 113 generates a YUV image with a size of FHD, a chroma format of 422 and a bit depth of 8 bits based on the YUV image as the original image sig11, as a screen thumbnail (hereinafter also referred to as a JPEG screen thumbnail) sig32 to be a JPEG stream.

[0204] In addition, in the encoding control unit 42, the generation unit 113 generates a YUV image with a size of 160×120, a chroma format of 422 and a bit depth of 8 bits based on the YUV image as the original image sig11, as a thumbnail (hereinafter also referred to as a JPEG thumbnail) sig33 to become a JPEG stream.

[0205] As described above, the HEVC main image sig11, HEVC screen thumbnail sig12, HEVC thumbnail sig13, and the JPEG main image sig31, JPEG screen thumbnail sig32, and JPEG thumbnail sig33 are all generated based on the common (same) (one) original image sig11. As a result, multiple files having the same image content can be easily generated simultaneously with the simplest possible configuration, in which file streams having different data formats are stored.

[0206] Note that it is assumed that the data format (excluding the codec) of the original image is a data format greater than or equal to the data format of the image to become the file stream to be stored in the file to be generated by the file control unit 43. That is, it is assumed that the size, chroma format, and bit depth of the original image are respectively equal to the size, chroma format, and bit depth of the image to become the file stream, or are a higher image quality value.

[0207] For example, in Figure 16 In the case where size L is adopted as the size of the JPEG main image, when size L of the JPEG main image is greater than size S of the HEVC main image but smaller than size M of the HEVC main image, size M or size L of the HEVC main image whose size is greater than or equal to size L of the JPEG main image is adopted as the size of the original image sig11.

[0208] In addition, for example, Figure 16 In the example, 422 or 420 can be used as the color format of the original image sig11. However, 422 is used as the color format of the JPEG main image sig31, the JPEG screen thumbnail sig32, and the JPEG thumbnail sig33. For this reason, as the color format of the original image sig11, 422 is used, which is greater than or equal to the color format 422 (resolution) of the JPEG main image sig31, the JPEG screen thumbnail sig32, and the JPEG thumbnail sig33.

[0209] As described above, as the data format of the original image, a data format greater than or equal to the data format (specifications) of the image to be the file stream can be adopted. In this case, for example, the image quality of the main image can be prevented from being substantially the same as that of the screen thumbnail or thumbnail.

[0210] Note that the data format of the original image can be a smaller format than that of the image to be used as the file stream. In other words, the data format of the original image can be a data format with lower specifications than that of the image to be used as the file stream. In this case, the image to be used as the file stream is generated using the original image through interpolation or other means.

[0211] Thereafter, in the encoding control unit 42, the generation unit 112 performs HEVC encoding on the HEVC main image sig11, the HEVC screen thumbnail sig12, and the HEVC thumbnail sig13. As a result, HEVC streams st11, st12, and st13 of the main image, screen thumbnail, and thumbnail are generated (respectively as first file streams). Similarly, in the encoding control unit 42, the generation unit 113 performs JPEG encoding on the JPEG main image sig31, the JPEG screen thumbnail sig32, and the JPEG thumbnail sig33. As a result, JPEG streams st21, st22, and st23 of the main image, screen thumbnail, and thumbnail are generated (respectively as second file streams).

[0212] The encoding control unit 42 supplies the HEVC streams st11 , st12 , and st13 of the main image, screen thumbnail, and thumbnail, and the JPEG streams st21 , st22 , and st23 to the file control unit 43 .

[0213] The file control unit 43 generates a HEIF file f11 that stores the HEVC streams st11, st12, and st13 of the main image, screen thumbnail, and thumbnail from the encoding control unit 42. In addition, the file control unit 43 generates a JPEG file f12 that stores the JPEG streams st21, st22, and st23 of the main image, screen thumbnail, and thumbnail from the encoding control unit 42.

[0214] Note that in Figure 16 In the example, the HEVC thumbnail is generated by directly using the original image, but the HEVC thumbnail may be generated by using an image generated in a process of generating images of various data formats in the encoding control unit 42.

[0215] For example, HEVC screen thumbnail sig13 can be generated by using HEVC screen thumbnail sig12, as shown by the outline arrow in the figure. Similarly, for example, JPEG screen thumbnail sig32 can be generated by using JPEG main image sig31, and JPEG thumbnail sig33 can be generated by using JPEG screen thumbnail sig32.

[0216] In addition, for example, in the case where the data formats (excluding codec) of the HEVC screen thumbnail sig12 and the JPEG screen thumbnail sig32 are the same, that is, in the case where the HEVC screen thumbnail sig12 and the JPEG screen thumbnail sig32 are YUV images with a size of FHD, a chroma format of 422 and a bit depth of 8 bits, for example, the HEVC screen thumbnail sig12 generated based on the original image sig11 can be used as JPEG screen thumbnail sig32 as it is.

[0217] The same applies to the generation of multiple files with the same image content described below.

[0218] Figure 17 1 is a diagram illustrating another example of generating a HEIF file and a JPEG file having the same image content when the encoding control unit 42 includes a one-chip semiconductor chip 110 .

[0219] exist Figure 17 In the embodiment, as the original image sig41, for example, a YUV image having a size of S, M, or L of a JPEG main image, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. The encoding control unit 42 receives such an original image sig41 as the JPEG main image sig41.

[0220] Note that in Figure 17 In

[15] , it is assumed that the sizes S, M, and L of the JPEG main image are larger than the sizes S, M, and L of the HEVC main image, respectively.

[0221] In the encoding control unit 42, the generation unit 112 generates a YUV image having sizes of S, M, and L of a HEVC main image, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as a HEVC main image sig51 based on the YUV image as the original image sig41 from the optical system / image sensor control unit 41.

[0222] In addition, in the encoding control unit 42, the generation unit 112 generates a YUV image with a size of QFHD or FHD, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as an HEVC screen thumbnail sig52 based on the YUV image as the original image sig41.

[0223] Furthermore, in the encoding control unit 42 , the generation unit 112 generates a YUV image having a size of 320×240, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as an HEVC thumbnail sig53 from the YUV image as the original image sig41 .

[0224] In addition, in the encoding control unit 42 , the generation unit 113 generates a YUV image having a size of FHD, a chroma format of 422, and a bit depth of 8 bits as a JPEG screen thumbnail sig42 from the YUV image as the original image sig41 .

[0225] In addition, in the encoding control unit 42 , the generation unit 113 generates a YUV image having a size of 160×120, a chroma format of 422, and a bit depth of 8 bits as a JPEG thumbnail sig43 from the YUV image as the original image sig41 .

[0226] As described above, the HEVC main image sig51 , HEVC screen thumbnail sig52 , HEVC thumbnail sig53 , and the JPEG main image sig41 , JPEG screen thumbnail sig42 , and JPEG thumbnail sig43 are all generated based on the same original image sig41 .

[0227] Afterwards, similar to Figure 16 In the case of [ ], in the encoding control unit 42, the generation unit 112 performs HEVC encoding on the HEVC main image sig51, the HEVC screen thumbnail sig52, and the HEVC thumbnail sig53. As a result, HEVC streams st31, st32, and st33 of the main image, screen thumbnail, and thumbnail are generated. Similarly, in the encoding control unit 42, the generation unit 113 performs JPEG encoding on the JPEG main image sig41, the JPEG screen thumbnail sig42, and the JPEG thumbnail sig43. As a result, the JPEG streams st41, st42, and st43 of the main image, screen thumbnail, and thumbnail are generated.

[0228] The encoding control unit 42 supplies the HEVC streams st31 , st32 , and st33 of the main image, screen thumbnail, and thumbnail, and the JPEG streams st41 , st42 , and st43 to the file control unit 43 .

[0229] The file control unit 43 generates a HEIF file f31 that stores the HEVC streams st31, st32, and st33 of the main image, screen thumbnail, and thumbnail from the encoding control unit 42. In addition, the file control unit 43 generates a JPEG file f32 that stores the JPEG streams st41, st42, and st43 of the main image, screen thumbnail, and thumbnail from the encoding control unit 42.

[0230] Figure 18 1 is a diagram illustrating an example of generating a HEIF file and an ARW file having the same image content when the encoding control unit 42 includes a one-chip semiconductor chip 110 .

[0231] exist Figure 18 In, similar to Figure 16 As the original image sig11, for example, a YUV image having a size of S, M, or L of an HEVC main image, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. The encoding control unit 42 receives such an original image sig11 as the HEVC main image sig11.

[0232] Note that in Figure 18 In, similar to Figure 16 , assuming that the sizes S, M, and L of the HEVC main image are larger than the sizes S, M, and L of the JPEG main image, respectively.

[0233] In the encoding control unit 42, the generation unit 112 generates a YUV image corresponding to the original image sig11 from the optical system / image sensor control unit 41. Figure 16 The cases are similar to those of HEVC screen thumbnail sig12 and HEVC thumbnail sig13.

[0234] In addition, in the encoding control unit 42, the generation unit 113 generates the YUV image corresponding to the original image sig11 from the YUV image. Figure 16 Those similar to the case of JPEG main image sig31, JPEG screen thumbnail sig32 and JPEG thumbnail sig33.

[0235] Afterwards, with Figure 16Similarly, in the encoding control unit 42, the generation unit 112 performs HEVC encoding on the HEVC main image sig11, the HEVC screen thumbnail sig12, and the HEVC thumbnail sig13. As a result, HEVC streams st11, st12, and st13 of the main image, screen thumbnail, and thumbnail are generated. Similarly, in the encoding control unit 42, the generation unit 113 performs JPEG encoding on the JPEG main image sig31, the JPEG screen thumbnail sig32, and the JPEG thumbnail sig33. As a result, JPEG streams st21, st22, and st23 of the main image, screen thumbnail, and thumbnail are generated.

[0236] The encoding control unit 42 supplies the HEVC streams st11 , st12 , and st13 of the main image, screen thumbnail, and thumbnail, and the JPEG streams st21 , st22 , and st23 to the file control unit 43 .

[0237] In addition, Figure 18 , the encoding control unit 42 supplies the RAW image sig71 from the optical system / image sensor control unit 41 to the file control unit 43 as a RAW stream sig71. The file control unit 43 generates a HEIF file f11 that stores the HEVC streams st11, st12, and st13 of the main image, screen thumbnail, and thumbnail from the encoding control unit 42. In addition, the file control unit 43 generates an ARW file f51 that stores the RAW stream sig71 from the encoding control unit 42 and the JPEG streams st21, st22, and st23 of the main image, screen thumbnail, and thumbnail.

[0238] Figure 19 1 is a diagram illustrating an example of generating two HEIF files having the same image content when the encoding control unit 42 includes a one-chip semiconductor chip 110 .

[0239] Here, in Figure 19 In the two HEIF files generated in the above embodiment, HEVC main images (HEVC streams) of different sizes in data formats are stored. For example, an HEVC main image of size L is stored in one of the two HEIF files, and an HEVC main image of size S is stored in the other HEIF file.

[0240] The HEIF file in which the HEVC main image with size L is stored is also referred to as the first HEIF file, and the HEIF file in which the HEVC main image with size S is stored is also referred to as the second HEIF file.

[0241] In addition, the HEVC main image stored in the first HEIF file (in which the HEVC stream is stored) is also referred to as a first HEVC main image, and the HEVC main image stored in the second HEIF file is also referred to as a second HEVC main image.

[0242] exist Figure 19 In the embodiment, as the original image sig101, for example, a YUV image having a size of L of an HEVC main image, a chroma format of 422, and a bit depth of 10 bits is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. The encoding control unit 42 receives such an original image sig101 as the first HEVC main image sig101.

[0243] In the encoding control unit 42 , the generation unit 112 generates a YUV image with a size of QFHD, a chroma format of 422, and a bit depth of 10 bits as an HEVC screen thumbnail sig102 based on the YUV image as the original image sig101 from the optical system / image sensor control unit 41 .

[0244] Furthermore, in the encoding control unit 42 , the generation unit 112 generates a YUV image having a size of 320×240, a chroma format of 422, and a bit depth of 10 bits as an HEVC thumbnail sig103 from the YUV image as the original image sig101 .

[0245] In addition, in the encoding control unit 42, the generation unit 113 generates a YUV image with a size of S of the HEVC main image, a chroma format of 422, and a bit depth of 10 bits as a second HEVC main image sig111 from the YUV image as the original image sig101.

[0246] After that, in the encoding control unit 42, the generation unit 112 performs HEVC encoding on the first HEVC main image sig101, HEVC screen thumbnail sig102, and HEVC thumbnail sig103. As a result, HEVC streams st101, st102, and st103 of the first HEVC main image, HEVC screen thumbnail, and HEVC thumbnail are generated.

[0247] In addition, in the encoding control unit 42 , the generation unit 113 performs HEVC encoding on the second HEVC main image sig111 to generate a HEVC stream st111 of the second HEVC main image.

[0248] The encoding control unit 42 supplies the first HEVC main image, the second HEVC main image, the HEVC screen thumbnail, and the HEVC streams st101 , st111 , st102 , and st103 of the HEVC thumbnail to the file control unit 43 .

[0249] The file control unit 43 generates a first HEIF file f101 that stores the HEVC streams st101, st102, and st103 of the first HEVC main image, HEVC screen thumbnail, and HEVC thumbnail from the encoding control unit 42. In addition, the file control unit 43 generates a second HEIF file f102 that stores the HEVC streams st111, st102, and st103 of the second HEVC main image, HEVC screen thumbnail, and HEVC thumbnail from the encoding control unit 42.

[0250] Figure 20 1 is a diagram illustrating an example of generating two JPEG files having the same image content in a case where the encoding control unit 42 includes a one-chip semiconductor chip 110 .

[0251] Here, in Figure 20 In the two JPEG files generated in the above example, a JPEG main image (a JPEG stream) having a different bit depth from each other is stored. For example, a JPEG main image having a bit depth of 10 bits is stored in one of the two JPEG files, and a JPEG main image having a bit depth of 8 bits is stored in the other JPEG file.

[0252] A JPEG file in which a JPEG main image with a bit depth of 10 bits is stored is also referred to as a first JPEG file, and a JPEG file in which a JPEG main image with a bit depth of 8 bits is also referred to as a second JPEG file.

[0253] In addition, the JPEG main image stored in the first JPEG file (in which the JPEG stream is stored) is also referred to as a first JPEG main image, and the JPEG main image stored in the second JPEG file is also referred to as a second JPEG main image.

[0254] Note that, in the current JPEG, a bit depth of 10 bits cannot be handled, but it is assumed here that JPEG will be extended in the future and can handle a bit depth of 10 bits.

[0255] exist Figure 20 In the embodiment, as the original image sig131, for example, a YUV image having a size of L of a JPEG main image, a chroma format of 422, and a bit depth of 10 bits is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. The encoding control unit 42 receives such an original image sig131 as a first JPEG main image sig131.

[0256] In the encoding control unit 42, the generation unit 112 generates a YUV image with a size of FHD, a chroma format of 422 and a bit depth of 8 bits as a JPEG screen thumbnail sig132 based on the YUV image as the original image sig131 from the optical system / image sensor control unit 41.

[0257] Furthermore, in the encoding control unit 42 , the generation unit 112 generates a YUV image having a size of 160×120, a chroma format of 422, and a bit depth of 8 bits as a JPEG thumbnail sig133 from the YUV image as the original image sig131 .

[0258] In addition, in the encoding control unit 42, the generation unit 112 generates a YUV image with a size of L of a JPEG main image, a chroma format of 422, and a bit depth of 8 bits as a second JPEG main image sig141 based on the YUV image as the original image sig131.

[0259] After that, in the encoding control unit 42, the generation unit 112 performs JPEG encoding on the first JPEG main image sig131, the JPEG screen thumbnail sig132, and the JPEG thumbnail sig133. As a result, JPEG streams st131, st132, and st133 of the first JPEG main image, the JPEG screen thumbnail, and the JPEG thumbnail are generated.

[0260] In addition, in the encoding control unit 42 , the generation unit 113 performs JPEG encoding on the second JPEG main image sig141 to generate a JPEG stream st141 of the second JPEG main image.

[0261] The encoding control unit 42 supplies the JPEG streams st131 , st141 , st132 , and st133 of the first JPEG main image, the second JPEG main image, the JPEG screen thumbnail, and the JPEG thumbnail to the file control unit 43 .

[0262] The file control unit 43 generates a first JPEG file f111 that stores the first JPEG main image, the JPEG screen thumbnail, and the JPEG streams st131, st132, and st133 of the JPEG thumbnail from the encoding control unit 42. Furthermore, the file control unit 43 generates a second JPEG file f112 that stores the second JPEG main image, the JPEG screen thumbnail, and the JPEG streams st141, st132, and st133 of the JPEG thumbnail from the encoding control unit 42.

[0263] <Second Configuration Example of the Encoding Control Unit 42>

[0264] Figure 21: is a block diagram showing a second configuration example of the encoding control unit 42 .

[0265] exist Figure 21 In the embodiment, the encoding control unit 42 includes two chips: semiconductor chips 210 and 220. The semiconductor chips 210 and 220 are connected in series, with the semiconductor chip 210 being arranged at the front stage and the semiconductor chip 220 being arranged at the back stage.

[0266] The semiconductor chip 210 (functionally) includes an input I / F 211 , a generating unit 212 , and an output I / F 213 .

[0267] The input I / F 211 receives external signals and supplies them to necessary blocks. For example, the input I / F 211 receives a YUV image as a raw image from the optical system / image sensor control unit 41 and supplies the YUV image to the generation unit 212 and the output I / F 213. Furthermore, for example, the input I / F 211 receives a RAW image from the optical system / image sensor control unit 41 and supplies the RAW image to the output I / F 213.

[0268] The generation unit 212 generates a first file stream based on the original image from the input I / F 211 and supplies the first file stream to the output I / F 213. For example, the generation unit 212 encodes the original image using a first encoding method and supplies the resulting stream as the first file stream to the output I / F 213. Furthermore, for example, the generation unit 212 converts one or more of the size, chroma format, or bit depth of the original image to generate a first converted image. The generation unit 212 then encodes the first converted image using the first encoding method and supplies the resulting stream as the first file stream to the output I / F 213.

[0269] The output I / F 213 outputs (supplies) signals to the outside. For example, the output I / F 213 supplies the original image and the RAW image supplied from the input I / F 211 and the first file stream supplied from the generation unit 212 to the semiconductor chip 220.

[0270] The semiconductor chip 220 (functionally) includes an input I / F 221 , a generation unit 222 , and an output I / F 223 .

[0271] The input I / F 221 receives external signals and supplies them to necessary blocks. For example, the input I / F 221 receives a YUV image as a raw image from the semiconductor chip 210 (output I / F 213) and supplies the YUV image to the generation unit 222. Furthermore, for example, the input I / F 221 receives a first file stream and a RAW image from the semiconductor chip 210 and supplies the first file stream and the RAW image to the output I / F 223.

[0272] The generation unit 222 generates a second file stream based on the original image from the input I / F 221 and supplies the second file stream to the output I / F 223. For example, the generation unit 222 encodes the original image using the second encoding method and supplies the resulting stream as the second file stream to the output I / F 223. Alternatively, for example, the generation unit 222 converts one or more of the size, chroma format, or bit depth of the original image to generate a second converted image. The generation unit 222 then encodes the second converted image using the second encoding method and supplies the resulting stream as the second file stream to the output I / F 223.

[0273] The output I / F 223 outputs (supplies) signals to the outside. For example, the output I / F 223 supplies the first file stream and the RAW image supplied from the input I / F 221 and the second file stream supplied from the generation unit 222 to the file control unit 43.

[0274] In the file control unit 43, a file in which the first file stream is stored and a file in which the second file stream is stored are generated.

[0275] As described above, when the digital camera 10 generates multiple files having the same image content but having different file formats, the encoding control unit 42 including the two semiconductor chips 210 and 220 is useful. For example, a file stream (a first file stream) to be stored in a file of one file format can be generated by one of the semiconductor chips 210 and 220. Alternatively, a file stream (a second file stream) to be stored in a file of another file format can be generated by the other semiconductor chip.

[0276] Figure 22 1 is a diagram illustrating an example of generating a HEIF file and a JPEG file having the same image content when the encoding control unit 42 includes two semiconductor chips 210 and 220 .

[0277] exist Figure 22In the encoding control unit 42, as the original image sig11, for example, a YUV image having a size of S, M, or L of an HEVC main image, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. In the encoding control unit 42, the semiconductor chip 210 at the previous stage receives such an original image sig11 as the HEVC main image sig11.

[0278] Note that in Figure 22 In , it is assumed that the sizes S, M, and L of the HEVC main image are larger than the sizes S, M, and L of the JPEG main image, respectively.

[0279] In the encoding control unit 42, the preceding semiconductor chip 210 generates a YUV image having a size of QFHD or FHD, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as a HEVC screen thumbnail sig12 based on the YUV image as the original image sig11 from the optical system / image sensor control unit 41.

[0280] Furthermore, in the encoding control unit 42 , the semiconductor chip 210 generates a YUV image having a size of 320×240, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as an HEVC thumbnail sig13 from the YUV image as the original image sig11 .

[0281] After that, in the encoding control unit 42, the semiconductor chip 210 performs HEVC encoding on the HEVC main image sig11, HEVC screen thumbnail sig12, and HEVC thumbnail sig13. As a result, HEVC streams st11, st12, and st13 of the main image, screen thumbnail, and thumbnail are generated as first file streams, respectively.

[0282] The semiconductor chip 210 supplies the original image sig11 and the main image, screen thumbnail, and HEVC streams st11 , st12 , and st13 of the thumbnail to the semiconductor chip 220 of the subsequent stage.

[0283] Therefore, in the encoding control unit 42 including the two chips of the semiconductor chips 210 and 220 , the original image sig11 is forwarded from the semiconductor chip 210 at the preceding stage to the semiconductor chip 220 at the succeeding stage.

[0284] In the encoding control unit 42 , the semiconductor chip 220 at the subsequent stage receives the original image sig11 and the main image, screen thumbnail, and HEVC streams st11 , st12 , and st13 of the thumbnail from the semiconductor chip 210 at the preceding stage.

[0285] Then, the semiconductor chip 220 generates a YUV image of size S, M, or L, chroma format 422, and bit depth 8 as a JPEG main image sig31 based on the YUV image as the original image sig11 from the semiconductor chip 210 of the previous stage.

[0286] Furthermore, the semiconductor chip 220 generates a YUV image having a size of FHD, a chroma format of 422, and a bit depth of 8 bits as a JPEG screen thumbnail sig32 from the YUV image as the original image sig11.

[0287] In addition, the semiconductor chip 220 generates a YUV image having a size of 160×120, a chroma format of 422, and a bit depth of 8 bits as a JPEG thumbnail sig33 from the YUV image as the original image sig11 .

[0288] Thereafter, the semiconductor chip 220 performs JPEG encoding on the JPEG main image sig31, the JPEG screen thumbnail sig32, and the JPEG thumbnail sig33. As a result, JPEG streams st21, st22, and st23 of the main image, screen thumbnail, and thumbnail are generated (respectively as second file streams).

[0289] Then, the semiconductor chip 220 supplies the main image, the screen thumbnail, and the JPEG streams st21 , st22 , and st23 of the thumbnail to the file control unit 43 .

[0290] Furthermore, the semiconductor chip 220 supplies the main image, the screen thumbnail, and the HEVC streams st11 , st12 , and st13 of the thumbnail from the semiconductor chip 210 at the preceding stage to the file control unit 43 .

[0291] In the file control unit 43, a HEIF file f11 is generated, which stores the HEVC streams st11, st12, and st13 of the main image, screen thumbnail, and thumbnail from the semiconductor chip 220. In addition, in the file control unit 43, a JPEG file f12 is generated, which stores the JPEG streams st21, st22, and st23 of the main image, screen thumbnail, and thumbnail from the semiconductor chip 220.

[0292] Figure 23 1 is a diagram illustrating another example of generating a HEIF file and a JPEG file having the same image content when the encoding control unit 42 includes two semiconductor chips 210 and 220 .

[0293] exist Figure 22In the embodiment, HEVC streams st11, st12, and st13 are generated in the semiconductor chip 210 at the preceding stage, and JPEG streams st21, st22, and st23 are generated in the semiconductor chip 220 at the subsequent stage. Figure 23 and Figure 22 The case is different in that the JPEG streams st21 , st22 , and st23 are generated in the semiconductor chip 210 at the preceding stage, and the HEVC streams st11 , st12 , and st13 are generated in the semiconductor chip 220 at the subsequent stage.

[0294] That is, in Figure 23 In the encoding control unit 42, as the original image sig11, for example, a YUV image having a size of S, M, or L of an HEVC main image, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits is supplied from the optical system / image sensor control unit 41 to the encoding control unit 42. In the encoding control unit 42, the semiconductor chip 210 of the preceding stage receives such an original image sig11 as the HEVC main image sig11.

[0295] Then, the semiconductor chip 210 generates a YUV image of size S, M or L, chroma format 422 and bit depth 8 as a JPEG main image sig31 based on the YUV image as the original image sig11 from the optical system / image sensor control unit 41.

[0296] Furthermore, the semiconductor chip 210 generates a YUV image having a size of FHD, a chroma format of 422, and a bit depth of 8 bits as a JPEG screen thumbnail sig32 from the YUV image as the original image sig11.

[0297] In addition, the semiconductor chip 210 generates a YUV image having a size of 160×120, a chroma format of 422, and a bit depth of 8 bits as a JPEG thumbnail sig33 from the YUV image as the original image sig11 .

[0298] Thereafter, the semiconductor chip 210 performs JPEG encoding on the JPEG main image sig31, the JPEG screen thumbnail sig32, and the JPEG thumbnail sig33. As a result, JPEG streams st21, st22, and st23 of the main image, screen thumbnail, and thumbnail are generated (respectively as first file streams).

[0299] The semiconductor chip 210 supplies the original image sig11 and the main image, screen thumbnail, and JPEG streams st21 , st22 , and st23 of the thumbnail to the semiconductor chip 220 of the subsequent stage.

[0300] In the encoding control unit 42, the semiconductor chip 220 of the subsequent stage receives the original image sig11 from the semiconductor chip 210 of the preceding stage as the HEVC main image sig11. In addition, the semiconductor chip 220 receives the main image, screen thumbnails, and JPEG streams st21, st22, and st23 of the thumbnails from the semiconductor chip 210 of the preceding stage.

[0301] Then, the subsequent semiconductor chip 220 generates a YUV image with a size of QFHD or FHD, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as an HEVC screen thumbnail sig12 based on the YUV image as the original image sig11 from the previous semiconductor chip 210.

[0302] Furthermore, the semiconductor chip 220 generates a YUV image having a size of 320×240, a chroma format of 422 or 420, and a bit depth of 10 bits or 8 bits as an HEVC thumbnail sig13 from the YUV image as the original image sig11 .

[0303] Thereafter, the semiconductor chip 220 performs HEVC encoding on the HEVC main image sig11, the HEVC screen thumbnail sig12, and the HEVC thumbnail sig13. As a result, HEVC streams st11, st12, and st13 of the main image, screen thumbnail, and thumbnail are generated (respectively as second file streams).

[0304] Then, the semiconductor chip 220 supplies the main image, the screen thumbnail, and the HEVC streams st11 , st12 , and st13 of the thumbnail to the file control unit 43 .

[0305] Furthermore, the semiconductor chip 220 supplies the main image, the screen thumbnail, and the JPEG streams st21 , st22 , and st23 of the thumbnail from the semiconductor chip 210 at the preceding stage to the file control unit 43 .

[0306] In the file control unit 43, a HEIF file f11 is generated, which stores the HEVC streams st11, st12, and st13 of the main image, screen thumbnail, and thumbnail from the semiconductor chip 220. In addition, in the file control unit 43, a JPEG file f12 is generated, which stores the JPEG streams st21, st22, and st23 of the main image, screen thumbnail, and thumbnail from the semiconductor chip 220.

[0307] In the above, the generation of HEIF files and JPEG files has been described as an example of generating multiple files with the same image content by the encoding control unit 42 including the two semiconductor chips 210 and 220 (hereinafter also referred to as the encoding control unit 42 with a two-chip configuration). In addition to generating HEIF files and JPEG files, the encoding control unit 42 with a two-chip configuration can also generate, for example, Figure 18 HEIF files and ARW files described in . In addition, the encoding control unit 42 with a two-chip configuration can generate, for example Figure 19 The two HEIF files described in Figure 20 In addition, the encoding control unit 42 having a two-chip configuration can generate, for example, three or more files, as well as a plurality of files including files other than HEIF files, JPEG files, and ARW files.

[0308] In addition, in the digital camera 10, for example, one or more of the data formats (codec, size, chroma format, or bit depth) of file streams to be stored in multiple files having the same image content can be set according to user operations (designations).

[0309] <Description of Computer to Which the Present Technology is Applied>

[0310] Next, the signal processing unit 13 ( Figure 1 ) blocks. When the series of processes is executed by software, a program constituting the software is installed in a computer or the like.

[0311] Figure 24 : is a block diagram showing a configuration example of an embodiment of a computer in which a program for executing the above-described series of processes is installed.

[0312] The program can be recorded in advance on the hard disk 905 or the ROM 903 as a recording medium included in the computer.

[0313] Alternatively, the program may be stored (recorded) in a removable recording medium 911 driven by the drive 909. Such a removable recording medium 911 may be provided as so-called packaged software. Examples of the removable recording medium 911 include a floppy disk, a compact disc read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a magnetic disk, a semiconductor memory, and the like.

[0314] Note that the program can be installed on the computer from the removable recording medium 911 as described above, or can be downloaded to the computer via a communication network or a broadcast network and installed on the included hard disk 905. In other words, for example, the program can be wirelessly transmitted from a download site to the computer via an artificial satellite used for digital satellite broadcasting, or can be transmitted to the computer by cable via a network such as a local area network (LAN) or the Internet.

[0315] The computer includes a central processing unit (CPU) 902 , and an input / output I / F 910 is connected to the CPU 902 via a bus 901 .

[0316] When a user who operates the input unit 907 or the like inputs a command via the input / output I / F 910, the CPU 902 executes a program stored in a read-only memory (ROM) 903 according to the command. Alternatively, the CPU 902 loads a program stored in the hard disk 905 into a random access memory (RAM) 904 and executes the program.

[0317] The CPU 902 thus executes the processing according to the above-described flowchart or the processing executed by the configuration of the above-described block diagram. The CPU 902 then causes the processing result to be output from the output unit 906 or transmitted from the communication unit 908 via the input / output I / F 910, and also recorded on, for example, the hard disk 905 as needed.

[0318] Note that the input unit 907 includes a keyboard, a mouse, a microphone, etc. In addition, the output unit 906 includes a liquid crystal display (LCD), a speaker, and the like.

[0319] Here, in this specification, the processing performed by the computer according to the program does not necessarily have to be performed in chronological order according to the order described in the flowchart. That is, the processing performed by the computer according to the program also includes processing performed in parallel or individually (for example, parallel processing or object processing).

[0320] In addition, the program may be processed by one computer (processor), or may be distributed and processed by a plurality of computers. In addition, the program may be transferred to a remote computer and executed.

[0321] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all components are located in the same cabinet. Therefore, multiple devices housed in separate cabinets and interconnected via a network, as well as a single device housing multiple modules in a single cabinet, are both considered systems.

[0322] Note that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present technology.

[0323] For example, the present technology may adopt a configuration of cloud computing that shares one function among a plurality of devices via a network to cooperatively perform processing.

[0324] In addition, each step described in the above flowchart may be performed by being shared among a plurality of devices instead of being performed by one device.

[0325] Furthermore, in the case where a plurality of processes are included in one step, the plurality of processes included in this one step may be executed by being shared among a plurality of devices in addition to being executed by one device.

[0326] In addition, the advantageous effects described in this specification are merely examples and are not limited thereto, and other effects may be included.

[0327] Note that the present technology can have the following configurations.

[0328] <1>

[0329] A data processing device comprising

[0330] an encoding control unit that generates file data obtained by encoding the image and to be stored in each of a plurality of files based on the same image, wherein

[0331] First file data to be stored in one file of the plurality of files and second file data to be stored in another file are data different from each other in at least one of codec, chroma format, or bit depth.

[0332] <2>

[0333] according to <1> The data processing device, wherein

[0334] The size, chroma format and bit depth of the same image are greater than or equal to the size, chroma format and bit depth of the image whose file data is stored in each file.

[0335] <3>

[0336] according to <1> or <2> The data processing device, wherein

[0337] One or more of the codec, chroma format, or bit depth of the file data is set according to a user's designation.

[0338] <4>

[0339] according to <1> to <3> Any one of the data processing devices described in

[0340] The one file or the other file is a RAW file in which a RAW image is stored.

[0341] <5>

[0342] according to <1> to <4> Any one of the data processing devices described in

[0343] The encoding control unit includes a semiconductor chip, and

[0344] The semiconductor chip of the chip

[0345] include:

[0346] a first generating unit, generating one of the first file data and the second file data; and

[0347] The second generating unit generates another file data.

[0348] <6>

[0349] according to <1> to <4> Any one of the data processing devices described in

[0350] The encoding control unit includes a two-chip semiconductor chip, and

[0351] One of the semiconductor chips of the two chips generates one of the first file data or the second file data, and

[0352] Another semiconductor chip generates another file data.

[0353] <7>

[0354] according to <6> The data processing device, wherein

[0355] The one semiconductor chip receives the same image, generates a file data based on the same image, and forwards the same image to the other semiconductor chip, and

[0356] The other semiconductor chip generates another file data based on the same image from the one semiconductor chip.

[0357] <8>

[0358] according to <1> to <7> Any one of the data processing devices described in

[0359] The first file data is data obtained by performing HEVC encoding on an image, and

[0360] The second file data is data obtained by performing JPEG encoding on an image.

[0361] <9>

[0362] A data processing method comprising:

[0363] File data obtained by encoding an image and to be stored in each of a plurality of files is generated based on the same image, wherein

[0364] First file data to be stored in one file of the plurality of files and second file data to be stored in another file are data different from each other in at least one of codec, chroma format, or bit depth.

[0365] <10>

[0366] A program for causing a computer to function as an encoding control unit that generates file data obtained by encoding an image and to be stored in each of a plurality of files based on the same image, wherein

[0367] First file data to be stored in one file of the plurality of files and second file data to be stored in another file are data different from each other in at least one of codec, chroma format, or bit depth.

[0368] <11>

[0369] A data processing device comprising

[0370] an encoding control unit that generates file data obtained by encoding an image and to be stored in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file based on the same image, wherein

[0371] First file data to be stored in a HEIF file among the plurality of files and second file data to be stored in another file other than the HEIF file are data different from each other in at least one of codec, chroma format, bit depth, or image size.

[0372] <12>

[0373] according to <11> The data processing device, wherein

[0374] The size, chroma format and bit depth of the same image are greater than or equal to the size, chroma format and bit depth of the image in which the file data is stored in the file.

[0375] <13>

[0376] according to <11> or <12> The data processing device, wherein

[0377] One or more of the codec, color format, bit depth, or image size of the file data is set according to a user's designation.

[0378] <14>

[0379] according to <11> to <13> Any one of the data processing devices described in

[0380] The other file is a RAW file in which a RAW image is stored.

[0381] <15>

[0382] according to <11> to <14> Any one of the data processing devices described in

[0383] The encoding control unit includes a semiconductor chip, and

[0384] The semiconductor chip of the chip

[0385] include:

[0386] a first generating unit, generating one of the first file data and the second file data; and

[0387] The second generating unit generates another file data.

[0388] <16>

[0389] according to <11> to <14> Any one of the data processing devices described in

[0390] The encoding control unit includes a two-chip semiconductor chip, and

[0391] One of the semiconductor chips of the two chips generates one of the first file data or the second file data, and

[0392] Another semiconductor chip generates another file data.

[0393] <17>

[0394] according to <16> The data processing device, wherein

[0395] The one semiconductor chip receives the same image, generates a file data based on the same image, and forwards the same image to the other semiconductor chip, and

[0396] The other semiconductor chip generates another file data based on the same image from the one semiconductor chip.

[0397] <18>

[0398] according to <11> to <17> Any one of the data processing devices described in

[0399] The first file data is data obtained by performing HEVC encoding on an image.

[0400] The second file data is data obtained by performing JPEG encoding on an image.

[0401] <19>

[0402] A data processing method comprising

[0403] File data obtained by encoding an image and to be stored in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file is generated from the same image, wherein

[0404] First file data to be stored in a HEIF file among the plurality of files and second file data to be stored in another file other than the HEIF file are data different from each other in at least one of codec, chroma format, bit depth, or image size.

[0405] <20>

[0406] A program for causing a computer to function as an encoding control unit that generates file data obtained by encoding an image and to be stored in each of a plurality of files including at least a High Efficiency Image File Format (HEIF) file, based on the same image, wherein

[0407] First file data to be stored in a HEIF file among the plurality of files and second file data to be stored in another file other than the HEIF file are data different from each other in at least one of codec, chroma format, bit depth, or image size.

[0408] Label list

[0409] 10. Digital Camera

[0410] 11 Optical System

[0411] 13 Signal Processing Unit

[0412] 14 Medium

[0413] 15, 16I / F

[0414] 17 buttons / keys

[0415] 18 Touch Panel

[0416] 19 LCD panel

[0417] 20 Viewfinder

[0418] 21I / F

[0419] 41 Optical system / image sensor control unit

[0420] 42 Encoding Control Unit

[0421] 43 File Control Unit

[0422] 44 Media Control Unit

[0423] 45 Operation control unit

[0424] 46 Display Control Unit

[0425] 47UI control unit

[0426] 110 semiconductor chips

[0427] 111 Input I / F

[0428] 112, 113 generation unit

[0429] 114 output I / F

[0430] 210 semiconductor chips

[0431] 211 Input I / F

[0432] 212 generation unit

[0433] 213 output I / F

[0434] 220 semiconductor chips

[0435] 221 Input I / F

[0436] 222 generation unit

[0437] 223 Output I / F

[0438] 901 bus

[0439] 902CPU

[0440] 903ROM

[0441] 904RAM

[0442] 905 hard drive

[0443] 906 output unit

[0444] 907 Input Unit

[0445] 908 Communication Unit

[0446] 909 Drive

[0447] 910 input / output interface

[0448] 911 Removable Recording Media

Claims

1. A data processing device comprising: an encoding control unit that generates, based on the same image, first file data and second file data that are obtained by encoding the same image and are to be stored in each of a plurality of files; as well as The file control unit stores the first file data in one of the plurality of files and stores the second file data in another of the plurality of files, wherein The first file data and the second file data each include at least one complete encoded image and are data that differ from each other in at least one of codec, chroma format, or bit depth, and The one file is a RAW file in which a RAW image and the first file data are stored.

2. The data processing device according to claim 1, wherein The size, chroma format and bit depth of the same image are greater than or equal to the size, chroma format and bit depth of the images of the first file data and the second file data stored in each file.

3. The data processing device according to claim 1, wherein One or more of the codec, chroma format, or bit depth of the first file data and the second file data are set according to a user's designation.

4. The data processing device according to claim 1, wherein The encoding control unit includes a semiconductor chip, and The semiconductor chip of the chip include: A first generating unit generates one of the first file data and the second file data; and The second generating unit generates another file data.

5. The data processing apparatus according to claim 1, wherein The encoding control unit includes a two-chip semiconductor chip, and One of the semiconductor chips of the two chips generates one of the first file data or the second file data, and Another semiconductor chip generates another file data.

6. The data processing apparatus according to claim 5, wherein The one semiconductor chip receives the same image, generates the one file data based on the same image, and forwards the same image to the other semiconductor chip, and The other semiconductor chip generates the other file data based on the same image from the one semiconductor chip.

7. The data processing apparatus according to claim 1, wherein The first file data is data obtained by performing JPEG encoding on an image, and The second file data is data obtained by performing HEVC encoding on an image.

8. The data processing apparatus according to claim 1, wherein The other file is a High Efficiency Image File Format HEIF file.

9. A data processing method comprising: generating, based on the same image, first file data and second file data obtained by encoding the same image and to be stored in each of a plurality of files; as well as The first file data is stored in one of the plurality of files, and the second file data is stored in another of the plurality of files, wherein The first file data and the second file data each include at least one complete encoded image and are data that differ from each other in at least one of codec, chroma format, or bit depth, and The one file is a RAW file in which a RAW image and the first file data are stored.

10. A computer program product for causing a computer to: an encoding control unit that generates, from the same image, first file data and second file data that are obtained by encoding the same image and are to be stored in each of a plurality of files; and The file control unit stores the first file data in one of the plurality of files and stores the second file data in another of the plurality of files, wherein The first file data and the second file data each include at least one complete encoded image and are data that differ from each other in at least one of codec, chroma format, or bit depth, and The one file is a RAW file in which a RAW image and the first file data are stored.

Images