Image processing device, method and program for controlling image processing device

Abstract

To provide an image processing device, a method and a program for controlling the image processing device that can make reliable information available to a user.SOLUTION: A digital camera 100 includes metadata creation means (system control unit 50) that converts private metadata 303 to create public metadata 306 that can be made public, and storage means (system control unit 50) that stores the public metadata 306 in an image file 300. When storing the public metadata 306 in the image file 300, the storage means changes the second storage area 304 that stores the public metadata 306 to an area different from the first storage area 301 in which the private metadata 303 is stored, depending on the method of conversion applied to the private metadata 303 by the metadata creation means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing apparatus, a control method for an image processing apparatus, and a program. [Background technology]

[0002] Conventionally, information set as shooting conditions when an image is shot (e.g., exposure control, color settings, etc.) and information acquired after an image is shot (e.g., AF information, etc.) are stored in the makernote. The "makernote" is an area that stores metadata specific to a specific manufacturer included in Exif data in a format that is generally not public. Therefore, software created by manufacturers other than the specific manufacturer could not utilize the information stored in the makernote. In recent years, cameras compatible with APP11 have become known, and public metadata is added to image files shot with the cameras, so that software created by manufacturers other than the specific manufacturer can utilize the public metadata. In addition, with APP11, public metadata can be added to image files in a description format such as JSON or XML. On the other hand, image files shot with cameras that do not support APP11, i.e., cameras that do not support APP11, do not have public metadata added, and there is a risk that there will be a relatively large difference in utility value between image files shot with cameras that support APP11. Therefore, by converting manufacturer-specific metadata contained in image files taken with cameras that do not support APP11 into metadata in APP11 format, the value of the metadata can be increased. In this case, there is a risk that the reliability of the metadata will vary depending on various conditions such as a lack of information at the time of conversion. For example, Patent Document 1 describes a technology that detects a human face from an image and stores as metadata the "confidence level indicating the certainty that the face detection result is a face" contained in the face detection information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2008-5427 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technique described in Patent Document 1 has a problem in that, for example, when private metadata is converted into public metadata, it is unclear how reliable the public metadata is.

[0005] The present invention has been made in view of the above problems, and has an object to provide an image processing device, a control method for the image processing device, and a program for making reliable information available to users. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the image processing device of the present invention is an image processing device that processes an image file in which a captured image and private metadata related to the captured image are stored, and is equipped with a metadata creation means that converts the private metadata to create public metadata that can be made public, and a storage means that stores the public metadata in an image file, and is characterized in that when storing the public metadata in the image file, the storage means changes a public metadata storage area that stores the public metadata to an area different from the private metadata storage area in which the private metadata is stored, depending on the method of conversion of the private metadata by the metadata creation means. Effect of the Invention

[0007] According to the present invention, reliable information can be made available to users. [Brief description of the drawings]

[0008] [Figure 1]FIG. 1 is a block diagram showing an example of a hardware configuration when an image processing device is applied to a digital camera. [Diagram 2] FIG. 2 is a diagram showing an example of an image file. [Diagram 3] FIG. 13 is a diagram showing a modified example of an image file. [Figure 4] FIG. 2 is a diagram relating to the area of ​​a subject's face included in image data. [Diagram 5] FIG. 2 is a diagram relating to the area of ​​a subject's face included in image data. [Figure 6] 2 is a flowchart showing a process executed by the digital camera shown in FIG. 1. [Figure 7] 7 is a flowchart showing processes that can be used in each step (subroutine) of steps S703 to S706 in the flowchart shown in FIG. [Figure 8] FIG. 2 is a diagram showing an example of an image file. [Figure 9] FIG. 13 is a diagram for explaining a case where public metadata is “reliability: high.” [Figure 10] FIG. 13 is a diagram for explaining a case where public metadata is “reliability: medium”. [Figure 11] FIG. 13 is a diagram for explaining a case where public metadata is “reliability: low.” [Figure 12] FIG. 13 is a diagram showing a modified example of an image file. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Each embodiment of the present invention will be described in detail below with reference to the drawings. However, the configurations described in each of the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in each embodiment. For example, each part constituting the present invention can be replaced with any configuration that can exert a similar function. In addition, any configuration may be added. In addition, any two or more configurations (features) of each embodiment can be combined.

[0010] Fig. 1 is a block diagram showing an example of a hardware configuration when the image processing device is applied to a digital camera. As shown in Fig. 1, the digital camera 100 has a barrier 10, a photographing lens 11, a shutter 12, an image sensor 13, a timing generator 14, an A / D converter 15, an image processing section 20, a D / A converter 21, a memory control section 22, and a display section 23. The digital camera 100 also has an image display memory 24, a memory 25, an exposure control section 40, a focus control section 41, a zoom control section 42, a barrier control section 43, a flash 44, and a system control section 50. The digital camera 100 also has a non-volatile memory 51, a shutter button 60, an operation section 63, a metadata generating and analyzing section 70, a power supply 80, a power supply control section 81, a card controller 90, and an external recording medium 91.

[0011] The barrier 10 is a protective member that covers the imaging section (image acquisition means) 101 including the photographing lens 11, the shutter 12, and the image sensor 13 to prevent the imaging section 101 from being soiled or damaged. The operation of the barrier 10 is controlled by the barrier control section 43. The photographing lens 11 forms an optical image on the imaging surface of the image sensor 13. The shutter 12 has an aperture function. The image sensor 13 is composed of, for example, a CCD or CMOS sensor. The image sensor 13 converts the optical image formed on the imaging surface by the shutter 12 and the photographing lens 11 into an electric signal. The A / D converter 15 converts the analog image signal output from the image sensor 13 into a digital image signal. The digital image signal converted by the A / D converter 15 is written into the memory 25 as so-called RAW image data. In addition, development parameters corresponding to each RAW image data are generated based on information at the time of shooting. The development parameters are written into the memory 25. The development parameters include various parameters used in image processing for recording in the JPEG format, such as exposure setting, white balance, color space, contrast, etc. The timing generator 14 supplies clock signals and control signals to the image sensor 13, the A / D converter 15, and the D / A converter 21. The operation of the timing generator 14 is controlled by the memory control unit 22 and the system control unit 50.

[0012] The image processing unit 20 performs various image processing such as predetermined pixel interpolation processing, color conversion processing, correction processing, and resizing processing on data from the A / D converter 15 and data from the memory control unit 22. The image processing unit 20 also performs predetermined image processing and arithmetic processing using image data obtained by imaging in the imaging unit 101. The arithmetic processing result is provided to the system control unit 50. The system control unit 50 performs AF (autofocus) processing, AE (auto exposure) processing, and EF (flash pre-flash) processing by controlling the exposure control unit 40 and the focus control unit 41 based on the arithmetic processing result in the image processing unit 20. The image processing unit 20 also performs AWB (auto white balance) processing based on the arithmetic processing result. The image processing unit 20 also reads image data stored in the memory 25 and performs compression processing and decompression processing. Examples of compression processing include JPEG, MPEG-4 AVC, HEVC (High Efficiency Video Coding), and lossless compression of uncompressed RAW data. The image processing unit 20 then writes the image data after each process has been completed to the memory 25. The image processing unit 20 also performs editing processing of various image data based on the results of the calculation processing. Examples of editing processing include trimming and resizing. Trimming is a process of adjusting the display range and size of an image by hiding unnecessary parts around the image data. Resize is a process of changing the size by enlarging or reducing the image data and display elements on the screen. The image processing unit 20 can also perform RAW development to create image data by adding image processing such as color conversion to data that has been compressed or expanded, and converting it to JPEG format. The image processing unit 20 can also perform video clipping processing to clip out a designated frame of a video format such as MPEG-4, convert it to JPEG format, and save it. The image processing unit 20 also performs processing such as superimposing an OSD (On-Screen Display) such as a menu or arbitrary characters on the image data displayed on the display unit 23.Furthermore, the image processing unit 20 performs subject detection processing to detect the subject itself present in the image data and the area including the subject based on the image data and distance information from the subject obtained from the image sensor 13 at the time of shooting, etc. Information that can be detected by the subject detection processing includes, for example, the position, size, inclination, and likelihood of the subject in the image.

[0013] The memory control unit 22 controls the operations of the A / D converter 15, the timing generator 14, the image processing unit 20, the image display memory 24, the D / A converter 21, and the memory 25. The RAW image data generated by the A / D converter 15 is written to the image display memory 24 or the memory 25 via the image processing unit 20 and the memory control unit 22, or directly via the memory control unit 22. The image data written to the image display memory 24 is displayed on the display unit 23 via the D / A converter 21. The display unit 23 is composed of, for example, an LCD composed of TFTs. By sequentially displaying the image data on the display unit 23, an electronic finder function for displaying live images can be realized. The memory 25 stores still images and moving images captured by the imaging unit 101. The memory 25 has a storage capacity sufficient to store a predetermined number of still images and moving images for a predetermined time. The memory 25 can also be used as a working area for the system control unit 50. The exposure control unit 40 controls the aperture operation of the shutter 12. The exposure control unit 40 also has a flash dimming function by working in conjunction with the flash 44. The focus control unit 41 drives a focus lens (not shown) included in the photographing lens 11 based on instructions from the system control unit 50. This driving performs focus adjustment. The zoom control unit 42 drives a zoom lens (not shown) included in the photographing lens 11. This driving performs zooming. The flash 44 has an AF assist light projection function and a flash dimming function.

[0014] The system control unit 50 controls the entire digital camera 100. The nonvolatile memory 51 is an electrically erasable and recordable nonvolatile memory. The nonvolatile memory 51 is not particularly limited, and may be, for example, an EEPROM. The nonvolatile memory 51 stores, for example, programs and map information. The programs include programs for causing a computer to execute each part and each means (a control method of an image processing device) of the digital camera 100. The shutter button 60 has a function as a shutter switch 61 (SW1) and a function as a shutter switch 62 (SW2). The shutter switch 61 is turned on during the operation of the shutter button 60. This instructs the start of operations such as AF processing, AE processing, AWB processing, and EF processing. The shutter switch 62 is turned on when the operation of the shutter button 60 is completed. This instructs the start of a series of shooting operations including exposure processing, development processing, and recording processing. In the exposure process, a signal read from the image sensor 13 is written as RAW image data to the memory 25 via the A / D converter 15 and the memory control unit 22. In the development process, the RAW image data written to the memory 25 is developed using the results of calculations in the image processing unit 20 and the memory control unit 22, and written as image data to the memory 25. In the recording process, the image data is read from the memory 25 and compressed by the image processing unit 20. Then, in the recording process, the compressed image data is stored in the memory 25, and then written to the external recording medium 91 via the card controller 90.

[0015] When recording image data on the external recording medium 91, the metadata generating and analyzing unit 70 generates metadata to be added to the image data based on information at the time of shooting. When reading image data recorded on the external recording medium 91, the metadata generating and analyzing unit 70 analyzes the metadata added to the image data. The metadata is, for example, data conforming to the Exchangeable image file format (Exif) standard. Examples of the metadata include shooting setting information at the time of shooting, image data information related to the image data, and feature information of the subject included in the image data. When recording moving image data, the metadata generating and analyzing unit 70 can also generate and add metadata for each frame.

[0016] The operation unit 63 has operation members such as various buttons and a touch panel. Specifically, the operation unit 63 includes a power button, a menu button, a mode switch for switching between a shooting mode, a playback mode, and other special shooting modes, a cross key, a set button, a macro button, a multi-screen playback page break button, and the like. The operation unit 63 also includes a flash setting button, a single shot / continuous shot / self-timer switching button, a menu movement + (plus) button, a menu movement - (minus) button, a shooting image quality selection button, an exposure compensation button, a date / time setting button, and the like. The power source 80 is composed of, for example, a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, or an AC adapter, and the like. The power source control unit 81 supplies the power supplied from the power source 80 to each unit of the digital camera 100. The card controller 90 is detachably connected to an external recording medium 91, and transmits and receives data to and from the external recording medium 91 in the connected state. The external recording medium 91 records still images and videos taken by the digital camera 100. The external recording medium 91 is not particularly limited, and may be, for example, a memory card.

[0017] FIG. 2 is a diagram showing an example of an image file. The image file 300 shown in FIG. 2 is an image file processed by the digital camera 100. In this embodiment, the image file 300 is a file defined by the box file format of JPEG XT Part 3, and a box-based file format in which data can be written arbitrarily is specified. PEG XT Part 3 is an extension function conforming to the JPEG (Joint Photographic Experts Group) standard. The image file 300 includes a first storage area 301, a second storage area 304, and image data 307. The image data 307 is data of a captured image captured (obtained) by the imaging unit 101. The image data 307 is stored in the image file 300 as image data (compressed and encoded data) of a JPEG file, for example.

[0018] The first storage area 301 is a private metadata storage area that stores private metadata 303. Examples of the private metadata 303 include metadata related to a captured image (e.g., data related to shooting conditions) and metadata specific to the manufacturer of the digital camera 100 (e.g., data related to a serial number). In this embodiment, an APP1 area is used as the first storage area 301. The APP1 area represents an application marker segment 1, and is an area that stores auxiliary information for an image (Exif: Exchangeable image file format). This APP1 area includes a maker note 302 in which metadata can be written in a format that is private in principle. The private metadata 303 is stored in the maker note 302.

[0019] The second storage area 304 is a public metadata storage area that stores public metadata 306. The public metadata 306 is metadata that can be made public by converting the private metadata 303 as described later. In this embodiment, the APP11 area is used as the second storage area 304. The APP11 area represents an application marker segment 11, and includes a JUMBF_BOX area 305 that stores the public metadata 306 in a Box format of XML or JSON format by a method conforming to the ISO / IEC 19566-5 (JUMBF) standard. The storage method in the Box format is shown in the ISO / IEC 15444-1 (JPEG 2000 Part 1) and ISO / IEC 18477-3 (JPEG-XT Part 3) standards.

[0020] FIG. 3 is a diagram showing a modified example of an image file. As shown in FIG. 3, an image file 400 includes a first storage area 401, a second storage area 404, and image data 409. The image data 409 is data of a captured image (image) captured by the imaging unit 101, similar to the image data 307. The first storage area 401 uses an APP1 area, similar to the first storage area 301. The first storage area 401 includes a maker note 402 that stores private metadata 403. The second storage area 404 uses an APP11 area, similar to the second storage area 304. The second storage area 404 includes an area 405 that stores public metadata 406 in a Box format of XML or JSON format, and an area 407 that stores public metadata 408 in a Box format of XML or JSON format, similar to the area 405. Thus, the image file 400 includes two areas for storing public metadata, but the number of areas is not limited to two, and may be, for example, three or more.

[0021] An example of the public metadata will be described with reference to FIG. 4 and FIG. 5. FIG. 4 and FIG. 5 are diagrams relating to the face area of ​​the subject included in the image data. FIG. 4(a) is a diagram when the face area of ​​the image data shown in FIG. 5(a) is described in JSON format. In the photographed image 501 shown in FIG. 5(a), a face area 503 including the face 502 of the subject is shown as a rectangle. In FIG. 4(a), the face area 503 is expressed by "rectangle" indicating the face area 503, the coordinates [26, 120] of the start point of the face area 503, and the coordinates [46, 140] of the end point of the face area 503. Also, in FIG. 4(a), "object:face" expresses that the face area 503 is an area including the face 502. FIG. 4(b) is a diagram for explaining a layer structure. By using the layer structure, it is possible to separate data hierarchies. For example, in a captured image 504 shown in Fig. 5(b), a face region 506 including a subject's face 505 and a face region 508 including a subject's face 507 are each shown as a rectangle. Fig. 4(b) shows the detection results of face region detection performed by two AI engines (AI-A and AI-B) on this captured image 504. As shown in Fig. 4(b), AI-A detects only one face region, i.e., face region 506 or face region 508, and AI-B detects two face regions, i.e., face region 506 and face region 508.

[0022] Fig. 6 is a flow chart showing the processing executed by the digital camera shown in Fig. 1. As shown in Fig. 6, in step S701, the system control unit 50 of the digital camera 100 judges whether or not there is an APP11 area in the image file, i.e., the second storage area. If it is judged in step S701 that there is an APP11 area, the processing ends. On the other hand, if it is judged in step S701 that there is no APP11 area, the processing proceeds to step S702.

[0023] In step S702, the system control unit 50 reads out the private metadata stored in APP1, i.e., the first storage area. As described above, the private metadata is metadata related to the captured image, metadata unique to the manufacturer of the digital camera 100, etc.

[0024] In step S703, the system control unit 50 (metadata creating means) executes a conversion process for converting user settings. In the conversion process in step S703, the private metadata can be converted to create public metadata (metadata creating step). The conversion process will be described with reference to Figs. 7, 8, and 9. Fig. 7 is a flowchart showing processes that can be used in each step (subroutine) of steps S703 to S706 in the flowchart shown in Fig. 6. Fig. 8 is a diagram showing an example of an image file. Fig. 9 is a diagram for explaining a case where the public metadata is "reliability: high". As shown in Fig. 7, in step S801, the system control unit 50 converts the private metadata stored in the maker note (first storage area) as it is, that is, converts the description format for public use. Then, the system control unit 50 judges whether the converted metadata can be made public metadata. If it is judged that the metadata can be made public metadata as a result of the judgment in step S801, the process proceeds to step S802. On the other hand, if it is determined in step S801 that the metadata cannot be made public, the process proceeds to step S805.

[0025] As shown in FIG. 8(a), an image file 900 includes a first storage area 901 and image data 906. In the first storage area 901, private metadata 903 is stored in an ExifIFD 902, and private metadata 905 is stored in a maker note 904. The private metadata 905 includes, for example, user settings and AF information, but is not limited thereto, and may include at least one of these pieces of information. An image file 910 shown in FIG. 8(b) is an image file obtained by performing a conversion process on the image file 900. The image file 910 includes the first storage area 901 and image data 906, and further includes a second storage area 911. The second storage area 911 includes a JUMBF_BOX area (first area) 912, a JUMBF_BOX area (second area) 914, and a JUMBF_BOX area (third area) 916. The JUMBF_BOX area 912 stores public metadata 913 with high reliability. The JUMBF_BOX area 914 stores public metadata 915 with medium reliability. The JUMBF_BOX area 916 stores public metadata 917 with low reliability. As shown in FIG. 8(b), the JUMBF_BOX area 912, the JUMBF_BOX area 914, and the JUMBF_BOX area 916 are arranged in order from the top. Therefore, the higher the reliability of the public metadata, the more the public metadata will be stored in the higher JUMBF_BOX area. This allows public metadata with high reliability to be stored first. The JUMBF_BOX area 912, the JUMBF_BOX area 914, and the JUMBF_BOX area 916 may be arranged in order from the bottom.

[0026] As shown in FIG. 9(a), the maker note stores user settings. The "user settings" are information about the shooting conditions set by the user when shooting. In this embodiment, for example, exposure control information about exposure control when acquiring a shot image and color information about the color when acquiring a shot image are included. The exposure control information is "Fv mode". This "Fv mode" is a mode in which the aperture value, shutter speed, and ISO sensitivity can be set automatically or arbitrarily. The color information is "AdobeRGB". This is a color space defined by Adobe. These pieces of information can become public metadata as valid information simply by converting the description format. Therefore, in step S801, it is determined that they can be made public metadata, and the process proceeds to step S802.

[0027] In step S802, the system control unit 50 acquires private metadata, that is, information related to user settings, from the makernote.

[0028] In step S803, the system control unit 50 converts the private metadata acquired in step S802 into a format suitable for public metadata (for example, a Box format such as XML or JSON format) to create public metadata. For example, if the private metadata "Fv mode" is expressed as "N (for example, N is a number)", public metadata can be created, for example, as a character string, by converting the "N" into a Box format. Note that in this embodiment, when the private metadata is converted directly into public metadata, the public metadata is treated as metadata having high reliability.

[0029] In step S804, the system control unit 50 (storage means) stores the public metadata created in step S803 in the image file (storage step). As described above, the public metadata created in step S803 is treated as metadata having high reliability. This public metadata having high reliability is then stored in an area in which metadata having high reliability is stored. For example, in FIG. 8(b), public metadata 913 having high reliability is stored in the JUMBF_BOX area 912 of the second storage area 911. More specifically, in the image file 1000 shown in FIG. 9(b), the user settings, which are public metadata 1003, are stored in the JUMBF_BOX area 1002 of the second storage area 1001. The JUMBF_BOX area 1002 corresponds to the JUMBF_BOX area 912. In FIG. 9(c), the user settings are "Fv mode" and "AdobeRGB", and the face area is expressed by a "rectangle" indicating the face area, the coordinates of the start point of the face area [194, 125], and the coordinates of the end point of the face area [206, 136]. In FIG. 9(c), "object: focused AF frame 1" indicates that the face area is an area including the face in frame 1 obtained in a focused state. After step S804 is executed, the process ends.

[0030] As shown in FIG. 6, in step S704 after step S703, the system control unit 50 executes a conversion process to convert the AF information into public metadata. The AF information is information about focusing when a captured image is acquired, and includes information at the time of shooting, such as an AF frame selectable position, an AF area (ranging area), and a focused AF frame (ranging point). As with the user settings, the AF information can also be generated as public metadata 1003 simply by converting the format. Therefore, this public metadata 1003 is treated as metadata having high reliability, and is stored in the JUMBF_BOX area 1002 of the second storage area 1001. After step S704 is executed, the process proceeds to step S705.

[0031] In step S705, the system control unit 50 executes a conversion process for converting the group ID information (group ID). The conversion process in step S705 will be described with reference to FIG. 7 and FIG. 10. FIG. 10 is a diagram for explaining the case where the public metadata is "reliability: medium". The "group ID information" refers to an ID or the like that is assigned so that the same ID can be assigned to a group of images captured in one continuous shooting session or a group of images captured in one shooting session such as bracket shooting, so that the images can be handled as one group image during playback or editing. This group ID information is assigned to an image file during shooting and stored in the maker note, but some types of digital cameras 100 do not assign the group ID information during shooting. When the group ID information is assigned during shooting, the process in the flowchart shown in FIG. 7 can proceed in the order of steps S801, S802, S803, and S804. On the other hand, when the group ID information is not assigned during shooting, the process is different.

[0032] 7, in step S801, the system control unit 50 judges whether or not the converted metadata can be made public by converting the format of the private metadata stored in the makernote. Here, since the group ID information is not stored in the makernote, it is judged in step S801 that the metadata cannot be made public, and the process proceeds to step S805.

[0033] In step S805, the system control unit 50 determines whether or not it is possible to generate group ID information as public metadata based on the information stored in the makernote. Note that group ID information may be generated by combining multiple pieces of information stored in the makernote. If it is determined in step S805 that group ID information can be generated, the process proceeds to step S806. On the other hand, if it is determined in step S805 that group ID information cannot be generated, the process proceeds to step S813.

[0034] As shown in Fig. 10(a), the maker note stores multiple types of mode information related to the shooting method, such as continuous shooting, high-speed continuous shooting, and super-high-speed continuous shooting, that is, the mode in which images are shot by continuous shooting. In this case, as shown in Fig. 10(b), if the shooting times of consecutive image files (IMG_0001.JPG to IMG_0005.JPG) are close to each other, the shot images can be determined to be images shot by continuous shooting. As a result, the shot images are given group ID information as public metadata, and therefore can be treated as one group image during playback and editing as described above. Therefore, as a result of the determination in step S805, it is determined that group ID information can be generated, and the process can proceed to step S806.

[0035] In step S806, the system control unit 50 judges the reliability of the information stored in the maker note. If the result of the judgment in step S806 is that the reliability is high, the process proceeds to step S807. On the other hand, if the result of the judgment in step S806 is that the reliability is low, the process proceeds to step S810. Here, the information stored in the maker note includes the shooting method. In addition to the shooting method, the recording time of each of the consecutive image files (IMG_0001.JPG to IMG_0005.JPG) is also included as a target for the reliability judgment in step S806. Both the shooting method and the recording time are confirmed information, and therefore have high reliability. Therefore, the result of the judgment in step S806 is that the reliability is high, and the process can proceed to step S807.

[0036] In step S807, the system control unit 50 acquires the shooting method, which is information stored in the maker note, that is, the information shown in FIG. 10(a).

[0037] In step S808, the system control unit 50 creates group ID information, which is public metadata, by combining the multiple pieces of information acquired in step S807. Since this group ID information is information acquired by combining the multiple pieces of information, it is treated as metadata having medium reliability, although it is lower than high reliability.

[0038] In step S809, the system control unit 50 stores the group ID information created in step S808 in the image file. Specifically, in the case of the image file 1100 shown in Fig. 10(c), the group ID information, which is public metadata 1103, is stored in the JUMBF_BOX area 1102 of the second storage area 1101. The JUMBF_BOX area 1102 corresponds to the JUMBF_BOX area 914 (see Fig. 8) in which public metadata of medium reliability is stored. Also, Fig. 10(d) shows that the group ID information is "0x12345678". After step S809 is executed, the process ends.

[0039] As shown in FIG. 6, in step S706 after step S705, the system control unit 50 executes a conversion process to convert the subject information, and the process ends. The conversion process in step S706 will be described with reference to FIG. 7 and FIG. 11. FIG. 11 is a diagram for explaining a case where the public metadata is "reliability: low". "Subject information" is metadata that indicates what kind of image data in an image file is. For example, two human HMs are shown as subjects in the image shown in FIG. 11(a). In this case, the subject information is information about each human HM in the image, and specifically, is position information (coordinates) of the face, eyes, nose, etc. of each human HM in the image. Note that, depending on the image, the subject may be, for example, an animal or a vehicle. This subject information may be added to the image file at the time of shooting and stored in the maker note, as with group ID information, or may not be added at the time of shooting. When subject information is added at the time of shooting, the process in the flowchart shown in FIG. 7 can proceed in order through steps S801, S802, S803, and S804. On the other hand, if subject information is not added at the time of shooting, the processing is different.

[0040] 7, in step S801, the system control unit 50 determines whether or not the converted metadata can be made public by converting the format of the private metadata stored in the makernote. Here, since no subject information is stored in the makernote, the result of the determination in step S801 is that the metadata cannot be made public, and the process proceeds to step S805.

[0041] In step S805, the system control unit 50 determines whether or not it is possible to generate subject information as public metadata based on the information stored in the makernote. Note that, like group ID information, subject information may be generated by combining multiple pieces of information stored in the makernote. If it is determined in step S805 that it is possible to generate subject information, the process proceeds to step S806. On the other hand, if it is determined in step S805 that it is not possible to generate subject information, the process proceeds to step S813.

[0042] In step S806, the system control unit 50 judges the reliability of the information stored in the makernote. If the reliability is judged to be high as a result of the judgment in step S806, the process proceeds to step S807. On the other hand, if the reliability is judged to be low as a result of the judgment in step S806, the process proceeds to step S810. The subject information is generated based on the information shown in FIG. 11(b) stored in the makernote and the information shown in FIG. 11(c) stored in the makernote. The information shown in FIG. 11(b) is a user setting set at the time of shooting. This user setting sets the priority of the AF tracking target, and one of "no priority", "people priority", "animal priority", and "vehicle priority" can be selected. For example, when "people priority" is selected, the focused AF frame position is likely to be "people". However, the user setting sets the tracking target to be prioritized, and when "people priority" is selected, the focused AF frame position is not necessarily "people". Therefore, since the in-focus AF frame position is estimated to be a "person," the information shown in FIG. 11(b) is estimated information and is not necessarily highly reliable. The information shown in FIG. 11(c) is the AF mode. This AF mode includes the "focus position" which is the in-focus AF frame position. The information included in the AF mode is information that is determined at the time of shooting, and is therefore highly reliable. In this way, the subject information is estimated by converting it into a combination of the information shown in FIG. 11(b), which is estimated information that is not necessarily highly reliable, and the information shown in FIG. 11(c), which is highly reliable. Therefore, as a result of the determination in step S806, it is determined that the reliability is not high, and the process proceeds to step S810.

[0043] In step S810, the system control unit 50 acquires information stored in the maker note, that is, the information shown in FIG. 11(b) and the information shown in FIG. 11(c).

[0044] In step S811, the system control unit 50 creates object information, which is public metadata, by combining the two pieces of information acquired in step S810. Since this object information is not highly reliable as described above, it is treated as metadata having low reliability.

[0045] In step S812, the system control unit 50 stores the subject information created in step S810 in the image file. Specifically, in the case of the image file 1200 shown in Fig. 11(d), the subject information, which is public metadata 1203, is stored in the JUMBF_BOX area 1202 of the second storage area 1201. The JUMBF_BOX area 1202 corresponds to the JUMBF_BOX area 916 (see Fig. 8) in which low-reliability public metadata is stored. Also, Fig. 11(e) shows that the subject information includes two face areas. After step S812 is executed, the process ends.

[0046] In step S813, the system controller 50 does not generate public metadata, and the process ends.

[0047] As described above, when digital camera 100 stores public metadata in an image file, it judges the reliability of the public metadata according to the conversion method for the private metadata (e.g., direct conversion, combined conversion, etc.), that is, for each conversion method. Then, according to the result of this judgment, it is possible to change the storage destination of the public metadata to a public metadata storage area different from the private metadata storage area. In the public metadata storage area, it is possible to make the information stored in the public metadata storage area available to the user according to its reliability.

[0048] In this embodiment, the JUMBF_Box is divided according to reliability, but this is not limiting. For example, as shown in FIG. 12, the reliability may be divided into "high reliability", "medium reliability", and "low reliability" according to a layer structure within one JUMBF_Box. In this embodiment, the private metadata and the public metadata are stored in the same image file, but this is not limiting, and for example, they may be stored in different image files. In this case, a new image file may be created in addition to the image file in which the private metadata is stored, and the public metadata may be stored in the new image file.

[0049] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. The present invention can also be realized by a process in which a program that realizes one or more functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. In the above embodiment, the image processing device is applied to a digital camera, but the device to which the image processing device can be applied is not limited to a digital camera. Applicable devices include, for example, desktop or notebook personal computers, tablet terminals, smartphones, etc.

[0050] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) An image processing device that processes an image file in which a captured image and private metadata related to the captured image are stored, comprising: a metadata generating means for converting the private metadata to generate public metadata that can be made public; a storage means for storing the public metadata in an image file, The image processing device is characterized in that, when storing the public metadata in an image file, the storage means changes the public metadata storage area in which the public metadata is stored to an area different from the private metadata storage area in which the private metadata is stored, depending on the method of conversion of the private metadata by the metadata creation means. (Configuration 2) The image processing device described in Configuration 1, wherein when storing the public metadata, the storage means judges the reliability of the public metadata depending on the method of conversion applied to the private metadata by the metadata creation means, and depending on the result of the judgment, changes the public metadata storage area to an area different from the private metadata storage area. (Configuration 3) The image processing device according to configuration 2, wherein the storage means stores the public metadata in a higher level of the public metadata storage area as the reliability of the public metadata increases. (Configuration 4) The image processing device described in configuration 2 or 3, wherein when the metadata creation means converts the private metadata directly into the public metadata, the storage means generates an area as the public metadata storage area in which metadata having high reliability is stored, and stores the public metadata in the area. (Configuration 5) The image processing device according to configuration 4, wherein the private metadata includes at least one of information regarding exposure at the time of acquisition of the captured image and information regarding focusing at the time of acquisition of the captured image. (Configuration 6) A plurality of types of the private metadata are stored in the private metadata storage area, The image processing device according to configuration 2 or 3, wherein when the metadata creation means creates the public metadata by converting the multiple types of private metadata by combining them, the storage means creates an area in which highly reliable metadata is stored as the public metadata storage area, and stores the public metadata in the area. (Configuration 7) The storage means is When the metadata creation means converts the private metadata directly into the public metadata, a first area is generated as the public metadata storage area in which metadata having high reliability is stored, and the public metadata is stored in the first area; When the metadata creating means creates the public metadata by combining and converting the plurality of types of private metadata, a second area is created as the public metadata storage area in which metadata having high reliability is stored, and the public metadata is stored in the second area; 7. The image processing device according to configuration 6, wherein the reliability of the public metadata in the first area is higher than the reliability of the public metadata in the second area. (Configuration 8) The image processing device according to configuration 6 or 7, wherein the private metadata includes at least a plurality of types of information relating to a mode for shooting the captured images by continuous shooting. (Configuration 9) A plurality of types of the private metadata are stored in the private metadata storage area, The image processing device according to configuration 2 or 3, wherein when the metadata creation means estimates and creates the public metadata by converting the multiple types of private metadata by combining them, the storage means generates an area in which metadata having low reliability is stored as the public metadata storage area, and stores the public metadata in the area. (Configuration 10) The image processing device according to configuration 9, wherein the private metadata includes at least information relating to a subject in the captured image. (Configuration 11) The image processing device according to any one of configurations 1 to 10, wherein the storage means generates an APP11 area as the public metadata storage area. (Configuration 12) The image processing apparatus according to configuration 11, wherein the storage means is capable of dividing the APP11 area for each of the conversion methods. (Configuration 13) The image processing device according to any one of configurations 1 to 12, wherein in the image file, an APP1 area is used as the private metadata storage area. (Configuration 14) The image processing device according to configuration 13, wherein the APP1 area includes a maker note. (Configuration 15) The image processing device according to any one of configurations 1 to 14, further comprising an image acquisition means for acquiring the photographed image. (Method 1) A method for controlling an image processing device that processes an image file in which a captured image and private metadata related to the captured image are stored, comprising the steps of: a metadata creation step of converting the private metadata to create public metadata that can be made public; storing the public metadata in an image file; A control method for an image processing device, characterized in that, in the storage step, when storing the public metadata in an image file, a public metadata storage area in which the public metadata is stored is changed to an area different from a private metadata storage area in which the private metadata is stored, depending on a method of conversion applied to the private metadata in the metadata creation step. (Program 1) A program for causing a computer to execute each means of the image processing device according to any one of configurations 1 to 10. [Explanation of symbols]

[0051] 50 System control section 100 Digital Camera 101 Imaging unit (image acquisition means) 300 image files 301 First storage area 303 Private Metadata 304 Second storage area 306 Public Metadata 307 Image data

Claims

1. An image processing device that processes an image file in which a captured image and private metadata related to the captured image are stored, comprising: a metadata generating means for converting the private metadata to generate public metadata that can be made public; a storage means for storing the public metadata in an image file, The image processing device is characterized in that, when storing the public metadata in an image file, the storage means changes the public metadata storage area in which the public metadata is stored to an area different from the private metadata storage area in which the private metadata is stored, depending on the method of conversion of the private metadata by the metadata creation means.

2. 2. The image processing device according to claim 1, wherein, when storing the public metadata, the storage means judges the reliability of the public metadata depending on the method of conversion applied to the private metadata by the metadata creation means, and changes the public metadata storage area to an area different from the private metadata storage area depending on the result of the judgment.

3. 3. The image processing apparatus according to claim 2, wherein the storage means stores the public metadata in a higher public metadata storage area as the reliability of the public metadata increases.

4. 3. The image processing device according to claim 2, wherein when the metadata creation means converts the private metadata directly into the public metadata, the storage means generates an area in which highly reliable metadata is stored as the public metadata storage area, and stores the public metadata in said area.

5. The image processing device according to claim 4 , wherein the private metadata includes at least one of information related to exposure when the photographed image was acquired and information related to focusing when the photographed image was acquired.

6. a plurality of types of the private metadata are stored in the private metadata storage area; 3. The image processing device according to claim 2, wherein when the metadata creation means creates the public metadata by combining and converting the multiple types of private metadata, the storage means creates an area in which highly reliable metadata is stored as the public metadata storage area, and stores the public metadata in the area.

7. The storage means includes: When the metadata creation means converts the private metadata directly into the public metadata, a first area is generated as the public metadata storage area in which metadata having high reliability is stored, and the public metadata is stored in the first area; When the metadata creating means creates the public metadata by combining and converting the plurality of types of private metadata, a second area is created as the public metadata storage area in which metadata having high reliability is stored, and the public metadata is stored in the second area; The image processing apparatus according to claim 6 , wherein the reliability of the public metadata in the first area is higher than the reliability of the public metadata in the second area.

8. 7. The image processing apparatus according to claim 6, wherein the private metadata includes at least a plurality of types of information relating to a mode for shooting the photographed images by continuous shooting.

9. a plurality of types of the private metadata are stored in the private metadata storage area; 3. The image processing device according to claim 2, wherein when the metadata creation means estimates and creates the public metadata by converting the multiple types of private metadata by combining them, the storage means generates an area in which metadata having low reliability is stored as the public metadata storage area, and stores the public metadata in the area.

10. The image processing device according to claim 9 , wherein the private metadata includes at least information relating to a subject in the captured image.

11. 2. The image processing apparatus according to claim 1, wherein the storage means generates an APP11 area as the public metadata storage area.

12. 12. The image processing apparatus according to claim 11, wherein said storage means is capable of dividing said APP11 area for each of said conversion methods.

13. 2. The image processing apparatus according to claim 1, wherein an APP1 area is used as the private metadata storage area in the image file.

14. The image processing device according to claim 13 , wherein the APP1 area includes a maker note.

15. The image processing apparatus according to claim 1 , further comprising an image acquisition unit for acquiring the photographed image.

16. 1. A method for controlling an image processing device that processes an image file in which a captured image and private metadata related to the captured image are stored, comprising: a metadata creation step of converting the private metadata to create public metadata that can be made public; storing the public metadata in an image file; A control method for an image processing device, characterized in that, in the storage step, when storing the public metadata in an image file, a public metadata storage area in which the public metadata is stored is changed to an area different from a private metadata storage area in which the private metadata is stored, depending on a method of conversion applied to the private metadata in the metadata creation step.

17. 2. A program for causing a computer to execute each of the means of the image processing apparatus according to claim 1.

Images