Image file creation method and image file creation device

Abstract

The invention provides an image file creation method and an image file generation device, which are used for more effectively utilizing an attached image of attached information as a main image. The image file creation method of the present invention creates an image file including a main image and an additional image as additional information relating to the main image, the image file creation method including: a type reception step of receiving a type of the additional image by associating with a correction process for the main image; and an acquisition step for acquiring, on the basis of the type, an additional image that does not change in accordance with the change in the main image caused by the correction process.

Description

Technical Field

[0001] One embodiment of the present invention relates to an image file creation method and an image file creation device. Background Art

[0002] Sometimes, an image file stores an auxiliary image such as a thumbnail image as auxiliary information related to the main image together with the main image (for example, see Patent Document 1). For example, the auxiliary image is used to easily confirm the content of the main image like a thumbnail image.

[0003] Previous technical literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2009-20266 Summary of the invention

[0006] Technical issues to be solved by the invention

[0007] There is a demand for improving the utilization value of supplementary images. For example, if correction is performed on a main image as an original image of a supplementary image, this fact can be understood from the supplementary image, and the supplementary image can be used more effectively.

[0008] An object of one embodiment of the present invention is to solve the above-mentioned problems of the prior art and to provide an image file creation method and an image file creation device for more effectively utilizing an auxiliary image as auxiliary information of a main image.

[0009] Means for solving technical problems

[0010] In order to achieve the above-mentioned purpose, an image file creation method involved in one embodiment of the present invention creates an image file including a main image and an auxiliary image as auxiliary information related to the main image, and the image file creation method includes: a type receiving process, receiving the type of the auxiliary image by establishing a corresponding association with the correction processing for the main image; and an acquisition process, acquiring the auxiliary image that will not change corresponding to the change of the main image caused by the correction processing according to the type.

[0011] Furthermore, the above-mentioned acquisition step may be implemented when the correction process is performed on the main image.

[0012] Furthermore, the correction processing described above may be executed within the imaging device that captured the main image.

[0013] Furthermore, in the category receiving step, at least one of a first supplementary image, a second supplementary image, and a third supplementary image may be received as a category. In this case, the first supplementary image may be generated based on a first image obtained by performing a development correction process for a RAW image as a main image. The second supplementary image may be generated based on a second image obtained by performing a color correction process including color correction on the main image. The third supplementary image may be generated based on a third image obtained by performing a processing correction process on the main image to process a subject in the main image.

[0014] Furthermore, the development correction process may not include a part of the various color corrections that can be performed in the color correction process.

[0015] Furthermore, when two or more additional images among the first additional image, the second additional image, and the third additional image are acquired, the image file may include the two or more additional images as the additional images.

[0016] Furthermore, when a third image is acquired by performing processing correction on a first image or a second image which is a main image included in an image file, the main image included in the image file can be replaced from the first image or the second image to the third image.

[0017] Furthermore, the image file creation method according to one embodiment of the present invention may include: a selection acquisition step of acquiring a user's selection as to whether to acquire a third supplementary image when the processing and correction processing is performed on the main image. Then, when the processing and correction processing is performed on the main image, the acquisition step of acquiring the third supplementary image is performed according to the selection.

[0018] Furthermore, the image file creation method according to one embodiment of the present invention may include a determination step of determining whether to implement an acquisition step of acquiring a third supplementary image based on the correction content of the main image when the processing correction process is performed on the main image.

[0019] Furthermore, the incidental information may include information on the type of the incidental image.

[0020] Furthermore, the accompanying information may include information obtained by encrypting the accompanying image. In this case, the information obtained by encrypting the accompanying image may include a hash value obtained by hashing the accompanying image.

[0021] Furthermore, the supplementary image may include a first thumbnail image which does not change in accordance with a change in the main image due to the correction process and is generated by reducing all or part of the main image at the time of generating the supplementary image.

[0022] Furthermore, the supplementary image may include a second thumbnail image that changes in accordance with a change in the main image caused by the correction process and is generated by reducing the entirety or a portion of the main image.

[0023] Furthermore, an image file creation device involved in one embodiment of the present invention creates an image file including a main image and an auxiliary image as auxiliary information related to the main image. The image file creation device includes a processor, which receives the type of the auxiliary image by establishing a corresponding association with the correction processing for the main image, and the processor obtains the auxiliary image that does not change corresponding to the change of the main image caused by the correction processing based on the type. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a diagram showing an example of the image data structure.

[0025] Figure 2 This is an illustration of an image file.

[0026] Figure 3A This is an explanatory diagram regarding the development correction process.

[0027] Figure 3B This is a diagram for explaining the color correction process.

[0028] Figure 3C This is an explanatory diagram regarding the machining correction process.

[0029] Figure 4 It is a diagram showing a configuration example of an image file creation device according to an embodiment of the present invention.

[0030] Figure 5 This is a diagram showing an example of a screen for receiving the type of attached image.

[0031] Figure 6 This is a diagram showing the functions of the image file creation device according to the first embodiment of the present invention.

[0032] Figure 7 This is a diagram showing an example of a screen for selecting whether to obtain the third supplementary image.

[0033] Figure 8 It is a diagram showing a first example of a generation mode of an incidental image.

[0034] Fig. 9 It is a diagram showing a second example of the generation mode of the incidental image.

[0035] Fig.10 It is a diagram showing a third example of the generation mode of the incidental image.

[0036] Fig.11This is an explanatory diagram regarding the first thumbnail image.

[0037] Fig. 12A This is a diagram (part 1) showing the image file creation flow according to the first embodiment of the present invention.

[0038] Fig. 12B This is a diagram (part 2) showing the image file creation flow according to the first embodiment of the present invention.

[0039] Fig.13A This is an explanatory diagram regarding the synthesis processing of the first mode.

[0040] Fig. 13B This is an explanatory diagram regarding the synthesis processing of the second mode.

[0041] Fig. 13C This is an explanatory diagram regarding the synthesis processing of the third mode.

[0042] Fig.13D This is an explanatory diagram regarding the synthesis processing of the fourth mode.

[0043] Fig.14 This is a diagram showing the functions of the image file creation device according to the second embodiment.

[0044] Fig.15 A diagram showing an example of a screen for receiving a designation of an image generation mode.

[0045] Fig.16 This is a diagram showing an example of a screen for receiving a designation of a base image used for generating an additional image.

[0046] Fig.17 This is an explanatory diagram regarding thumbnail images generated when the image generation mode is the first mode.

[0047] Fig.18 This is an explanatory diagram regarding thumbnail images generated when the image generation mode is the second mode.

[0048] Fig.19 This is an explanatory diagram regarding thumbnail images generated when the image generation mode is the third mode.

[0049] Fig. 20 This is an explanatory diagram regarding thumbnail images generated when the image generation mode is the fourth mode.

[0050] Fig.21 This is a diagram showing the image file creation flow involved in the second embodiment.

[0051] Fig. 22 This is an explanatory diagram regarding the effectiveness of the image file creation method according to the third embodiment.

[0052] Fig.23 It is a figure which shows the accompanying image produced|generated in 3rd Embodiment.

[0053] Fig.24 This is a diagram showing the functions of the image file creation device according to the third embodiment.

[0054] Fig.25 2 is a diagram showing an example of a screen for specifying the type of a main subject.

[0055] Fig.26 This is a diagram showing an example of a screen for receiving selection of a mode for increasing the resolution or grayscale number of an accompanying image.

[0056] Fig. 27 It is a diagram showing the image file creation flow involved in the third embodiment.

[0057] Fig.28 It is a diagram showing the functions of the image file creation device involved in the fourth embodiment.

[0058] Fig.29 It is a figure which shows the accompanying image produced|generated in 4th Embodiment.

[0059] Fig.30 This is a diagram showing an example of a screen for receiving a setting for increasing the resolution of an accompanying image.

[0060] Fig.31 This is a diagram showing the outline of generating an incidental image in the fourth embodiment.

[0061] Fig.32 This is a diagram showing the structure of an image file created in the fourth embodiment.

[0062] Fig.33 It is a diagram showing the image file creation flow involved in the fourth embodiment. DETAILED DESCRIPTION

[0063] The specific embodiments of the present invention are described. However, the embodiments described below are only examples for easy understanding of the present invention and do not limit the present invention. That is, the present invention can be changed or improved from the embodiments described below as long as it does not deviate from its purpose. In addition, the present invention includes its equivalents.

[0064] Furthermore, in this specification, the concept of “device” includes a single device that performs a specific function, and also includes a combination of a plurality of devices that exist independently of each other and cooperate (coordinate) to perform a specific function.

[0065] Furthermore, in the present specification, a "user" is a person who uses the image file creation device of the present invention, specifically, for example, a person who uses the later-described camera device equipped with the image file creation device of the present invention. Furthermore, in this specification, a "person" refers to a subject that performs a specific behavior, including individuals, groups, legal persons such as enterprises, and groups, and further, may also include computers and devices that constitute artificial intelligence (AI). Artificial intelligence uses hardware resources and software resources to implement intelligent functions such as reasoning, prediction, and judgment. The algorithm of artificial intelligence is arbitrary, such as an expert system, case-based reasoning (CBR: Case-Based Reasoning), a Bayesian network, or an inclusive architecture.

[0066] Furthermore, in this specification, “image” refers to data (image data) indicating the grayscale of each pixel, and a main example is still image data, but it may also be moving image data composed of a plurality of frame images.

[0067] <<About the outline of specific embodiments of the present invention>>

[0068] One embodiment of the present invention relates to an image file creation method and an image file creation device.

[0069] The image file is created by, for example, a well-known camera device such as a digital camera. Alternatively, the image file can be created by an information processing terminal based on an image read from a camera device or an image received from a device on a network. The image file is created according to the file format adopted by the device that creates the image file. Examples of file formats include JPEG (Joint Photographic Experts Group), Tiff (Tagged Image File Format), GIF (Graphics Interchange Format), BMP (Microsoft Windows Bitmap Image), PNG (Portable Network Graphics), and HEIF (High Efficiency Image File Format). The file format is reflected in the data structure of the image file. For example, in JPEG, such as Figure 1 As shown, the image file starts with a marker segment of SOI (Start of Image) and ends with a marker segment of EOI (End of Image).

[0070] like Figure 2As shown, the image file includes a main image MP and accompanying information AD. The main image MP is an image that becomes the main image in the image file, for example, an image with the largest resolution or data capacity in the image file. The main image MP is obtained, for example, by photographing a subject with a camera. Specifically, by the user pressing a release button of the camera, the camera element of the camera generates and outputs an analog image signal using a light signal received from the subject. The analog image signal is converted into a RAW image that is a digital image. Correction processing such as γ correction is performed on the RAW image, and the corrected RAW image is compressed in a specified compression method. Thus, the RAW image is developed to create compressed image data representing the developed image. In addition, the number of grayscales (number of bits) of the RAW image can be greater than the number of grayscales (number of bits) of the developed image.

[0071] The subject in the main image MP is an object to be photographed that exists within the imaging range of the imaging device, and is not limited to a specific tangible object, but also includes intangible objects such as scenery, scenes, or patterns.

[0072] Furthermore, the main image MP changes due to the correction processing. Correction of an image is data processing that changes the content, color, or image quality of the image. The content of the image refers to the subject and background of the image and the viewing angle of the image. The viewing angle is the range of data processing for displaying or depicting an image, and the range is defined as a two-dimensional coordinate space with two axes orthogonal to each other as coordinate axes. The color of an image is represented by the grayscale (color scale) of multiple pixels that constitute the image. The image quality is determined by many factors including resolution, number of grayscales, color gamut, spatial frequency, and dynamic range.

[0073] The correction process may be executed in the imaging device that has captured the main image MP, or may be executed by an information processing terminal that has read the main image MP from the imaging device. Furthermore, the correction process may be executed in response to an operation performed by a user, or may be automatically executed by the function of the imaging device or the information processing terminal. In addition, the main image MP that is the target of the correction process may include a RAW image before development.

[0074] In one embodiment of the present invention, a development correction process, a color correction process, or a processing correction process can be performed on the main image MP. Figure 3A As shown in FIG. 1 , the development correction process is a correction process for development performed on a RAW image as a main image. Specifically, a correction process such as gamma correction for developing a RAW image to make it displayable corresponds to the development correction process.

[0075] Hereinafter, an image obtained by executing the development correction process on the RAW image as the main image is referred to as a “developed image.” The developed image corresponds to the first image of the present invention.

[0076] The color correction process is a correction process including a plurality of color corrections, and the color correction process is performed on the main image MP, for example, the color correction process is performed on the developed image. In the main image MP on which the color correction process is performed, Figure 3B As shown, the color of the correction part changes from the stage before correction. Here, the aforementioned development correction process does not include some of the various color corrections that can be performed in the color correction process. In other words, the development correction process is a correction process that is simpler than the color correction process.

[0077] Examples of color correction include white balance correction, gradation correction, and color reproduction correction. Color reproduction correction is a color correction that adjusts the hue, grayscale, saturation, and contrast of an image according to a reproduction style selected by a user from among multiple reproduction styles related to the color of the image. Color reproduction styles are classified based on the overall impression or atmosphere of the image, and multiple styles are prepared in advance on the device side, and the user can select any one of them.

[0078] The correction content to be performed in each color correction can be determined, for example, based on the user's operation and the shooting conditions. For example, if the user selects white balance correction, white balance correction can be performed according to the light source during shooting. Alternatively, the correction content in the color correction can also be determined only based on the user's operation.

[0079] Hereinafter, an image obtained by performing a color correction process on a main image is referred to as a “color-corrected image.” The color-corrected image corresponds to the second image of the present invention.

[0080] The processing correction process is a correction process for processing the subject in the main image. The processing correction process is performed on the main image MP, for example, the processing correction process is performed on the developed image or the color correction image. The processing correction process includes a cropping process for extracting a region containing the subject (hereinafter, the subject region) from the main image and a process for adding characters or illustrations to the main image of the correction target. Figure 3C As shown, the processing and correction process may include processing to synthesize a part or all of another image with the main image.

[0081] Hereinafter, an image obtained by performing the modification correction process on the main image is referred to as a “modified image.” The modified image corresponds to the third image of the present invention.

[0082] In one embodiment of the present invention, the number of main images MP included in one image file is one, and the main image MP corresponds to any one of the above-mentioned developed image, color correction image, or processed image. Therefore, when a color correction process is performed on a developed image that is the main image MP included in the image file, the main image MP in the image file is replaced (overwritten) from the developed image to the color correction image. Furthermore, when the main image MP included in the image file is a developed image or a color correction image and a processing correction process is performed on the main image MP, the main image MP in the image file is replaced from the developed image or the color correction image to the processed image.

[0083] Furthermore, when the main image MP is changed by the correction process, the main image MP before correction (before replacement) may be deleted from the image file, or the main image MP before correction may remain in the image file as a backup image (thumbnail image).

[0084] The supplementary information AD is information related to the main image MP. Specifically, the supplementary information AD can be written in the header area of ​​the image file. For example, in JPEG XT Part 3, which is a type of JPEG, Figure 1 As shown in FIG. 1 , marker segments “APP1” and “APP11” are provided as header areas where supplementary information can be written. Supplementary information related to the shooting date and time, shooting location, shooting conditions, etc. of the main image MP is stored in “APP1” as tag information. “APP11” includes a JUMBF (JPEG Universal Metadata box format) box as a storage area for metadata, specifically, a JUMBF1 and JUMBF2 box. As a standard related to supplementary information in JPEG, for example, Exif (Exchangeable image file format) can be cited.

[0085] like Figure 1 As shown in FIG. 1 , the APP 11 stores a thumbnail image as supplementary information. The thumbnail image is equivalent to the supplementary image, which is created by thinning out the main image MP and has a resolution lower than that of the main image MP. That is, by setting the resolution lower than that of the main image MP, a thumbnail image such as the one shown in FIG. 1 is generated. Figure 2 The thumbnail image TP shown is composed of a reduced image of the main image MP, and the thumbnail image TP is stored in the APP 11 as supplementary information.

[0086] In one embodiment of the present invention, a thumbnail image TP is acquired in response to a change in the main image MP caused by the correction process. Specifically, a thumbnail image TP based on the changed main image MP is generated in conjunction with the change in the main image MP. Specifically, a thumbnail image TP based on a developed image (hereinafter referred to as a developed thumbnail image) is generated in response to a change in the main image MP (specifically, a RAW image) caused by the development correction process. Furthermore, a thumbnail image TP based on a color correction image (hereinafter referred to as a color correction thumbnail image) is generated in response to a change in the main image MP caused by the color correction process. Furthermore, a thumbnail image TP based on a processed image (hereinafter referred to as a processed thumbnail image) is generated in response to a change in the main image MP caused by the processing correction process.

[0087] The developed thumbnail image, the color-corrected thumbnail image, and the processed thumbnail image represent the types of the accompanying images, and correspond to the first accompanying image, the second accompanying image, and the third accompanying image of the present invention, respectively. In one embodiment of the present invention, the user can specify which type of thumbnail image (accompanying image) to obtain. Then, the thumbnail image of the type specified by the user is obtained, and the obtained thumbnail image is stored in the image file as the accompanying information of the main image MP.

[0088] Here, in one embodiment of the present invention, generating a thumbnail image is equivalent to acquiring a thumbnail image. However, this is not limited to this, and a plurality of thumbnail images may be generated in advance, and a thumbnail image stored in an image file may be selected from the plurality of thumbnail images to acquire the thumbnail image.

[0089] In addition, the thumbnail image TP may be an image obtained by reducing the entire main image MP. Alternatively, the thumbnail image TP may be an image obtained by reducing a portion of the main image MP, for example, an extracted image of an area in which the main subject is extracted. The main subject refers to a subject selected from the subjects in the main image MP, for example, based on the positional relationship with the focus position of the camera device during shooting or the user's line of sight position. Furthermore, the main subject may be selected based on image quality such as resolution (specifically, the degree of blur or shaking, etc.) or clarity in the main image MP. Alternatively, a subject specified by the user may be selected as the main subject in the main image MP.

[0090] The compression method of the thumbnail image TP may be the same as or different from that of the main image MP on which it is based. The image on which the thumbnail image TP is based (hereinafter also referred to as the original image) refers to the main image MP at the time when the thumbnail image TP is generated.

[0091] Furthermore, the number of thumbnail images TP that can be stored in one image file only needs to be one or more, and multiple thumbnail images TP may be stored in one image file. That is, when two or more thumbnail images of a developed thumbnail image, a color-corrected thumbnail image, and a processed thumbnail image are generated, the image file may include the two or more thumbnail images. Alternatively, the number of thumbnail images TP that can be stored in an image file may be only one.

[0092] like Figure 2 As shown, the supplementary information AD may store management information related to the thumbnail image TP. The management information may include, for example, information related to the creation history (creation date and time) of the thumbnail image and information related to the type of the thumbnail image. The information related to the type of the thumbnail image is information indicating whether the thumbnail image stored in the image file is a developed thumbnail image, a color-corrected thumbnail image, or a processed thumbnail image. The information may be text information describing the type of the thumbnail image, or may be information consisting of an identification symbol corresponding to the type of the thumbnail image.

[0093] And, if Figure 2 As shown, the supplementary information AD may include information that encrypts the thumbnail image for the purpose of preventing tampering of the thumbnail image. Specifically, the supplementary information AD may include a hash value obtained by hashing the thumbnail image. The information stored in the image file for the purpose of preventing tampering is not limited to the hash value, and may also be information that encrypts or encodes the thumbnail image by a method other than hashing.

[0094] And, if Figure 2 As shown in FIG. 1 , the supplementary information AD may include information on the type of the main image MP stored in the image file. The information indicates whether the main image MP is a developed image, a color-corrected image, or a processed image.

[0095] <<Configuration example of image file creation device according to one embodiment of the present invention>>

[0096] like Figure 4 As shown, an image file creation device (hereinafter referred to as image file creation device 10 ) according to an embodiment of the present invention includes a processor 11 , a memory 12 , and a communication interface 13 .

[0097] The processor 11 is composed of, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or a TPU (Tensor Processing Unit). The memory 12 is composed of, for example, a semiconductor memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The communication interface 13 is composed of, for example, a network interface card or a communication interface board.

[0098] A program for creating an image file (hereinafter referred to as an image file creation program) is stored in the memory 12. The image file creation program is a program for causing the processor 11 to implement each step in the image file creation method described below. The image file creation program can be obtained by reading from a computer-readable recording medium or by downloading from a communication network such as the Internet or an intranet.

[0099] The image file creation device 10 can communicate with other devices through the communication interface 13 and send and receive data between the devices. Figure 4 As shown, the image file creation apparatus 10 further includes an input device 14 and an output device 15. The input device 14 includes a device for receiving user operations such as a touch panel and a cursor button. The output device 15 includes a display device such as a display.

[0100] The image file creation device 10 can freely access various data stored in the storage device 16. The data stored in the storage device 16 includes data required for creating an image file, such as an image, image capturing conditions, information related to image correction processing, and information related to operations received from a user. The storage device 16 stores a created image file.

[0101] In addition, the storage device 16 may be built-in or external to the image file creation device 10, or may be composed of a NAS (Network Attached Storage) etc. Alternatively, the storage device 16 may be an external device capable of communicating with the image file creation device 10 via the Internet or a mobile communication network, such as an online storage device.

[0102] In one embodiment of the present invention, Figure 4As shown, the image file creation device 10 is installed in an imaging device such as a digital camera. The mechanical structure of the imaging device (hereinafter referred to as the imaging device 20) including the image file creation device 10 is substantially the same as that of a known imaging device having a function of creating an image file.

[0103] That is, the camera 20 captures the subject according to the camera conditions set in advance to obtain a RAW image as the main image MP. The camera conditions include exposure time (exposure amount), ISO sensitivity, focus position and focal length at the time of capturing, etc. Then, the camera 20 develops the RAW image and appropriately performs color correction processing or processing correction processing on the developed image. Thus, the main image MP is obtained. In addition, the camera 20 generates supplementary information such as thumbnail images TP based on the main image MP, and creates an image file containing the main image MP and the supplementary information.

[0104] In one embodiment of the present invention, the camera device 20 may have an autofocus (AF) function that automatically focuses on a specified position within the camera range during shooting and a function of determining a focal position (AF point). The AF point is determined as a coordinate position when a reference position within the camera range is used as an origin. In addition, the camera device 20 may have a viewfinder that the user peeks into during shooting. In this case, the camera device 20 may have a function of detecting the line of sight and the position of each pupil of the user who is using the viewfinder to determine the user's line of sight position. The user's line of sight position is equivalent to the intersection position of the user's line of sight peeking into the viewfinder and the display screen in the viewfinder.

[0105] Furthermore, in one embodiment of the present invention, the imaging device 20 may have a function of changing the position of the imaging element (i.e., the imaging position) by shifting the imaging element by one pixel or half a pixel in the vertical and horizontal directions within the imaging device 20, a so-called pixel shift function. Here, the imaging position refers to the position of each pixel in the imaging element.

[0106] Furthermore, in one embodiment of the present invention, the camera device 20 may have a function of performing shooting in the live preview mode before shooting the main image MP. According to this function, for example, if the user half-presses the release button as an operation for shooting preparation, this is used as a trigger to continuously shoot the subject within the shooting range at a predetermined time interval. The image shot in the live preview mode (live preview image) is displayed in real time on the display of the camera device 20 as a live preview image. The shooting interval of the image in the live preview mode may be a constant interval, or the shooting interval may be changed according to the shooting conditions, etc.

[0107] Furthermore, the live preview images are temporarily stored in the memory 12 or the storage device 16 as images captured in the same imaging range as the main image before the main image is captured. The live preview images temporarily stored in the memory 12 or the storage device 16 may be all the live preview images captured before the main image is captured, or may be a part of the live preview images.

[0108] Furthermore, in one embodiment of the present invention, the camera device 20 may have a function of continuing to shoot in a record view mode after shooting the main image. This function allows the subject in the camera device to be photographed at a point in time when a certain time has passed since the user pressed the release button. The image shot in the record view mode (rec view image) is temporarily stored in the memory 12 or the storage device 16. In addition, with respect to the shooting in the record view mode, the subject in the shooting range may be continuously photographed at a predetermined time interval after shooting the main image. In this case, the shooting interval of the image in the record view mode may be constant, or the shooting interval may be changed according to the shooting conditions, etc.

[0109] Furthermore, in one embodiment of the present invention, the camera device 20 may have a function of analyzing the main image and detecting the subject area in the main image. As a method of detecting the subject area, for example, a known subject detection technology can be used. Furthermore, in one embodiment of the present invention, the camera device 20 may have a function of analyzing the detected subject area to determine the resolution (specifically, the degree of blur or shaking, etc.) or sharpness of the subject.

[0110] Furthermore, in one embodiment of the present invention, the camera device 20 may have a function of determining the type of the subject present in the subject area, and, when the subject is a person, may have a function of determining the expression. As a method of determining the type or expression of the subject, for example, a known subject recognition technology may be used.

[0111] Furthermore, in one embodiment of the present invention, the camera device 20 may have a function of calculating a score for a subject area (i.e., an image of the subject) based on the resolution, sharpness, and expression of the determined subject. As a method of calculating a score based on each evaluation item related to the image of the subject, for example, the technology described in Japanese Patent Publication No. 2019-118021 can be used.

[0112] <<Generation of thumbnail images>>

[0113] In one embodiment of the present invention, when a correction process is executed on the main image MP in the imaging device 20 that has captured the main image MP, the thumbnail image TP is acquired (generated). For example, when a development correction process is executed on a RAW image that is the main image MP, a developed thumbnail image is generated based on the developed main image, i.e., the developed image.

[0114] Furthermore, in one embodiment of the present invention, it is possible to set which correction processing is performed on the main image MP when generating the thumbnail image, in other words, it is possible to set the type of thumbnail image to be generated. Specifically, at the stage before the main image MP is captured, Figure 5 The setting screen shown is displayed on the display of the imaging device 20. In the setting screen, developed thumbnail images, color correction thumbnail images, and processed thumbnail images are selectably displayed as types of thumbnail images. The user specifies one or more types of thumbnail images on the setting screen.

[0115] Then, when the user specifies the type of thumbnail image and a correction process corresponding to the specified type is performed on the main image MP, a thumbnail image of the type is acquired. For example, when the user specifies "color correction thumbnail image" and a color correction process is performed on the main image MP, a color correction thumbnail image is generated and acquired.

[0116] In addition, Figure 5 In the setting screen of , the type of thumbnail image can be selectively displayed, but as long as the type of thumbnail image to be generated can be determined, the display content is not particularly limited. For example, an explanation article of each correction process or a sample image on which each correction process has been executed can be selectively displayed on the screen. In this case, when the user selects the type of any correction process and the selected type of correction process is executed on the main image MP, a thumbnail image corresponding to the correction process can be generated. In this way, the type of thumbnail image to be generated can be indirectly received by receiving the type of correction process from the user.

[0117] Furthermore, the type of thumbnail image is not limited to the type received in advance before the main image MP is captured, and may be received, for example, when or just after the correction process is executed on the main image MP. Furthermore, each time the correction process is executed on the main image MP, the user may be asked whether to generate a thumbnail image corresponding to the correction process.

[0118] Furthermore, in one embodiment of the present invention, the thumbnail image includes a first thumbnail image. The first thumbnail image is generated by reducing all or part of a main image at the time when the thumbnail image is generated. Furthermore, the first thumbnail image is an image that does not change in accordance with changes in the main image caused by a correction process when a correction process is performed on the main image as its original image. For example, a developed thumbnail image is generated as the first thumbnail image. In this case, even if a color correction process or a processing correction process is performed on the developed image as its original image after the developed thumbnail image is generated, the developed thumbnail image will not be corrected and will maintain the state at the time of creation.

[0119] By generating (acquiring) the first thumbnail image, the first thumbnail image can be used as evidence to prove the existence of the main image as its original image. That is, even if the main image MP as its original image has changed due to the correction process after the first thumbnail image is generated, the original main image MP before the correction can be confirmed by referring to the first thumbnail image. Therefore, in the case where an undesirable correction process such as tampering is performed on the main image MP, the user can discover the tampering by using the first thumbnail image generated based on the main image MP before the correction.

[0120] The structure in which the first thumbnail image is not corrected in accordance with the correction of the main image MP is not particularly limited. For example, the first thumbnail image can be made impossible to correct by storing the first thumbnail image in an area in the image file that cannot be corrected (changed), specifically, in the aforementioned header area "APP11". Furthermore, the first thumbnail image can also be made impossible to correct by storing information that limits the correction of the first thumbnail image in the aforementioned header area. Here, the information that limits the correction of the first thumbnail image can be information that encrypts the first thumbnail image (for example, a hash value, etc.).

[0121] Furthermore, in one embodiment of the present invention, the thumbnail image includes a second thumbnail image in addition to the first thumbnail image. The second thumbnail image is generated by reducing the entirety or a portion of the main image MP at the time of generating the thumbnail image. Furthermore, when a correction process is performed on the main image MP as the original image, the second thumbnail image is an image that changes in accordance with the change of the main image MP caused by the correction process. That is, the second thumbnail image is equivalent to a variable incidental image that changes in accordance with the change of the main image MP caused by the correction process.

[0122] <<Regarding the image file creation device according to each embodiment of the present invention>>

[0123] As embodiments related to the present invention, a plurality of specific examples are given to explain the functions and operations of the image file creation device. In addition, regarding the second embodiment and later, the differences from the previous embodiment are mainly explained, and the same contents are appropriately omitted.

[0124] Furthermore, the embodiment of the present invention may be a mode in which two or more embodiments (first to fourth embodiments) described below are combined.

[0125] [About the first embodiment]

[0126] like Figure 6 As shown, the image file creation device 10 according to the first embodiment of the present invention includes a type receiving unit 21, a first generating unit 22, a correction unit 23, a determination unit 24, a selection acquisition unit 25, a second generating unit 26, and a creation unit 27. These functional units are implemented by the cooperation of the hardware device of the image file creation device 10 and the software including the image file creation program. In addition, some functions can be implemented by using artificial intelligence (AI). The following describes each functional unit.

[0127] (Type receiving unit)

[0128] The type receiving unit 21 receives the type of the thumbnail image TP in association with the correction process for the main image MP. Figure 5 The setting screen of the image pickup device 20 is displayed on the display of the image pickup device 20. On this screen, a development thumbnail image, a color correction thumbnail image, and a processed thumbnail image are selectively displayed as the types of thumbnail images. If the user specifies at least one of these, the type receiving unit 21 receives the specified result as the type of the thumbnail image. At this time, the user can specify two or more types from the three types of thumbnail images. In this case, the type receiving unit 21 receives the two or more types.

[0129] The timing at which the type receiving unit 21 receives the type of the thumbnail image can be arbitrarily determined as long as it is before a thumbnail image corresponding to the type is generated. Furthermore, when the user specifies the type of correction processing for the main image MP, the type receiving unit 21 can receive the specified type of correction processing as the type of the thumbnail image.

[0130] (Generation Part 1)

[0131] The first generation unit 22 is triggered by the user fully pressing the release button of the imaging device 20, and controls each unit of the device to capture a subject existing in the imaging range of the imaging device 20 to generate a main image MP. At this time, the first generation unit 22 captures the subject according to the set imaging conditions. Thus, a main image MP corresponding to the imaging conditions is generated, specifically, a RAW image is generated.

[0132] (Calibration Department)

[0133] The correction unit 23 performs correction processing on the main image MP automatically or according to the user's instruction. For example, when a RAW image is generated as the main image MP, the correction unit 23 performs development correction on the RAW image. Also, when the user instructs color correction on the developed image as the main image MP, the correction unit 23 performs color correction processing corresponding to the user's instruction on the developed image. Also, when the user performs a processing operation on the developed image or the color correction image as the main image MP, the correction unit 23 performs processing correction processing on the developed image or the color correction image according to the user's processing operation.

[0134] In the first embodiment, the image file creation device 10 including the correction unit 23 is mounted in the imaging device 20. Therefore, in the first embodiment, the correction process by the correction unit 23 is executed in the imaging device 20 that has captured the main image MP. In this way, since the correction process is executed in the imaging device 20, the user can more easily correct the main image MP.

[0135] (Confirmation Department)

[0136] When the correction unit 23 performs the modification correction process on the main image MP, the determination unit 24 determines whether to generate (acquire) a modified thumbnail image, that is, determines whether to implement the acquisition process described later. That is, in the first embodiment, when the modification correction process is performed, the modified thumbnail image is generated only when the determination unit 24 determines to generate the modified thumbnail image. As a result, the burden of data processing can be reduced compared to the case where the modified thumbnail image is generated every time the modification correction process is performed. In addition, by suppressing the number of generated modified thumbnail images, the required data capacity can be reduced.

[0137] In the first example of the first embodiment, the determination unit 24 automatically determines whether to generate a processed thumbnail image (i.e., whether to implement the acquisition process) based on the correction content in the processing correction process. The correction content refers to the content of the subject or the angle of view of the main image MP changed by the processing correction process, specifically, the degree of change (correction amount) of the main image MP caused by the correction process. The correction content can be quantitatively determined by, for example, comparing the main image MP before and after the correction and calculating the amount of change. Then, when the correction content satisfies a predetermined condition, for example, when the correction amount is greater than a reference value, the determination unit 24 automatically determines to acquire a processed thumbnail image corresponding to the execution of the processing correction process.

[0138] Furthermore, in the second example of the first embodiment, when the modification correction process is executed on the main image MP, the determination unit 24 acquires the user's selection as to whether to generate (acquire) a modified thumbnail image and determines it.

[0139] (Select the acquisition section)

[0140] When the modification correction process is executed on the main image MP, whether to generate (acquire) the modified thumbnail image is automatically determined by the determination unit 24, or the selection acquisition unit 25 acquires the user's selection. In the latter case, when the modification correction process is executed on the main image, the display of the camera device 20 displays the modified thumbnail image. Figure 7 The selection screen shown in FIG. 1 is displayed on the selection screen. An option as to whether or not to generate a processed thumbnail image is displayed. The selection acquisition unit 25 acquires the user's selection regarding the generation of a processed thumbnail image through the above-mentioned selection screen. Then, the processed thumbnail image is generated only when the processing correction process is executed and the user selects to generate a processed thumbnail image. Thus, the burden of data processing can be reduced compared to the case where a processed thumbnail image is generated every time the processing correction process is executed. In addition, by suppressing the number of processed thumbnail images generated, the required data capacity can be reduced.

[0141] As described above, in the first embodiment, whether to generate a processed thumbnail image can be determined by the determination unit 24 based on the correction content of the processing correction process. Furthermore, whether to generate a processed thumbnail image can be determined based on the user's selection acquired by the selection acquisition unit 25. Furthermore, when determining whether to generate a processed thumbnail image, the user can specify whether to use automatic determination based on the correction content or user's selection.

[0142] (Part 2 of Generation)

[0143] The second generation unit 26 generates supplementary information related to the main image MP. The supplementary information generated by the second generation unit 26 includes a thumbnail image TP. Specifically, when the correction processing by the correction unit 23 is performed, the second generation unit 26 generates a thumbnail image TP based on the corrected main image MP in conjunction with the correction processing. In this way, the thumbnail image TP is acquired as supplementary information related to the main image MP.

[0144] The thumbnail image TP is generated by reducing all or part of the main image MP at the time of generation. When a part of the main image MP is reduced to generate the thumbnail image TP, the second generation unit 26 reduces the area specified by the user in the main image MP or the area selected on the side of the camera 20 to generate the thumbnail image TP. Alternatively, the area selected on the side of the camera 20 may be, for example, an area where the main subject of the main image MP exists.

[0145] Furthermore, in the first embodiment, the second generating unit 26 generates the thumbnail image TP according to the type of the thumbnail image received by the type receiving unit 21. For example, the type of the thumbnail image received by the type receiving unit 21 is "developed thumbnail image". In this case, Figure 8 As shown in FIG. 1 , when the development correction process is executed on the RAW image as the main image MP, the second generating section 26 generates a developed thumbnail image based on the developed image as the corrected main image MP in conjunction with the development correction process.

[0146] As a second example, the types of thumbnail images received by the type receiving unit 21 are "developed thumbnail image" and "color-corrected thumbnail image". Fig. 9 As shown in FIG. 1 , when the development correction process is executed on the RAW image as the main image MP, the second generation unit 26 generates a developed thumbnail image based on the developed image in conjunction with the development correction process. Furthermore, when the color correction process is executed on the developed image, the second generation unit 26 generates a color-corrected thumbnail image based on the color-corrected image as the main image MP after correction in conjunction with the development correction process.

[0147] As a third example, the types of thumbnail images received by the type receiving unit 21 are "color correction thumbnail image" and "processed thumbnail image". In this case, when the development correction process is performed on the RAW image, no thumbnail image (developed thumbnail image) is generated. Fig.10 As shown in FIG. 1 , when a color correction process is performed on the developed image as the main image MP, the second generating unit 26 generates a color-corrected thumbnail image based on the color-corrected image in conjunction with the color correction process. Furthermore, when a processing correction process is performed on the developed image or the color-corrected image, the second generating unit 26 generates a processed thumbnail image based on the processed image in conjunction with the color correction process.

[0148] As described above, in the first embodiment, thumbnail images TP are generated according to the type of thumbnail images received from the user. Thus, when correction processing corresponding to the type received from the user is performed on the main image MP, thumbnail images can be generated from the corrected main image MP and stored in the image file.

[0149] Furthermore, in the first embodiment, as described above, when the modification correction process is executed on the main image MP, the determination unit 24 determines whether to acquire a modified thumbnail image. Then, when the determination unit 24 determines to acquire a modified thumbnail image, the second generation unit 26 generates a modified thumbnail image.

[0150] Furthermore, in the first embodiment, as described above, when the modification correction process is executed on the main image MP, the selection acquisition unit 25 acquires the user's selection as to whether to acquire the modified thumbnail image. Furthermore, when the selection acquisition unit 25 acquires the selection of acquiring the modified thumbnail, the second generation unit 26 generates the modified thumbnail image according to the selection.

[0151] Furthermore, in the first embodiment, the first thumbnail image is generated (acquired) as the thumbnail image. When the correction process is performed on the main image MP as the original image, as shown in FIG. Fig.11 As shown, the first thumbnail image does not change in accordance with the change of the main image caused by the correction process, but maintains the image at the time of acquisition. That is, the first thumbnail image is an image with high reliability (credibility) as evidence of which image the main image MP was at the time of its generation. Therefore, even if an undesired correction such as tampering is performed on the main image MP, as long as the first thumbnail image based on the main image MP before the correction is acquired, it can be recognized that the correction has been performed.

[0152] Furthermore, in the first embodiment, the second generation unit 26 may generate a second thumbnail image that can be corrected together with the first thumbnail image. When the correction process is performed on the main image MP as the original image, the second thumbnail image changes in accordance with the change of the main image MP caused by the correction process. If such a second thumbnail image is acquired together with the first thumbnail image, it is possible to easily recognize whether the correction process is performed on the main image MP as the original image by comparing the two thumbnail images.

[0153] In addition, the present invention is not limited to the above case, and the second generating unit 26 may generate only the first thumbnail image instead of the second thumbnail image.

[0154] Furthermore, in the first embodiment, the second generating unit 26 generates management information indicating the type of the generated thumbnail image as supplementary information about the main image MP. This makes it easy to recognize what type of thumbnail image is acquired for the main image MP.

[0155] Furthermore, in the first embodiment, the second generation unit 26 generates information that encrypts the generated thumbnail image as supplementary information related to the main image MP. The information includes a hash value obtained by hashing the thumbnail image. In this way, by acquiring the information that encrypts the thumbnail image, changes and alterations of the thumbnail image itself can be discovered. As a result, the reliability (security) of the thumbnail image, especially the first thumbnail image, can be further improved.

[0156] (Creation Department)

[0157] The creation unit 27 creates an image file, and more specifically, creates an image file in a file format that is initially set or selected by the user. For example, the image file is created when the main image MP is acquired. And each time a correction process is performed on the main image MP, the main image MP in the image file is replaced with the corrected image. For example, a processed image is acquired by performing a processing correction process on a developed image or a color-corrected image that is the main image MP included in the image file. In this case, the creation unit 27 replaces the main image MP included in the image file from the developed image or the color-corrected image to the processed image.

[0158] In addition, the developed image or the color correction image which is the main image MP before replacement may be deleted or may be left in the image file.

[0159] Furthermore, when the second generation unit 26 generates supplementary information related to the main image MP, the creation unit 27 includes the generated supplementary information in the image file. Specifically, when the second generation unit 26 generates a thumbnail image TP based on the main image MP, the creation unit 27 stores the thumbnail image TP in "APP11" which is the header area of ​​the image file.

[0160] Furthermore, when the correction process is performed multiple times on the main image MP and a thumbnail image TP based on the corrected main image is newly created each time the correction process is performed, the newly generated thumbnail image TP is added to the image file. That is, in the first embodiment, when two or more thumbnail images of the developed thumbnail image, the color-corrected thumbnail image, and the processed thumbnail image are generated, the image file includes two or more thumbnail images.

[0161] The thumbnail images TP included in the image file include a first thumbnail image that does not change in response to the correction of the main image MP. Furthermore, in addition to the first thumbnail image, the thumbnail images TP in the image file may also include a second thumbnail image that is a variable supplementary image that changes in response to the correction of the main image MP. Furthermore, the image file may include information other than the thumbnail images TP generated by the second generation unit 26, specifically, information related to the type of thumbnail image and a hash value of the thumbnail image (see Figure 2 ).

[0162] (Operation Example of Image File Creation Device in First Embodiment)

[0163] Next, as an example of the operation of the image file creation device 10 in the first embodiment, an image file creation process using the device is described. In the image file creation process described below, the image file creation method of the present invention is used. That is, each step in the image file creation process described below corresponds to the constituent elements of the image file creation method of the present invention.

[0164] In addition, the following process is only an example, and part of the steps in the process may be deleted, or new steps may be added to the process, or the execution order of two steps in the process may be reversed without departing from the spirit of the present invention.

[0165] Each step in the image file creation flow according to the first embodiment is executed by the processor 11 of the image file creation device 10. Fig. 12A and 12B That is, in each step in the image file creation flow, the processor 11 executes the processing corresponding to each step in the data processing specified in the image file creation program.

[0166] In the image file creation process according to the first embodiment, first, the processor 11 performs a type receiving step (S001). Figure 5 The screen shown is displayed on the display of the camera device 20, and the type of the thumbnail image to be acquired is received in association with the correction process for the main image MP. In addition, in the type receiving step, at least one of the developed thumbnail image, the color corrected thumbnail image, and the processed thumbnail image is received as the type of the acquisition object. In addition, the type receiving step can be implemented, for example, after the camera device 20 is activated, or it can also be implemented immediately after the later-described camera step or each correction step.

[0167] Next, when the release button of the camera device 20 is pressed, the processor 11 performs a camera process at that time (S002). In the camera process, the various components of the camera device 20 are controlled to capture the subject within the camera range and acquire a main image MP. The main image MP acquired at the time of the camera process is an undeveloped RAW image. Then, along with the acquisition of the main image MP, an image file containing the main image MP is created.

[0168] Next, the processor 11 performs a development correction process in which a development correction process is performed on the RAW image obtained in the imaging process ( S003 ), thereby acquiring a developed image as a main image MP.

[0169] Then, when the processor 11 receives "developed thumbnail image" as the type of thumbnail image to be acquired in the aforementioned type receiving step (S004), the processor 11 implements the acquisition step (S005) along with the implementation of the development correction step. In this acquisition step, a developed thumbnail image is generated by the function of the camera device 20 based on all or part of the developed image. Thus, the developed thumbnail image is acquired, and the developed thumbnail image is stored in the header area of ​​the image file as the accompanying information of the main image MP. Furthermore, the acquired developed thumbnail image is the first thumbnail image, and even if the main image MP is corrected thereafter, it is stored in the image file in a manner that does not change in accordance with the change of the main image MP.

[0170] In the acquisition step, a second thumbnail image that changes in conjunction with a change in the main image MP may be acquired as a developed thumbnail image simultaneously with the first thumbnail image.

[0171] Next, when the user performs an operation for color correction on the main image MP (S006), the processor 11 performs a color correction process according to the user operation (S007). In this process, a color correction process is performed on the developed image to obtain a color correction image as the main image MP. As a result, the main image MP in the image file is replaced from the developed image to the color correction image.

[0172] Furthermore, when the processor 11 receives "color-corrected thumbnail image" as the type of thumbnail image to be acquired in the aforementioned type receiving step (S008), the processor 11 implements an acquisition step (S009) along with the implementation of the color correction step. In this acquisition step, a color-corrected thumbnail image is generated by the function of the camera device 20 based on all or part of the color-corrected image. Thus, the color-corrected thumbnail image is acquired, and the color-corrected thumbnail image is stored in the image file as incidental information of the main image MP. Furthermore, the acquired color-corrected thumbnail image is the first thumbnail image, and even if the main image MP is corrected thereafter, it is stored in the image file in a manner that does not change in accordance with changes in the main image MP.

[0173] In the acquisition step, a second thumbnail image that changes in conjunction with a change in the main image MP may be acquired as a color-corrected thumbnail image simultaneously with the first thumbnail image.

[0174] Next, when the user performs an operation for processing correction on the main image MP (S010), the processor 11 performs a processing correction step according to the user operation (S011). In this step, the processing correction process is performed on the main image MP (specifically, the developed image or the color correction image), and a processed image as the main image MP is acquired. As a result, the main image MP in the image file is replaced from the developed image or the color correction image to the processed image.

[0175] Furthermore, when the processor 11 receives "modified thumbnail image" as the type of thumbnail image to be acquired in the type receiving step (S012), the processor 11 implements the determination step and the selection acquisition step (S013, S014) along with the execution of the modification correction step.

[0176] In the determination step, the processor 11 determines the correction content of the main image MP in the modification correction step, and specifically, determines the correction amount (the degree of change in the main image MP caused by the correction) by comparing the main image MP before and after the correction. Then, the processor 11 determines whether to implement the acquisition step of acquiring the modified thumbnail image based on the determined correction amount. Specifically, when the correction amount is greater than the reference value, the processor 11 determines to implement the acquisition step of acquiring the modified thumbnail image.

[0177] In the selection acquisition process, the processor 11 will Figure 7 The display screen shown is displayed on the display of the camera 20, and the user's selection as to whether to acquire the modified thumbnail image is acquired. Then, when the user selects acquisition of the modified thumbnail image, the processor 11 executes the acquisition process of the modified thumbnail image according to the selection.

[0178] In addition, Fig. 12B In the illustrated flow, both the determination step and the selection acquisition step are performed, but the present invention is not limited to this, and only one of the determination step or the selection acquisition step may be performed.

[0179] When the processed thumbnail image is acquired according to the result of the aforementioned determination process and the selection acquisition process (S015), the processor 11 implements the acquisition process (S016). In this acquisition process, the processed thumbnail image is generated by the function of the camera device 20 based on the whole or part of the processed image. In this way, the processed thumbnail image is acquired and stored in the image file as the accompanying information of the main image MP. In addition, the acquired processed thumbnail image is the first thumbnail image, and even if the main image MP is corrected thereafter, it is stored in the image file in a manner that does not change in accordance with the change of the main image MP.

[0180] In the acquisition step, a second thumbnail image that changes in conjunction with a change in the main image MP may be acquired as a modified thumbnail image simultaneously with the first thumbnail image.

[0181] When the series of steps above are completed, the image file creation process is terminated. As described above, in the image file creation process of the first embodiment, the main image MP included in the image file is changed due to the correction process. And, when the correction process is executed, if the correction process corresponds to the type of the thumbnail image received in advance, a thumbnail image is acquired according to the type. Specifically, the thumbnail image TP is generated based on all or part of the corrected main image MP.

[0182] Furthermore, the thumbnail images TP acquired in the first embodiment include at least the first thumbnail image. Even if the first thumbnail image is subjected to correction processing with respect to the main image after it is generated, it does not change in accordance with the conversion of the main image caused by the correction processing. When the main image, which is its original image, is changed due to the correction processing, this first thumbnail image can be used as evidence to prove which image the main image before correction was. That is, in the first embodiment, a thumbnail image with excellent credibility and high utilization value can be acquired as incidental information of the main image.

[0183] [About the Second Embodiment]

[0184] The image file creation device 10X according to the second embodiment has a function of executing a synthesis process as a correction process of the main image MP. In the synthesis process, two or more images including the main image MP are synthesized to obtain a new main image MP. Hereinafter, the image before synthesis is referred to as a "base image".

[0185] In the second embodiment, as the synthesis processing mode, a mode for reproducing multiple exposures, a mode for super-resolution, a mode for HDR (High Dynamic Range), and a mode for collage can be selected. That is, in the second embodiment, when a plurality of base images are synthesized to generate a synthesized image, the user selects any one of a plurality of image generation modes, and the user's selection is received. The following describes each synthesis processing mode.

[0186] (Mode for HDR synthesis processing)

[0187] In the mode of synthesis processing for HDR (hereinafter referred to as the first mode), a plurality of base images having different shooting conditions of the camera device 20 during shooting are synthesized to generate a synthetic image. Here, the shooting condition is the exposure amount, and in the first mode, a plurality of base images having different exposure amounts during shooting are synthesized to generate a synthetic image.

[0188] To explain in more detail, one of the plurality of base images used in the first mode is an image captured with an exposure greater than a standard exposure (an exposure set to an appropriate value at that time) (overexposed). The other base images are images captured with an exposure less than the standard exposure (underexposed). Then, as Fig.13A As shown in FIG. 1 , the overexposed base image and the underexposed base image are synthesized by connecting the properly exposed portions of the respective base images. The synthesized image obtained in this way becomes an image that properly expresses the bright and dark portions within the angle of view (i.e., an image in which overexposure and underexposure are eliminated).

[0189] In addition, in the first mode, it is not limited to the case of using multiple base images with different exposure amounts. Shooting conditions other than exposure amount can also be used. Specifically, multiple images with different ISO sensitivities, focus positions and focal lengths at the time of shooting can be used as base images.

[0190] (Super-resolution synthesis processing mode)

[0191] In the super-resolution synthesis processing mode (hereinafter referred to as the second mode), a plurality of basic images captured by shifting the imaging position of the imaging element in the imaging device 20 by one pixel or half a pixel by the pixel shift function of the imaging device 20 are synthesized. Fig. 13BAs shown, the resolution of the synthesized image obtained in this way is higher than the resolution of the base image. In addition, the time interval when taking multiple base images by the pixel shift function is, for example, about 1 / 100 second. As a method for creating a synthesized image using pixel shift, for example, the method described in Japanese Patent Application Laid-Open No. 2020-096301 and Japanese Patent Application Laid-Open No. 2019-161564 is known.

[0192] (Multiple exposure synthesis mode)

[0193] In the mode of the synthesis processing for multiple exposures (hereinafter referred to as the third mode), multiple basic images captured by the camera device 20 at different locations are synthesized. For example, with respect to one of the multiple basic images, the remaining basic images are overlapped in a state where their transmittance is smaller than the normal value. The transmittance is the degree of background transparency when displaying an image, and can be freely adjusted within the range of 0 to 100% by the function of the camera device 20. Fig. 13C As shown in FIG. 2 , the synthesized image obtained as described above is an image obtained by taking the subjects of a plurality of base images taken at different locations together.

[0194] (Synthesis processing mode for collage)

[0195] In the mode of synthesis processing for collage (hereinafter referred to as the fourth mode), a plurality of basic images are synthesized in an arranged manner. Fig.13D As shown in the figure, a composite image is obtained by arranging the basic images other than the template image according to the image layout specified in the template image that becomes the background of the composite image among the multiple basic images. In the fourth mode, the basic images other than the template image are appropriately enlarged or reduced according to the size of the template image. In addition, the multiple basic images used in the fourth mode may be images selected by the user, or may be images automatically selected as images whose subjects in the images meet the specified conditions.

[0196] (About the image file of the composite image)

[0197] In the second embodiment, when a composite image is generated by the composition process, an image file is created that uses the composite image as the main image MP. The header information of the image file stores supplementary information related to the composite image as the main image MP. The supplementary information includes a thumbnail image (supplementary image).

[0198] Then, in the second embodiment, a thumbnail image TP is created based on at least one of the plurality of base images used in the synthesis process. Specifically, when a synthesized image is generated according to an image generation mode specified by a user, the thumbnail image TP is generated according to the image generation mode and based on a part or all of the plurality of base images used in the generation of the synthesized image. Thus, it is possible to understand from the thumbnail image TP which base image the synthesized image is generated from, that is, it is possible to understand the base image used as the material of the synthesized image.

[0199] In addition, the thumbnail image TP based on the base image may be generated after the synthesis process is performed, or may be generated before the synthesis process is performed. Furthermore, the thumbnail image TP based on the synthesized image may be generated together with the thumbnail image TP based on the base image.

[0200] Furthermore, the thumbnail image generated in the second embodiment is an image that does not change in accordance with the change of the composite image when the composite image changes due to color correction processing or processing correction processing. If such a thumbnail image is generated, a thumbnail image with excellent reliability and high utilization value can be obtained as incidental information related to the composite image. However, this is not limited to this. In the second embodiment, a correctable thumbnail image, that is, a thumbnail image (a changing incidental image) that changes with the change of the composite image caused by the correction processing can also be generated.

[0201] Next, refer to Fig.14 Functions of the image file creation device 10X according to the second embodiment will be described. The hardware configuration of the image file creation device 10X according to the second embodiment is the same as that of the first embodiment, and software such as an image file creation program is installed in the image file creation device 10X.

[0202] like Fig.14 As shown, the image file creation device 10X according to the second embodiment includes a third generation unit 121, a receiving unit 122, an image determination unit 123, a synthesis unit 124, a fourth generation unit 125, and a creation unit 126. These functional units are implemented by the cooperation of the hardware device of the image file creation device 10X and the software including the image file creation program. In addition, some functions can be implemented by using artificial intelligence (AI). The following describes each functional unit.

[0203] (Generation Part 3)

[0204] The third generation unit 121 generates a base image used in the synthesis process. The steps of generating the base image are the same as the steps of generating the developed image, the color correction image, or the processed image in the first embodiment, so the description of the steps of generating the base image is omitted. In addition, the third generation unit 121 can give information indicating whether the base image is an image used in the generation of the thumbnail image (hereinafter referred to as usability information) to the base image. The usability information is stored in the image file of the base image as additional information, for example.

[0205] (Receiving Department)

[0206] When a plurality of basic images are synthesized to generate a synthesized image, the receiving unit 122 receives an arbitrary image generation mode from a plurality of image generation modes. Fig.15 In the mode designation screen, the first to fourth modes are selectively displayed. The user arbitrarily designates any mode from these, and the receiving unit 122 receives the arbitrary image generation mode designated by the user. In addition, the image generation mode may be received before the synthesis process is executed, for example, it may be performed just after the camera device 20 is started.

[0207] (Image determination unit)

[0208] When a composite image is generated by synthesizing a plurality of base images, the image determination unit 123 determines a base image for generating a thumbnail image from among the plurality of base images. The base image for generating a thumbnail image (hereinafter referred to as a generated base image as required) may be determined by the image determination unit 123 upon receiving a designation of a generated base image from a user, or may be determined automatically by the image determination unit 123. In the former case, for example, Fig.16 In the image designation screen, a plurality of base images used in generating a composite image can be displayed in a designatable manner. The user designates at least one of the plurality of base images, and the image determination unit 123 receives the user's designation of generating a base image. In addition, the reception of the designation of generating a base image can be performed before generating a thumbnail image, for example, it can be performed just after performing the synthesis process.

[0209] When the image determination unit 123 automatically determines the base image to be generated, the base image to be generated can be predetermined corresponding to each mode. For example, in the first mode, both an overexposed image and an underexposed image can be determined as the base image to be generated. Also, in the second mode, the first image in time can be determined as the base image to be generated. Also, in the third mode, all images that will be the basis of synthesis can be determined as the base image to be generated. In the fourth mode, as described later, an image with a high score as a result of subject analysis for the synthesized image can be determined as the base image to be generated.

[0210] (Synthesis Department)

[0211] The synthesis unit 124 performs synthesis processing according to the image generation mode received by the receiving unit 122, and synthesizes a plurality of basic images to generate a synthesized image in the synthesis processing. In addition, when the synthesis processing is performed in the third mode or the fourth mode, the synthesis unit 124 receives the designation of a plurality of basic images used in the synthesis processing from the user, and generates a synthesized image using the designated plurality of basic images.

[0212] (Generation 4)

[0213] The fourth generation unit 125 generates a thumbnail image as incidental information of the synthesized image in conjunction with the synthesis process. In the second embodiment, the fourth generation unit 125 generates a thumbnail image based on at least one of the plurality of base images according to the image generation mode received by the receiving unit 122 .

[0214] Specifically, when the receiving unit 122 receives the first mode, the fourth generating unit 125 generates one or more thumbnail images based on a part or all of the plurality of generated basic images. More specifically, Fig.17 As shown, the fourth generation unit 125 generates two or more thumbnail images based on at least two of the multiple generation base images with different exposures during shooting. The two or more generation base images for generating thumbnail images may include, for example, an overexposed generation base image and an underexposed generation base image. In this case, among the multiple base images used in the first mode, an image that has a greater impact on the synthesized image, such as an overexposed or underexposed image, is automatically selected. Then, thumbnail images can be generated based on the selected multiple generation base images.

[0215] In addition, the present invention is not limited to the above-mentioned case, and a thumbnail image may be generated based on a generated base image with a standard exposure (i.e., exposure of zero). Furthermore, all of a plurality of base images may be used as generated base images to generate thumbnail images based on each generated base image. Furthermore, a thumbnail image may be generated based on either an overexposed generated base image or an underexposed generated base image.

[0216] Furthermore, when the receiving unit 122 receives the first mode, the fourth generating unit 125 can generate a thumbnail image based on each of the plurality of generated base images for each generated base image. In this case, a thumbnail image based on the generated base image can be obtained for each of the plurality of generated base images that participate in the generation of the composite image.

[0217] In the configuration in which a thumbnail image is generated for each base image, the plurality of base images may include base images for which thumbnail images are not generated.

[0218] When the receiving unit 122 receives the second mode, the fourth generating unit 125 generates a thumbnail image based on a part of the plurality of basic images. Fig.18 As shown in FIG. 1 , the fourth generation unit 125 generates a thumbnail image based on only a part of the plurality of base images captured by shifting the image capturing position by pixel shift (for example, the first base image in terms of time). This is because, although the image capturing positions are shifted among the plurality of base images used in the second mode, most of the subjects in the images are common. That is, when performing synthesis processing in the second mode, as long as the thumbnail image is generated based on only a part of the base images, it is possible to properly grasp which base image is used in the generation of the synthesized image.

[0219] When the receiving unit 122 receives the third mode, Fig.19 As shown, the fourth generation unit 125 generates thumbnail images based on all of the plurality of base images. That is, when synthesizing a plurality of base images shot at different locations, the fourth generation unit 125 generates the same number of thumbnail images as the base images based on each base image. This is because the shooting locations are different between the plurality of base images used in the third mode, and therefore the subjects in the images are different. Therefore, when performing synthesis processing in the third mode, by generating thumbnail images based on all of the base images, it is possible to appropriately grasp which base image is used in the generation of the synthesized image.

[0220] When the receiving unit 122 receives the fourth mode, the fourth generating unit 125 may generate a thumbnail image based on a portion of the plurality of basic images. Fig. 20 As shown, the fourth generation unit 125 generates a thumbnail image by generating a basic image based on only a part of a plurality of basic images arranged in a predetermined layout on a template image. Here, the basic image for generating the thumbnail image may be, for example, a template image, that is, an image from which the arrangement layout of the basic images to be synthesized can be known.

[0221] As another example of the case where the receiving unit 122 receives the fourth mode, the fourth generating unit 125 may select a part of the basic images (strictly speaking, images other than the template image) from the plurality of basic images according to the sizes of the plurality of basic images in the composite image. In this case, the fourth generating unit 125 may generate thumbnail images according to the selected basic images. Thus, for example, thumbnail images of basic images having a large influence in the composite image, such as images having a relatively large size in the composite image, may be obtained.

[0222] As another example of the case where the receiving unit 122 receives the fourth mode, the fourth generating unit 125 can calculate a score for each of the multiple basic images in the composite image. The score can be calculated based on the resolution such as blur or shake of the subject in the basic image, the expression of the subject, the brightness (brightness) of each part of the subject, and the size of the basic image in the template image. Then, the fourth generating unit 125 can select a part of the generated basic images (strictly speaking, images other than the template image) from the multiple basic images based on the calculated scores, and generate a thumbnail image based on the selected generated basic images. Thereby, for example, a thumbnail image of a basic image that has a large influence in the composite image, such as an image with a high score in the composite image, can be obtained.

[0223] Furthermore, when the receiving unit 122 receives the fourth mode, when the number of the plurality of base images used in the synthesis process is less than the predetermined number, the fourth generating unit 125 may generate a thumbnail image from all of the plurality of base images. In addition, the number of base images that serves as a reference for determining whether to generate a thumbnail image from all of the plurality of base images can be determined to be an arbitrary number.

[0224] In the second embodiment, the fourth generation unit 125 may generate a thumbnail image based on at least one of the plurality of base images and a thumbnail image based on the synthesized image. By generating thumbnail images for the images before and after synthesis, it is possible to appropriately understand which synthesized image was generated from which base image.

[0225] In addition, depending on the image generation mode, sometimes it is obvious that the main image MP is a composite image. In this case, the base image may not be required. In this case, it may be set to generate a thumbnail image (first thumbnail image) based on the composite image instead of generating a thumbnail image based on the base image.

[0226] Furthermore, in the second embodiment, the fourth generation unit 125 may generate a thumbnail image that does not change in response to changes in the composite image caused by the correction process as a thumbnail image. Thus, a thumbnail image with excellent reliability and high utilization value as supplementary information related to the composite image can be obtained.

[0227] Furthermore, according to the image generation mode received by the receiving unit 122, the fourth generating unit 125 can generate a thumbnail image corresponding to the generation base image determined by the image determining unit 123. For example, when the receiving unit 122 receives the first mode or the second mode, the fourth generating unit 125 can generate a thumbnail image corresponding to the generation base image determined by the image determining unit 123.

[0228] In addition, the thumbnail image corresponding to the generated basic image is a thumbnail image obtained by reducing the whole or part of the basic image.

[0229] Furthermore, it is possible to determine which base image among the plurality of base images is used to generate the thumbnail image based on the availability information assigned to the base image. In this case, the fourth generation unit 125 may generate the thumbnail image based on the base image selected from the plurality of base images based on the availability information, i.e., the base image used to generate the thumbnail image. The availability information is information attached to each base image, and is information related to whether the base image is used as a base image to generate the thumbnail image when the base image is used as a base for a composite image. Thus, it is possible to appropriately determine whether it is necessary to generate the thumbnail image based on the availability information assigned to the base image, and generate the thumbnail image to be generated.

[0230] (Creation Department)

[0231] In the second embodiment, the creation unit 126 creates an image file of a composite image. The image file includes the composite image as the main image MP and supplementary information about the composite image, and the supplementary information includes the thumbnail image TP created by the fourth creation unit 125 .

[0232] (Operation Example of Image File Creation Device in Second Embodiment)

[0233] Next, the image file creation process using the image file creation device 10X in the second embodiment will be described. In the image file creation process described below, the image file creation method according to the second embodiment is used.

[0234] In addition, the following process is only an example, and part of the steps in the process may be deleted, or a new step may be added to the process, or the execution order of two steps in the process may be reversed without departing from the spirit of this embodiment.

[0235] Each step in the image file creation flow according to the second embodiment is executed by the processor 11 of the image file creation device 10X. Fig.21 That is, in each step of the image file creation process, the processor 11 executes the processing corresponding to each step in the data processing specified by the image file creation program. In addition, the image file creation process involved in the second embodiment is a processing flow for synthesizing a plurality of base images to generate a composite image.

[0236] In the image file creation flow involved in the second embodiment, first, the processor 11 performs a generation step A (S101). In the generation step A, a base image used in the subsequent synthesis step is generated. In general, the generation step A is performed multiple times, and the imaging conditions (specifically, exposure), the imaging location, or the imaging position of the imaging element during shooting may be different between the generation steps A.

[0237] Next, the processor 11 performs a receiving step (S102). In the receiving step, the processor 11 Fig.15 The mode designation screen shown is displayed on the display of the imaging device 20, and receives an arbitrary image generation mode designated by the user from among a plurality of image generation modes. The receiving step may be performed before the synthesis step is performed, for example, after the imaging device 20 is activated.

[0238] Next, the processor 11 implements an image determination step in which a generation base image for generating a thumbnail image is determined from among a plurality of base images to be synthesized (S103). In addition, the image determination step may be omitted depending on the image generation mode received in the receiving step.

[0239] Then, the processor 11 performs a synthesis process (S104). In the synthesis process, the processor 11 performs synthesis processing corresponding to the image generation mode received in the receiving process, synthesizes a plurality of base images to generate a synthesized image. Then, the processor 11 creates an image file including the main image MP using the synthesized image as the main image MP as the synthesized image is generated.

[0240] Next, the processor 11 performs a generation step B ( S105 ). In the generation step B, the processor 11 generates a thumbnail image TP based on at least one of the plurality of base images used in the synthesis step according to the image generation mode received in the reception step.

[0241] Specifically, when the first mode (synthesis mode for HDR) is received in the receiving step, in the generating step B, the processor 11 may generate one or more thumbnail images based on a portion of the plurality of base images. More specifically, in the generating step B, the processor 11 may generate two or more thumbnail images based on at least two of the plurality of base images with different exposures when photographed. At this time, it is preferred to generate thumbnail images corresponding to the respective base images based on the overexposed base image and the underexposed base image (refer to Fig.17 ).

[0242] As another example of the case where the first pattern is received in the receiving step, the processor 11 may generate thumbnail images from all of the plurality of images in the generating step B. Specifically, in the generating step B, the processor 11 may generate thumbnail images based on each of the plurality of base images for each base image.

[0243] When the second mode (synthesis mode for super-resolution) is received in the receiving step, in the generating step B, the processor 11 may generate a thumbnail image based on only a part of the plurality of basic images (see Fig.18 ). In this case, the processor 11 may generate a thumbnail image corresponding to the generation base image determined according to the user's designation received in the image determination step.

[0244] When the third mode (composition mode for multiple exposure) is received in the receiving step, the processor 11 may generate thumbnail images based on all of the plurality of base images in the generating step B (see Fig.19 ).

[0245] When the fourth mode (synthesis mode for collage) is received in the receiving step, the processor 11 may generate a thumbnail image based on only a part of the plurality of basic images in the generating step B (see Fig. 20 ). In this case, the processor 11 may generate a thumbnail image corresponding to the generation base image determined according to the user's designation received in the image determination step.

[0246] Alternatively, the processor 11 may refer to the availability information assigned to each base image and generate a thumbnail image based on a generation base image selected based on the availability information from among a plurality of base images.

[0247] Alternatively, the processor 11 may automatically select a portion of generated basic images from the plurality of basic images according to the size of each of the plurality of basic images in the composite image, and generate a thumbnail image according to the selected generated basic images.

[0248] Alternatively, the processor 11 may calculate a score corresponding to the resolution of the subject in each of the plurality of base images, and generate a thumbnail image by generating a base image from a portion of the plurality of base images automatically selected based on the score.

[0249] Then, the thumbnail image TP generated in the generation step B is stored in the image file of the composite image as incidental information of the composite image.

[0250] Furthermore, the thumbnail image may be an image that does not change in response to the conversion of the composite image caused by the correction process even if the composite image is corrected after it is generated. In this case, the thumbnail image becomes an image with high credibility and high utilization value as evidence of which base image was used to generate the composite image.

[0251] Furthermore, the processor 11 may generate a thumbnail image based on the composite image together with a thumbnail image based on at least one of the plurality of base images in the generation step B. In this case, thumbnail images are generated for the images before and after the synthesis, respectively, so that it is possible to appropriately grasp which composite image is generated using which base image.

[0252] When the series of steps above are completed, the image file creation process involved in the second embodiment is terminated. As described above, in the image file creation process of the second embodiment, thumbnail images are generated based on the basic image used in the synthesis process. Thus, it is possible to accurately understand which basic image is used in the synthesis process based on the generated thumbnail image.

[0253] Furthermore, in the second embodiment, a thumbnail image is generated based on at least one of the plurality of base images according to the image generation mode received in the receiving step. Thus, a thumbnail image based on a base image can be appropriately generated in accordance with the method corresponding to the synthesis process. Specifically, an image as an original image of a thumbnail image can be appropriately selected from the plurality of base images according to the type of synthesis process, and a thumbnail image can be generated based on the selected base image.

[0254] [About the third embodiment]

[0255] The image file creation device 10Y according to the third embodiment creates an image file including a main image MP and a thumbnail image TP as supplementary information related to the main image MP, similarly to the above-mentioned embodiment. The image file creation device 10Y has a function of analyzing a subject of the main image MP, determining a thumbnail image generation condition based on the analysis result, and generating a thumbnail image based on the determined condition.

[0256] Specifically, in the third embodiment, the image file creation device 10Y analyzes the main subject among the subjects of the main image MP, and more specifically, analyzes the category, size, or resolution of the main subject, and quantitatively determines these. Category refers to the classification (category) of the main subject. Size refers to the size of the area where the main subject exists in the main image MP. Resolution is an indicator of the clarity of the main subject in the main image MP, specifically, the degree of jitter or blur. As a method of determining these, it is possible to use well-known subject detection technology, subject recognition technology, image analysis technology, etc.

[0257] Furthermore, when a plurality of subjects exist in the main image MP, a main subject is selected from the plurality of subjects based on one or more of the following conditions i1 to i5.

[0258] i1: Focus position (AF point) of the imaging device 20 in the main image MP

[0259] i2: The user's line of sight position in the main image MP detected by the viewfinder provided in the imaging device 20

[0260] i3: Features of each object in the main image MP

[0261] i4: The degree of change of the subject between the live preview image captured earlier in time than the main image MP and the main image MP

[0262] i5: Subject category specified by the user

[0263] When the main subject is selected based on the conditions i1 and i2, for example, the subject located closest to the AF point or the line of sight position detected by the viewfinder of the imaging device 20 among the multiple subjects can be selected as the main subject.

[0264] The subject characteristics as condition i3 include, for example, the resolution such as the degree of blur or shaking of the subject in the main image MP, the image quality such as sharpness, and the expression of the subject. When the main subject is selected based on condition i3, for example, a subject whose degree of blur or shaking is suppressed to less than a standard value or a subject with a smiling expression can be selected as the main subject.

[0265] The degree of change of the subject as condition i4 is the amount of change (degree of change) in the position or state of the subject from the time when the live preview image is captured to the time when the main image MP is captured. When the main subject is selected based on condition i4, for example, a subject whose position or state has changed by a predetermined amount or more can be selected as the main subject.

[0266] When the main subject is selected based on the condition i5, the subject that matches the category specified by the user can be selected as the main subject.

[0267] In the third embodiment, the main image MP to be analyzed includes a RAW image before development. Furthermore, the image to be analyzed is not limited to the main image MP, but may be an image captured earlier than the main image MP, that is, a live preview image. The live preview image is an image similar to the main image MP, so substantially the same analysis result can be obtained between the live preview image and the main image MP.

[0268] In the third embodiment, the image file creation device 10Y determines the image quality factor of the thumbnail image or the viewing angle of the thumbnail image based on the analysis result related to the main subject. The image quality factor is an index for determining the image quality of the thumbnail image, and includes numerical factors such as resolution, number of colors, number of grayscales, dynamic range, and color gamut. Then, the image file creation device 10Y generates a thumbnail image TP based on the determined image quality factor or viewing angle.

[0269] According to the above, in the third embodiment, the utilization value of thumbnail images can be improved. Fig. 22 For example, a normal thumbnail image TP is formed by reducing the main image MP as an original image to a general thumbnail image size (for example, 400×320 pixels). Here, if the subject in the main image MP is relatively small, if the thumbnail image TP of the main image MP is created in the above manner, Fig. 22 As shown in FIG. 1 , the subject in the thumbnail image TP becomes noticeably small, making it difficult to recognize.

[0270] In contrast, in the third embodiment, the factors or viewing angles of the image quality of the thumbnail image TP are determined based on the type, size, or resolution of the subject in the main image MP. Fig. 22 As shown, the thumbnail image TP is generated according to the determined factors or viewing angles. Specifically, the thumbnail image TP is generated in such a way that its resolution or grayscale number is equal to or greater than a predetermined value (a value of a general thumbnail image). For example, the thumbnail image TP is generated at a resolution of 1200×960 pixels. Alternatively, Fig. 22 As shown, a subject region is extracted (cropped) from the main image MP according to the determined viewing angle, and a thumbnail image TP is generated based on the extracted subject region.

[0271] According to the above, in the third embodiment, the thumbnail image TP can be generated so that the region of the subject in the thumbnail image TP becomes clearer. As a result, the utility value of the thumbnail image TP can be improved, and specifically, the subject in the thumbnail image TP can be well and clearly recognized.

[0272] As an example of using the thumbnail image TP obtained in the third embodiment, for example, the thumbnail image can be used to create teacher data for machine learning (AI learning). Compared with a normal thumbnail image, the thumbnail image TP obtained in the third embodiment has a higher definition of the subject, and thus becomes an image effective as a material for teacher data.

[0273] Furthermore, the thumbnail image TP generated in the third embodiment may be an image that does not change in accordance with the change of the main image MP caused by the correction process. Such a thumbnail image TP is a highly reliable image that serves as evidence of which image the main image MP was at the time of its generation. Therefore, even if correction such as tampering is performed on the main image MP, the implementation of the correction can be recognized by referring to the thumbnail image TP.

[0274] Furthermore, in the third embodiment, it is possible to generate Fig.23 The two thumbnail images shown are thumbnail images TP based on the main image MP. The first thumbnail image is a thumbnail image TP based on the entire main image MP. Fig.23 The first thumbnail image is hereinafter referred to as a "proof thumbnail image". The proof thumbnail image is mainly used as evidence to prove which image the main image MP before correction is when correction processing is performed on the main image MP.

[0275] The second thumbnail image is a thumbnail image TP based on the area of ​​the main subject in the main image MP. Specifically, Fig.23 As shown, it is an image obtained by reducing the area of ​​the main subject extracted (cropped) from the main image MP. Hereinafter, the second thumbnail image is referred to as a "learning thumbnail image". The learning thumbnail image is mainly used for creating teacher data for machine learning (AI learning). However, it is not limited to this, and the thumbnail image (proof thumbnail image) obtained by reducing the entire main image MP can also be used for creating teacher data for machine learning.

[0276] Next, refer to Fig.24Functions of the image file creation device 10Y according to the third embodiment will be described. The hardware configuration of the image file creation device 10Y according to the third embodiment is the same as that of the first embodiment, and software such as an image file creation program is installed in the image file creation device 10Y.

[0277] like Fig.24 As shown, the image file creation device 10Y involved in the third embodiment includes a fifth generation unit 221, a selection unit 222, an analysis unit 223, a factor determination unit 224, a pattern receiving unit 225, a sixth generation unit 226, an encryption unit 227, and a creation unit 228. These functional units are implemented by the cooperation of the hardware device of the image file creation device 10Y and the software including the image file creation program. In addition, some functions can be implemented by using artificial intelligence (AI). The following describes each functional unit.

[0278] (Generation Part 5)

[0279] The fifth generation unit 221 generates the main image MP. The procedure for generating the main image MP in the third embodiment is the same as the procedure for generating the main image MP (developed image, color-corrected image, or processed image) in the first embodiment, and thus the description thereof will be omitted.

[0280] (Selected)

[0281] When there are multiple subjects in the main image MP, the selection unit 222 selects the main subject from the multiple subjects. Specifically, when the user does not specify the type of the main subject, the selection unit 222 selects the main subject based on one or more of the conditions i1 to i4 described above. The conditions i1 to i4 are contents that can be determined by analyzing the main image MP. Therefore, the main subject can be automatically selected from the multiple subjects based on at least one of the conditions i1 to i4.

[0282] On the other hand, when the user specifies the category of the main subject, the selection unit 222 selects the main subject based on one or more of the above-mentioned conditions i1 to i4 and the category specified by the user (i.e., condition i5). In this way, when the user specifies the category of the main subject, the selection unit 222 selects the main subject from a plurality of subjects based on the specified category.

[0283] In addition, when the user specifies the category of the main subject, for example, Fig.25 The category designation screen shown may be displayed on the display of the imaging device 20. In this case, the user designates (inputs) the category of the object through the category designation screen, thereby receiving the designation of the category.

[0284] (Analysis Department)

[0285] The analyzing unit 223 performs an analysis process on the main image MP. In the analysis process, the type, size, or resolution of the subject of the main image MP is analyzed. In addition, when the selecting unit 222 selects the main subject, in the analysis process, the type, size, or resolution of the selected main subject is analyzed.

[0286] In addition, the object of the analysis process is not limited to the subject of the main image MP, and may be a subject of a live preview image captured earlier in time than the main image MP is captured.

[0287] (Determination of factors, etc.)

[0288] The factor etc. determination unit 224 performs a determination process in which factors of the image quality of the thumbnail image TP or the viewing angle of the thumbnail image TP (hereinafter referred to as factors etc.) are determined based on the analysis result of the analysis unit 223. Specifically, the factor etc. determination unit 224 determines the resolution of the thumbnail image TP based on, for example, the size of the main subject analyzed by the analysis unit 223. More specifically, the factor etc. determination unit 224 determines the resolution of the thumbnail image in such a way that the spatial frequency of the area of ​​the main subject in the thumbnail image TP becomes greater than a specified value. As a result, when generating the thumbnail image TP, the clarity of the main subject in the thumbnail image can be fully ensured to generate the thumbnail image TP.

[0289] The predetermined value related to the spatial frequency may be a value set in advance as an initial value, or may be a value that can be arbitrarily set and changed by the user.

[0290] Furthermore, when the size of the main subject analyzed by the analysis unit 223 is smaller than the prescribed size, in the determination process, the resolution or grayscale number of the thumbnail image TP may be increased compared to the case where the size of the main subject is larger than the prescribed size. Alternatively, when the size of the main subject is smaller than the prescribed size, in the determination process, the viewing angle of the thumbnail image TP may be reduced compared to the case where the size of the main subject is larger than the prescribed size. Specifically, the viewing angle of the thumbnail image TP may be set to be limited to the area of ​​the main subject in the main image MP.

[0291] Based on the above, the resolution, grayscale number, or viewing angle of the thumbnail image can be determined so that the main subject in the thumbnail image becomes clearer and easier to recognize.

[0292] In addition, the thumbnail image TP generated based on the resolution, number of grayscales, or viewing angle determined as described above can be effectively used for the purpose of creating training data for machine learning.

[0293] (Mode receiving unit)

[0294] When the user selects the high-quality mode, the mode receiving unit 225 receives the selection of the high-quality mode. The high-quality mode refers to a mode that improves the image quality of the thumbnail image TP containing the main subject compared to the normal mode, specifically, the mode that increases the resolution or the number of grayscales. For example, when obtaining high-quality thumbnail images for the purpose of creating teacher data for machine learning, the user can select the high-quality mode. The user can select the high-quality mode by setting Fig.26 The mode selection screen shown is displayed on the display of the imaging device 20 for reception.

[0295] The timing of receiving the selection of the high-quality mode is not particularly limited as long as it is before the thumbnail image TP is generated. For example, the mode selection may be received immediately after the main image MP (specifically, the developed image) is captured.

[0296] When the mode receiving unit 225 receives the selection of the high-quality mode, the factor etc. determining unit 224 determines the resolution or grayscale number of the thumbnail image based on the selected high-quality mode and the analysis result of the analyzing unit 223. Thus, when the user selects the high-quality mode, the factors of the quality of the thumbnail image TP can be determined reflecting the user's intention of wanting a thumbnail image of high quality.

[0297] (Generation 6)

[0298] The sixth generation unit 226 performs a thumbnail generation process to generate a thumbnail image TP based on the main image MP. In the thumbnail generation process, the sixth generation unit 226 generates a thumbnail image TP based on the main image MP based on the factors, etc. determined by the factor, etc. determination unit 224. In the third embodiment, the sixth generation unit 226 generates a thumbnail image for certification and a thumbnail image for learning as thumbnail images based on the main image MP.

[0299] Furthermore, the user may arbitrarily specify which of the certification thumbnail image and the learning thumbnail image is to be generated, or both thumbnail images are to be generated. Furthermore, when generating the learning thumbnail image, the resolution or grayscale of the learning thumbnail image may be changed according to the user's input operation.

[0300] Furthermore, when the mode receiving unit 225 receives the selection of the high-quality mode, the sixth generating unit 226 may generate a thumbnail image TP based on the area of ​​the main subject in the main image MP, that is, a learning thumbnail image. In this case, the thumbnail image TP is generated with the resolution or grayscale number determined by the factor etc. determining unit 224. Thus, the thumbnail image TP can be generated in a manner that reflects the intention of the user who has selected the high-quality mode and makes the main subject clearer.

[0301] When the mode receiving unit 225 receives the selection of the high-quality mode, it may generate a thumbnail image TP based on the entire main image MP, ie, a certification thumbnail image, at the resolution or grayscale number determined by the factor etc. determining unit 224 .

[0302] Furthermore, in the third embodiment, the sixth generation unit 226 may generate a thumbnail image that does not change in accordance with the change of the main image MP caused by the correction process as the thumbnail image TP. However, this is not limited to the above case, and the sixth generation unit 226 may generate a thumbnail image (a change incidental image) that changes in accordance with the change of the main image MP caused by the correction process.

[0303] (Encryption Department)

[0304] The encryption unit 227 obtains information on the encrypted thumbnail image TP generated by the sixth generation unit 226. Specifically, when a thumbnail image for certification is generated as a thumbnail image, the encryption unit 227 obtains information on the encrypted thumbnail image for certification. Furthermore, when a thumbnail image for learning is generated as a thumbnail image, the encryption unit 227 obtains information on the encrypted thumbnail image for learning. The information on the encrypted thumbnail image is, for example, a hash value obtained by hashing the thumbnail image TP.

[0305] The information acquired by the encryption unit 227 is stored in the image file as supplementary information related to the main image MP. In this way, by storing the encrypted information of the thumbnail image TP in the image file, it is possible to detect tampering of the thumbnail image TP. For example, in the case where a third party tampered with the thumbnail image TP, the hash value Ha of the thumbnail image before tampering stored in advance and the hash value Hb obtained by hashing the thumbnail image after tampering are compared. In this case, the hash value Ha becomes a different value from the hash value Hb, so it is possible to detect that the thumbnail image TP has been tampered with.

[0306] The encrypted information may be information obtained by further encrypting the hash value using a key. Furthermore, in the thumbnail image TP, at least the area of ​​the main subject may be encrypted, and other areas may not be encrypted (in other words, may be tampered with).

[0307] (Creation Department)

[0308] The creation unit 228 creates an image file including the main image MP generated by the fifth generation unit 221 . The image file also includes supplementary information related to the main image MP. The supplementary information includes the thumbnail image TP generated by the sixth generation unit 226 and information acquired by the encryption unit 227 .

[0309] (Operation Example of Image File Creation Device in Third Embodiment)

[0310] Next, the image file creation process using the image file creation device 10Y in the third embodiment will be described. In the image file creation process described below, the image file creation method according to the third embodiment is used.

[0311] In addition, the following process is only an example, and part of the steps in the process may be deleted, or a new step may be added to the process, or the execution order of two steps in the process may be reversed without departing from the spirit of this embodiment.

[0312] Each step in the image file creation flow according to the third embodiment is executed by the processor 11 of the image file creation device 10Y. Fig. 27 That is, in each step in the image file creation flow, the processor 11 executes the processing corresponding to each step in the data processing specified by the image file creation program.

[0313] In the image file creation process involved in the third embodiment, first, the processor 11 implements a main image generation process (S201). In the main image generation process, a main image MP is generated through the same steps as the first embodiment. And, along with the generation of the main image MP, an image file of the main image MP is created. And, correction processing (development correction processing, color correction processing, and processing correction processing) is appropriately performed on the main image MP. The main image MP in the image file changes according to the execution of the correction processing.

[0314] Next, the processor 11 detects the subject in the main image MP. When there are multiple subjects in the main image MP, the processor 11 performs a selection process (S202, S203) before performing the subsequent analysis process. In the selection process, the processor 11 selects the main subject from the multiple subjects. In more detail, when the user does not specify the type of the main subject, the processor 11 selects the main subject based on one or more of the above-mentioned conditions i1 to i4. On the other hand, when the user selects the main subject through Fig.25When the category of the main subject is specified on the category specifying screen shown, the processor 11 selects the main subject based on one or more of the above-mentioned conditions i1 to i4 and the category specified by the user (ie, condition i5 ).

[0315] Next, the processor 11 performs an analysis process to analyze the type, size, or resolution of the main subject of the main image MP (S204). The analysis process is not limited to analyzing the main subject of the main image MP, and the main subject of the live preview image captured earlier in time than the main image can also be analyzed.

[0316] After the analysis step is performed, the factor etc. determination step ( S207 ) is performed. However, when the user selects the high-quality mode, the mode reception step ( S205 , S206 ) is performed at a stage before the factor etc. determination step is performed.

[0317] In the factor etc. determination step, the processor 11 determines factors etc. of the image quality of the thumbnail image generated in the subsequent thumbnail generation step based on the analysis results in the analysis step. Specifically, in the factor etc. determination step, the processor 11 determines the resolution of the thumbnail image TP in such a way that the spatial frequency of the region of the main subject in the thumbnail image becomes greater than a predetermined value.

[0318] Furthermore, when the size of the main subject analyzed in the analysis step is smaller than the specified size, in the factor determination step, the resolution or grayscale of the thumbnail image may be increased compared to a case where the size of the main subject is larger than the specified size. Alternatively, when the size of the main subject is smaller than the specified size, the viewing angle of the thumbnail image may be reduced compared to a case where the size of the main subject is larger than the specified size.

[0319] Furthermore, when the user selects the high-quality mode, in the factor etc. determination step, the resolution or grayscale number of the thumbnail image may be determined based on the selected high-quality mode and the analysis result in the analysis step.

[0320] Next, the processor 11 implements a thumbnail generation step (S208). In the thumbnail generation step, the processor 11 generates a thumbnail image TP based on the factors and the like determined in the factor and the like determination step. For example, when the size of the main subject in the main image MP is smaller than a predetermined size, the processor 11 generates the thumbnail image TP with a higher resolution or grayscale number than when the size of the main subject in the main image MP is larger than the predetermined size. Alternatively, the processor 11 generates the thumbnail image TP with a smaller viewing angle than when the size of the main subject in the main image MP is larger than the predetermined size, in detail, with a viewing angle limited to the area of ​​the main subject.

[0321] Furthermore, in the thumbnail generation step, the processor 11 generates a thumbnail image TP that does not change in accordance with the change of the main image MP caused by the correction process. Furthermore, in the thumbnail generation step, the processor 11 may generate a thumbnail image for certification and a thumbnail image for learning based on the factors determined in the determination step, etc. For example, when the user selects the high-quality mode, a thumbnail image for learning may be generated.

[0322] Next, the processor 11 performs an encryption process (S209). In the encryption process, the processor 11 obtains the thumbnail image TP generated in the thumbnail generation process, and more specifically, obtains information that encrypts the thumbnail image for certification or the thumbnail image for learning. Specifically, the processor 11 obtains a hash value obtained by hashing the thumbnail image TP. Then, the thumbnail image TP generated in the thumbnail generation process and the information obtained in the encryption process are stored in the image file as incidental information of the main image MP.

[0323] When the series of steps described above are completed, the image file creation process according to the third embodiment is terminated. As described above, in the image file creation process according to the third embodiment, the type, size, or resolution of the subject (strictly speaking, the main subject) of the main image MP is analyzed. Then, the analysis result is reflected in the factors or viewing angle of the image quality of the thumbnail image TP based on the main image MP. As a result, the thumbnail image TP based on the main image MP can be appropriately generated in consideration of the type, size, or resolution of the main subject of the main image MP.

[0324] Specifically, the thumbnail image TP can be generated while ensuring the clarity of the main subject in the thumbnail image TP. As a result, the thumbnail image TP with improved utilization value can be obtained. In particular, by making the main subject in the thumbnail image TP clearer, the thumbnail image TP can be effectively used as data for creating teacher data for machine learning for the purpose of image recognition.

[0325] [About the fourth embodiment]

[0326] The image file creation device 10Z according to the fourth embodiment creates an image file including a main image MP and thumbnail images TP as supplementary information related to the main image MP, similarly to the above-mentioned embodiment. In the fourth embodiment, the main image MP is an image generated by pressing the release button of the camera 20. The image file creation device 10Z has a function of generating a plurality of thumbnail images TP based on the main image MP and images captured before or after the capturing of the main image MP (specifically, live preview images or recorded browsing images).

[0327] Specifically, in the fourth embodiment, a thumbnail image TP is generated based on the main image MP, and the thumbnail image TP is generated based on at least one of the live preview image and the recorded browse image (see FIG. 2 ). Fig.29 ). Thus, it is possible to generate a thumbnail image TP based on an image captured earlier or later than the capturing of the main image MP, and store the thumbnail image TP in the image file.

[0328] Hereinafter, the thumbnail image TP generated based on the main image MP is referred to as the "main thumbnail image". Furthermore, the thumbnail image TP generated based on the live preview image is referred to as the "pre-shooting thumbnail image", and the thumbnail image TP generated based on the live preview image or the recorded browsing image is referred to as the "post-shooting thumbnail image".

[0329] In the fourth embodiment, the main image MP includes a RAW image, and similarly to the first embodiment, the main image MP is changed by executing the correction process, specifically, it is transformed into any one of a developed image, a color-corrected image, or a processed image by the correction process. In addition, in the fourth embodiment, the correction process of the main image MP can be linked to the correction process of the main image MP and the correction process of the same content as the correction process of the main image MP can be executed on the live preview image or the recorded review image.

[0330] Furthermore, in the fourth embodiment, the main thumbnail image, the pre-photographing thumbnail image, and the post-photographing thumbnail image may be images that do not change in response to changes in the main image MP caused by the correction process. In this case, it is possible to understand whether the correction process has been performed on the main image MP based on each of the thumbnail images TP.

[0331] In the fourth embodiment, there is no particular limitation on the time point from which the live preview image is used to generate the pre-shooting thumbnail image. Similarly, there is no particular limitation on the time point from which the live preview image is used to generate the post-shooting thumbnail image. Furthermore, the number of pre-shooting thumbnail images and the number of post-shooting thumbnail images to be generated can be arbitrarily determined.

[0332] In addition, as a configuration example of the fourth embodiment, a case where a plurality of thumbnail images TP are generated from the main image MP, the live preview image, and the recorded browse image is described below. However, this is not limited to this, and one thumbnail image may be generated from the main image MP and the live preview image or the recorded browse image. In this case, the thumbnail image may be, for example, an image obtained by reducing the main image MP and an image obtained by reducing the live preview image or the recorded browse image, arranged and aggregated into one image.

[0333] Next, refer to Fig.28Functions of the image file creation device 10Z according to the fourth embodiment will be described. The hardware configuration of the image file creation device 10Z according to the fourth embodiment is the same as that of the first embodiment, and software such as an image file creation program is installed in the image file creation device 10Z.

[0334] like Fig.28 As shown, the image file creation device 10Z according to the fourth embodiment includes a first acquisition unit 321, a second acquisition unit 322, a third acquisition unit 323, a correction unit 324, a setting receiving unit 325, a seventh generation unit 326, and a creation unit 327. These functional units are implemented by the cooperation of the hardware device of the image file creation device 10Z and the software including the image file creation program. In addition, some functions can be implemented by using artificial intelligence (AI). The following describes each functional unit.

[0335] (Part 1 Acquisition)

[0336] The first acquisition unit 321 acquires the captured main image MP. Specifically, when the user fully presses the release button of the camera device 20, the first acquisition unit 321 captures a subject existing in the imaging range of the camera device 20 to acquire a RAW image, and develops the RAW image to acquire a developed image as the main image MP.

[0337] (Second acquisition part)

[0338] The second acquisition unit 322 acquires a live preview image captured before the main image MP is captured. Specifically, the user half-presses the release button before fully pressing the release button. Then, while the release button is half-pressed, the second acquisition unit 322 continues to capture live preview images at predetermined time intervals.

[0339] In the fourth embodiment, among the captured live preview images, the live preview images that satisfy the prescribed condition are temporarily stored in the memory 12 or the storage device 16 as images captured before the main image MP is captured. Here, the prescribed condition refers to, for example, a condition that the state or position of the subject in the live preview image has changed by a prescribed amount or more compared with the image (the live preview image stored immediately before) that was last stored in the memory 12 or the storage device 16.

[0340] That is, when the state or position of the subject has changed by a predetermined amount or more from the last saved live preview image, the live preview image captured at that time is newly saved in the memory 12 or the storage device 16. In this case, the image saved in the memory 12 or the storage device is replaced (updated) from the last saved live preview image to a new live preview image that satisfies the above-mentioned predetermined conditions.

[0341] As described above, in the fourth embodiment, the live preview images temporarily stored in the memory 12 or the storage device 16 are limited to images that satisfy the prescribed conditions. This makes it possible to appropriately select live preview images used in generating thumbnail images TP, and more specifically, to select live preview images with a composition relatively close to the main image MP.

[0342] In addition, the present invention is not limited to the above case, and all images captured in the live preview mode may be stored in the memory 12 or the storage device 16 regardless of whether or not a predetermined condition is satisfied.

[0343] (Part 3 Acquisition)

[0344] The third acquisition unit 323 acquires a record browse image or a live preview image captured after the main image MP is captured. Specifically, the third acquisition unit 323 stores the record browse image or the live preview image captured after a predetermined time has passed since the user fully pressed the release button in the memory 12 or the storage device 16. Then, the post-capture thumbnail image is generated (acquired) based on the stored image.

[0345] (Calibration Department)

[0346] The correction unit 324 performs correction processing on the main image MP. Specifically, similar to the correction unit 23 of the first embodiment, the correction unit 324 performs color correction processing on the developed image as the main image MP and performs processing correction processing on the developed image or color correction image as the main image MP.

[0347] Furthermore, in the fourth embodiment, the correction unit 324 can execute the same correction processing as the correction processing for the main image MP on the pre-photographing thumbnail image or the post-photographing thumbnail image in conjunction with the execution of the correction processing for the main image MP. For example, when the color correction processing is executed for the main image MP, the correction unit 324 can execute the same color correction processing for the pre-photographing thumbnail image or the post-photographing thumbnail image. In this way, when the main image MP is corrected, by executing the same correction processing for the pre-photographing thumbnail image or the post-photographing thumbnail image, these images can be changed to match the corrected main image MP.

[0348] (Setting receiving unit)

[0349] When the user sets a setting to increase the resolution of each of the plurality of thumbnail images generated by the seventh generation unit 326, the setting receiving unit 325 receives the setting. Specifically, in the fourth embodiment, when generating thumbnail images, Fig.30The mode acceptance screen shown is displayed on the display of the imaging device 20. The user selects either the normal mode or the high-resolution mode through the mode acceptance screen.

[0350] Then, when the user selects the high-resolution mode, the setting receiving unit 325 receives the selection as a setting for increasing the resolution of the thumbnail image. Here, when the high-resolution mode is selected, the resolution of each of the multiple thumbnail images is higher than that of the normal mode. As a result, the main thumbnail image, the pre-photographing thumbnail image, and the post-photographing thumbnail image can be generated more vividly (clearly), and as a result, the utilization value of each thumbnail image TP can be improved. In particular, when each of the multiple thumbnail images is used for the purpose of creating teacher data for machine learning, as described above, the setting for increasing the resolution of each thumbnail image becomes more meaningful.

[0351] Furthermore, in the normal mode, the resolution of each of the plurality of thumbnail images is set to a general resolution of thumbnail images.

[0352] Furthermore, when the high-resolution mode is selected, the resolution of each of the plurality of thumbnail images may not be increased to a higher resolution than in the normal mode. The resolution of at least the main thumbnail image among the plurality of thumbnail images may be increased, and the resolutions of the other thumbnail images may be increased according to the user's instruction. Furthermore, in consideration of the capacity of the thumbnail images, the setting receiving unit 325 may receive a setting to increase the resolution of only the main thumbnail image among the plurality of thumbnail images.

[0353] (Generation 7)

[0354] The seventh generation unit 326 performs a process of generating a thumbnail image based on the main image MP, the live preview image, or the recorded browse image. In the fourth embodiment, the process of generating a thumbnail image includes a first generation process and a second generation process. In the first generation process, Fig.29 As shown, a main thumbnail image, a pre-photographing thumbnail image, and a post-photographing thumbnail image are generated based on the main image MP, the instant preview image, and the recorded browse image. The instant preview image of the original image constituting the pre-photographing thumbnail image and the recorded browse image or the instant preview image of the original image constituting the post-photographing thumbnail image are temporarily stored in the memory 12 or the storage device 16 of the camera device 20, and are read when the thumbnail image is generated.

[0355] Furthermore, each thumbnail image TP generated in the first generation process is an image that does not change in accordance with a change in the main image MP due to the correction process if the correction process is executed on the main image MP after the thumbnail image TP is generated.

[0356] Furthermore, the seventh generating unit 326 may generate a main thumbnail image and generate either a pre-capture thumbnail image or a post-capture thumbnail image.

[0357] Furthermore, when the setting receiving unit 325 receives a setting for increasing the resolution of the thumbnail image TP, each thumbnail image TP is generated at a resolution corresponding to the setting (i.e., a higher resolution than the normal mode) in the first generation process. Thus, each thumbnail image TP is generated at a higher resolution, and as a result, the image quality and utilization value of each thumbnail image TP can be improved.

[0358] Furthermore, the above setting is a setting for increasing the resolution of at least the main thumbnail image, and therefore, in the first generation process when the above setting is received, at least the main thumbnail image is generated with the resolution corresponding to the above setting.

[0359] The second creation process generates a change thumbnail image as a change incidental image based on the main image MP. The change thumbnail image is an image that changes in accordance with a change in the main image MP caused by the correction process when the correction process is performed on the main image MP after the change thumbnail image is generated. By storing the change thumbnail image together with the main thumbnail image that does not change in accordance with the change of the main image MP in the image file, the correction content of the main image MP can be understood by comparing these images.

[0360] Furthermore, when the correction process is performed on the main image MP, the seventh generation unit 326 may additionally perform the third generation process. Fig.31 As shown, a thumbnail image based on the main image MP subjected to correction processing and a thumbnail image based on the live preview image or recorded browse image subjected to the same correction processing as the main image MP are generated. In this way, if a thumbnail image based on the corrected main image MP and a thumbnail image based on the live preview image or recorded browse image subjected to the same correction processing are generated, the security of the thumbnail images is improved. The security of the thumbnail images will be described in detail later.

[0361] In addition, the content of the third generation process is not limited to the above content. For example, from the perspective of saving data capacity, in the third generation process, only thumbnail images based on the main image MP subjected to the correction process may be generated. In other words, thumbnail images based on the live preview image or the recorded browse image subjected to the correction in the same manner as the main image MP may not be generated.

[0362] (Creation Department)

[0363] The creation unit 327 creates an image file of the main image MP acquired by the first acquisition unit 321. The image file stores supplementary information related to the main image MP, and the supplementary information includes the thumbnail image TP generated by the seventh generation unit 326. The thumbnail image TP stored in the image file includes a main thumbnail image based on the main image MP, a pre-shooting thumbnail image based on the live preview image, and a post-shooting thumbnail image based on the record browsing image or the live preview image. Fig.32 As shown, the image file created in the fourth embodiment includes a plurality of thumbnail images TP generated from the main image, the live preview image, and the record browse image.

[0364] (Operation Example of Image File Creation Device in Fourth Embodiment)

[0365] Next, the image file creation process using the image file creation device 10Z in the fourth embodiment will be described. In the image file creation process described below, the image file creation method according to the fourth embodiment is used.

[0366] In addition, the following process is only an example, and part of the steps in the process may be deleted, or a new step may be added to the process, or the execution order of two steps in the process may be reversed without departing from the spirit of this embodiment.

[0367] Each step in the image file creation flow according to the fourth embodiment is executed by the processor 11 of the image file creation device 10Z. Fig.33 That is, in each step in the image file creation flow, the processor 11 executes the processing corresponding to each step in the data processing specified by the image file creation program.

[0368] In the image file creation process involved in the fourth embodiment, first, the user half-presses the release button before shooting the main image MP, which is used as a trigger, and the processor 11 implements the pre-shooting process (S301). In the pre-shooting process, shooting in the live preview mode is carried out to continuously generate (acquire) live preview images. In addition, during the implementation of the pre-shooting process, the live preview image that meets the specified conditions is temporarily stored in the memory 12 or the storage device 16 as an image shot before the shooting of the main image. The specified condition is that the state or position of the subject in the live preview image has changed by more than a specified amount compared with the last saved image (that is, the live preview image saved in the memory 12 or the storage device 16 at the immediately previous point in time).

[0369] Then, when the user fully presses the release button to capture the main image MP, the processor 11 executes an image capturing step (S302). In the image capturing step, the processor 11 acquires a RAW image and develops the RAW image to acquire a developed image as the main image MP.

[0370] Then, the processor 11 performs a post-shooting process (S303) at a point in time when a predetermined time has passed since the main image MP was shot. In the post-shooting process, the processor 11 obtains a real-time preview image after the main image MP is shot. The obtained real-time preview image is stored in the memory 12 or the storage device 16. In addition, the processor 11 may also obtain an image shot after the main image MP is shot, that is, a record preview image.

[0371] After the post-photography process is performed, the first generation process (S306) is performed. However, in the case where the user selects the high-resolution mode as the setting for increasing the resolution of the thumbnail image, the setting reception process (S304, S305) is performed before the first generation process is performed. In the setting reception process, the processor 11 receives a setting for increasing the resolution of at least the main thumbnail image among the plurality of thumbnail images generated in the subsequent first generation process. More specifically, the processor 11 receives the user's selection of the high-resolution mode as the above-mentioned setting.

[0372] In addition, in the setting receiving step, a setting to increase the resolution of each of the plurality of thumbnail images may be received, or a setting to increase the resolution of only the main thumbnail image may be received.

[0373] In the first generation process, the processor 11 generates a plurality of thumbnail images TP based on the main image MP and the live preview image and the recorded browse image stored in the memory 12 or the storage device 16. Specifically, the main thumbnail image is generated based on the main image MP, the pre-photographing thumbnail image is generated based on the live preview image, and the post-photographing thumbnail image is generated based on the recorded browse image or the live preview image. In addition, each thumbnail image TP generated in the first generation process is an image that does not change in accordance with the change of the main image MP caused by the correction process.

[0374] Furthermore, in the setting receiving step, when the user's selection of the high resolution mode is received as the setting for increasing the resolution of the thumbnail image TP, in the first generating step, each thumbnail image TP is generated at a resolution corresponding to the high resolution mode. Here, which thumbnail image among the plurality of thumbnail images TP is generated at a resolution corresponding to the high resolution mode is determined based on the above setting. However, at least the main thumbnail image is generated at a resolution corresponding to the high resolution mode.

[0375] The first generation process is not limited to the process of generating a plurality of thumbnail images TP, but may be the process of generating one thumbnail image. Specifically, the main image MP, the live preview image, and the recorded view image may be reduced and combined into one thumbnail image TP.

[0376] In the fourth embodiment, the second generation step (S307) is performed together with the first generation step. In the second generation step, the processor 11 generates a changed thumbnail image based on the main image MP. The changed thumbnail image is an image (changed incidental image) that changes in accordance with the change of the main image MP caused by the correction process.

[0377] In addition, the implementation of the second generation step may be omitted.

[0378] After the thumbnail image TP is generated in the first generation step and the second generation step, the processor 11 performs a creation step (S308). In the creation step, the processor creates an image file including the main image MP and the thumbnail image TP. Furthermore, if a modified thumbnail image is generated in the second generation step, an image file including the modified thumbnail image is created in the creation step (see Fig.32 ).

[0379] When the correction process is executed for the main image MP in the image file after the image file is created (S309), the processor 11 replaces (updates) the main image MP in the image file with the corrected main image MP. In addition, the processor 11 performs the third generation step (S310) in conjunction with the execution of the correction process. In the third generation step, the processor 11 generates a thumbnail image based on the main image MP on which the correction process is executed and a thumbnail image based on a live preview image or a recorded browse image on which the same correction process as the main image MP is executed.

[0380] Then, the thumbnail image generated in the third generation step is added to the image file created in the creation step and stored in the image file.

[0381] The content of the third generation step is not limited to the above content, and for example, only thumbnail images based on the main image MP subjected to correction processing may be generated. That is, thumbnail images based on the live preview image or the recorded browsing image subjected to correction similarly to the main image MP may not be generated.

[0382] When the series of steps above are completed, the image file creation process involved in the fourth embodiment is terminated. As described above, in the image file creation process of the fourth embodiment, thumbnail images TP are generated based on the main image MP and the live preview image or the recorded browse image, and are stored in the image file as incidental information of the main image MP. As a result, the security of the thumbnail images is improved. In addition, the image file contains multiple thumbnail images TP that are offset in time, so it is easy to implement machine learning on the changes in the movement of the subject through the machine learning model.

[0383] Specifically, assuming a case where the thumbnail image TP itself is tampered with, for example, if the thumbnail image TP based on the main image MP is tampered with, the credibility of the thumbnail image TP that proves the content of the main image MP is damaged.

[0384] On the other hand, in the fourth embodiment, as described above, thumbnail images TP are generated based on the main image MP and the instant preview image or the recorded browse image. In this case, even if the thumbnail image TP based on the main image MP is tampered with alone, the tampering can be easily discovered by referring to the thumbnail image TP based on the instant preview image or the recorded browse image. In addition, in order to avoid being discovered to have tampered with, each thumbnail image TP needs to be changed (tampered with) in the same way, but the time required for tampering with the thumbnail image TP increases, and the difficulty of tampering becomes higher. As a result, tampering with the thumbnail image TP can be suppressed and its security can be improved.

[0385] <<Other implementation methods>>

[0386] The embodiment described above is given to facilitate understanding of the present invention, and is merely an example. Embodiments other than the above are also conceivable.

[0387] (About the timing of creating a thumbnail image and creating an image file)

[0388] In the above embodiment, when the main image MP is generated (acquired) or when the correction process is performed on the main image MP, the thumbnail image TP is created in conjunction with it, and the image file is created. However, the time when the thumbnail image TP is generated (acquired) and the time when the image file is created are not particularly limited, and for example, it may be the time when a predetermined time has passed since the acquisition time of the main image MP. Furthermore, the above time may be arbitrarily determined by the user, and the thumbnail image may be generated and the image file may be created at the time desired by the user.

[0389] (Types of imaging devices)

[0390] In the above-mentioned embodiment, the image file creation device of the present invention is mounted in a camera device. Furthermore, the camera device is a digital camera that a user can carry or an information processing terminal such as a smart phone or a tablet terminal with a camera function. However, it is not limited to this, and the above-mentioned camera device may also be a fixed camera device such as a surveillance camera or a camera device having multiple camera units (image sensors).

[0391] (Regarding the Structure of the Image File Creation Device)

[0392] In the above embodiment, the image file including the main image MP and the accompanying information is created by the camera device that captures the main image MP. However, the present invention is not limited thereto, and the image file may be created by a device different from the camera device, specifically, by an information processing terminal such as a PC, a smart phone, or a tablet terminal connected to the camera device. In this case, after the main image MP is captured by the camera device, the main image MP is transmitted from the camera device to the above information processing terminal. Then, on the side of the information processing terminal that receives the main image MP, the above-mentioned image processing such as the correction processing and the synthesis processing is performed on the main image MP, and the accompanying information including the thumbnail image TP is generated.

[0393] (About the structure of the processor)

[0394] The processor included in the image file creation device of the present invention includes various processors, and the various processors include, for example, a general-purpose processor, namely, a CPU, which executes software (program) and functions as various processing units.

[0395] In addition, various processors include PLDs (Programmable Logic Devices) and other processors whose circuit configuration can be changed after manufacturing, such as FPGAs (Field Programmable Gate Arrays).

[0396] Various processors include processors such as ASIC (Application Specific Integrated Circuit) which have a circuit configuration specifically designed to perform specific processing, that is, dedicated circuits.

[0397] Furthermore, a functional unit of the image file creation device of the present invention may be composed of one of the various processors described above. Alternatively, a functional unit of the image processing device of the present invention may be composed of a combination of two or more processors of the same or different types, such as a combination of multiple FPGAs or a combination of an FPGA and a CPU.

[0398] Furthermore, the plurality of functional units included in the image file creation device of the present invention may be constituted by one of various processors, or two or more of the plurality of functional units may be integrated and constituted by one processor.

[0399] Furthermore, as in the above-described embodiment, a processor may be constituted by a combination of one or more CPUs and software, and the processor may function as a plurality of functional units.

[0400] Furthermore, for example, a processor that realizes the functions of the entire system including the plurality of functional units in the image file creation device of the present invention by a single IC (Integrated Circuit) chip, such as represented by SoC (System On Chip), etc. Furthermore, the hardware structure of the various processors described above may be a circuit (Circuitry) composed of a combination of circuit elements such as semiconductor elements.

[0401] Explanation of symbols

[0402] 10-image file creation device, 10X, 10Y, 10Z-image file creation device, 11-processor, 12-memory, 13-communication interface, 14-input device, 15-output device, 16-storage device, 20-camera, 21-type receiving unit, 22-first generation unit, 23-correction unit, 24-determination unit, 25-selection acquisition unit, 26-second generation unit, 27-creation unit, 121-third generation unit, 122-reception unit, 123-image determination unit, 124-synthesis unit, 125 -4th generation unit, 126-creation unit, 221-5th generation unit, 222-selection unit, 223-analysis unit, 224-factor etc. determination unit, 225-mode receiving unit, 226-6th generation unit, 227-encryption unit, 228-creation unit, 321-1st acquisition unit, 322-2nd acquisition unit, 323-3rd acquisition unit, 324-correction unit, 325-setting receiving unit, 326-7th generation unit, 327-creation unit, AD-supplementary information, MP-main image, TP-thumbnail image (supplementary image).

Claims

1. A method for creating an image file, the method for creating an image file including a main image and an accompanying image as accompanying information related to the main image, the method comprising: a category receiving step of receiving the category of the supplementary image in association with the correction process for the main image; and The acquisition step acquires, according to the type, the supplementary image that does not change in response to the change of the main image caused by the correction process.

2. The image file creation method according to claim 1, wherein: The acquisition step is performed when the correction process is performed on the main image.

3. The image file creation method according to claim 1, wherein: The correction process is executed in an image pickup device that has picked up the main image.

4. The image file creation method according to claim 1, wherein: In the category receiving step, at least one of a first supplementary image, a second supplementary image, and a third supplementary image is received as the category, generating the first supplementary image based on a first image obtained by executing a development correction process for development on the RAW image as the main image, generating the second supplementary image based on a second image obtained by performing a color correction process including color correction on the main image, The third supplementary image is generated based on a third image obtained by executing a processing correction process for processing a subject in the main image on the main image.

5. The image file creation method according to claim 4, wherein: The development correction process does not include a part of a plurality of color corrections executable in the color correction process.

6. The image file creation method according to claim 4, wherein: When two or more of the first, second, and third incidental images are acquired, the image file includes the two or more incidental images as the incidental images.

7. The image file creation method according to claim 4, wherein: When the third image is acquired by performing the processing correction process on the first image or the second image which is the main image contained in the image file, the main image contained in the image file is replaced from the first image or the second image to the third image.

8. The image file creation method according to claim 4, comprising: a selection acquisition step of acquiring a user's selection as to whether to acquire the third supplementary image when the processing correction process is performed on the main image, When the modification correction process is executed on the main image, the acquisition step of acquiring the third supplementary image is performed according to the selection.

9. The image file creation method according to claim 4, comprising: A determination step of determining whether or not to implement the acquisition step of acquiring the third supplementary image based on the correction content of the main image when the modification correction process is executed on the main image.

10. The image file creation method according to claim 1, wherein: The supplementary information includes information on the type of the supplementary image.

11. The image file creation method according to claim 1, wherein: The supplementary information includes information obtained by encrypting the supplementary image.

12. The image file creation method according to claim 11, wherein: The information obtained by encrypting the accompanying image includes a hash value obtained by hashing the accompanying image.

13. The image file creation method according to claim 1, wherein: The supplementary image includes a first thumbnail image which does not change in response to a change in the main image caused by the correction process and is generated by reducing the entirety or a portion of the main image at the time when the supplementary image is generated.

14. The image file creation method according to claim 13, wherein: The supplementary image includes a second thumbnail image that changes in accordance with a change in the main image caused by the correction process and is generated by reducing the entirety or a portion of the main image.

15. An image file creation device that creates an image file including a main image and an accompanying image as accompanying information related to the main image, The image file creation device comprises a processor, The processor receives the type of the supplementary image in association with the correction process for the main image, The processor acquires the supplementary image that does not change in response to the change of the main image caused by the correction process, according to the type.

Images