Imaging device and its control method, program, and storage medium

Abstract

The aim is to provide an imaging device that offers the functionality to track the history of an image while also improving user convenience. [Solution] The system includes an imaging unit that performs imaging processing to capture an image of a subject and generate an image file containing image data, a detection unit that detects tampering with the image file, and a control unit that controls the imaging unit and the detection unit so as not to perform tampering detection when processing is applied to an image file that has been recorded in the recording unit and played back, and not to perform tampering detection when processing is applied to the image data during the imaging process.

Description

Technical Field

[0001] The present invention relates to an imaging device having an image forgery prevention function.

Background Art

[0002] In recent years, information sharing via the Internet has been actively carried out, and anyone can now publicly disclose and transmit various information to an unspecified number of people. Also, various processes can now be performed on digital images. In such a situation, there are cases where information is transmitted from an untrustworthy source or where the publicly disclosed information has been illegally forged.

[0003] In Patent Document 1, in order to prevent such illegal forgery, when shooting is performed in a forgery prohibition mode, a hash value is generated and attached to the image, thereby prohibiting the forgery of the image.

[0004] Also, in Non-Patent Document 1, in order to authenticate the origin, background, and history of an image, it has been proposed to attach metadata indicating the editing content performed on the image to the image file.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

[0007] According to the technology described in Patent Document 1, users can know whether or not an image has been modified. Furthermore, according to the technology in Non-Patent Document 1, by embedding metadata from the moment of capture as the history within the image, it is possible to know what modifications were made if the image was edited afterward.

[0008] However, in situations where there is no room for tampering, such as during the shooting process, these technologies were not always convenient for the user.

[0009] This invention was made in view of the above-mentioned problems, and its purpose is to provide an imaging device that can improve user convenience while also having the function of knowing the history of an image. [Means for solving the problem]

[0010] The imaging apparatus according to the present invention is characterized by comprising: an imaging means for performing an imaging process that images a subject and generates an image file containing image data; a detection means for detecting tampering with the image file; and a control means that, when an image file that has been recorded in a recording means and reproduced is subjected to processing, the detection means performs a process to detect tampering with the reproduced image file, and when processing is performed on the image data during the imaging process, the detection means controls the imaging means and the detection means so as not to perform a process to detect tampering with the image data. [Effects of the Invention]

[0011] According to the present invention, it is possible to improve user convenience while maintaining the function of knowing the history of an image. [Brief explanation of the drawing]

[0012] [Figure 1] A diagram showing the external appearance of a digital camera, which is one embodiment of the imaging device of the present invention. [Figure 2] A block diagram showing the configuration of a digital camera. [Figure 3] A flowchart illustrating the main processing steps involved in the shooting operation of a digital camera. [Figure 4] A flowchart showing the steps for image processing settings. [Figure 5] A flowchart illustrating the procedure for processing still images. [Figure 6] A diagram showing an example of the structure of an image file. [Figure 7] A flowchart illustrating the procedure for generating the history. [Figure 8] A flowchart illustrating the procedure for adding historical information. [Figure 9] A flowchart illustrating the steps involved in image editing. [Modes for carrying out the invention]

[0013] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0014] <Appearance of Digital Camera 100> FIG. 1 is a diagram showing the appearance of a digital camera 100 which is an embodiment of the imaging device of the present invention. FIG. 1(a) is a front perspective view of the digital camera 100, and FIG. 1(b) is a rear perspective view of the digital camera 100.

[0015] In FIG. 1, the display unit 28 is a display unit provided on the rear surface of the digital camera 100, and displays images and various information. The touch panel 70a can detect a touch operation on the display surface (touch operation surface) of the display unit 28. The external-of-finder display unit 43 is a display unit provided on the upper surface of the digital camera 100, and displays various setting values of the digital camera 100 including the shutter speed and aperture value. The shutter button 61 is an operation member for giving a shooting instruction. The mode changeover switch 60 is an operation member for switching various modes. The terminal cover 40 is a cover for protecting a connector (not shown) for connecting the digital camera 100 to an external device such as a connection cable.

[0016] The main electronic dial 71 is a rotary operation member. By turning the main electronic dial 71, it is possible to change setting values such as the shutter speed and aperture value. The power switch 72 is an operation member for switching on and off the power of the digital camera 100. The sub - electronic dial 73 is a rotary operation member. By turning the sub - electronic dial 73, it is possible to move the selection frame (cursor) and perform image scrolling. The four - way key 74 is configured such that its upper, lower, left, and right parts can be pushed in respectively, and processing can be performed according to the part of the four - way key 74 that is pushed. The SET button 75 is a push button and is mainly used for determining selected items.

[0017] The video button 76 is used to give instructions to start and stop video shooting (recording). The AE lock button 77 is a push button. By pressing the AE lock button 77 in the shooting standby state, the exposure state can be fixed. The zoom button 78 is an operation button for switching on and off the zoom mode in the live view display (LV display) of the shooting mode. By operating the main electronic dial 71 after turning on the zoom mode, it is possible to zoom in and out the live view image (LV image). In the playback mode, the zoom button 78 functions as an operation button for enlarging the playback image or increasing its magnification. The playback button 79 is an operation button for switching between the shooting mode and the playback mode. By pressing the playback button 79 during the shooting mode, the user can shift to the playback mode and display the latest image recorded on the recording medium 200 (described later) on the display unit 28. The menu button 81 is a push button used to give an instruction operation to display the menu screen. When the menu button 81 is pressed, a menu screen enabling various settings is displayed on the display unit 28. The user can intuitively perform various settings using the menu screen displayed on the display unit 28, the four - way key 74, and the SET button 75.

[0018] The touch bar 82 (multifunction bar: M-Fn bar) is a line-shaped touch operation component (line touch sensor) capable of accepting touch operations. The touch bar 82 is positioned so that it can be touched with the right thumb when the grip section 90 is held with the right hand (with the little finger, ring finger, and middle finger) so that the shutter button 61 can be pressed with the right index finger. In other words, the touch bar 82 is positioned so that it can be operated when the user is looking through the viewfinder with their eyepiece 16 and is ready to press the shutter button 61 at any time (shooting posture). The touch bar 82 is a reception area that can accept tap operations (touching and releasing without moving within a predetermined period of time), left and right sliding operations (touching and then moving the touch position while keeping the touch on the bar). The touch bar 82 is a different operation component from the touch panel 70a and does not have a display function.

[0019] The communication terminal 10 is a communication terminal for the digital camera 100 to communicate with the detachable lens unit 150 (see Figure 2). The eyepiece 16 is the eyepiece of the eyepiece viewfinder 17 (a look-through type viewfinder), and the user can view the image displayed on the internal EVF 29 (Electronic View Finder: see Figure 2) through the eyepiece 16. The eyepiece detection unit 57 is an eyepiece detection sensor that detects whether or not the user (photographer) is looking through the eyepiece 16. The cover 202 is the cover of the slot for storing the recording medium 200 (see Figure 2). The grip 90 is a holding part shaped to be easy for the user to grip with their right hand when holding the digital camera 100. The shutter button 61 and the main electronic dial 71 are positioned so that they can be operated with the index finger of the right hand when the digital camera 100 is held with the grip 90 held by the little finger, ring finger, and middle finger of the right hand. Furthermore, in the same configuration, the sub-electronic dial 73 and touch bar 82 are positioned so that they can be operated with the right thumb. The thumb rest section 91 (thumb standby position) is a grip component located on the back of the digital camera 100, in a place where it is easy to rest the thumb of the right hand holding the grip section 90 when no other operating components are being used. The thumb rest section 91 is made of rubber or other material to enhance the holding power (grip).

[0020] <Configuration of Digital Camera 100> Figure 2 is a block diagram showing an example configuration of the digital camera 100. The lens unit 150 is a lens unit that incorporates an imaging optical system and is interchangeable with the digital camera 100. The lens 103 that constitutes the imaging optical system is usually composed of multiple lenses, but in Figure 2 it is simplified and shown as only one lens. Communication terminal 6 is a communication terminal for the lens unit 150 to communicate with the digital camera 100, and communication terminal 10 is a communication terminal for the digital camera 100 to communicate with the lens unit 150. The lens unit 150 communicates with the system control unit 50 via these communication terminals 6 and 10. In the lens unit 150, the internal lens system control circuit 4 controls the aperture 1 via the aperture drive circuit 2. Also in the lens unit 150, the lens system control circuit 4 focuses by displacing the position of the lens 103 via the AF drive circuit 3.

[0021] The shutter 101 is a focal-plane shutter that allows the exposure time of the imaging unit 22 to be freely controlled by the system control unit 50.

[0022] The imaging unit 22 includes an image sensor, such as a CCD or CMOS element, which converts an optical image into an electrical signal. The image sensor in the imaging unit 22 may have an image plane phase difference sensor that outputs defocus amount information to the system control unit 50. The A / D converter 23 converts the analog signal output from the imaging unit 22 into a digital signal.

[0023] The image processing unit 24 performs predetermined processing (such as resizing, pixel interpolation, and color conversion) on the image data from the A / D converter 23 or the image data from the memory control unit 15. The image processing unit 24 also performs predetermined calculations using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the calculation results obtained by the image processing unit 24. This enables TTL (through-the-lens) AF (autofocus), AE (automatic exposure), EF (flash pre-flash), etc. The image processing unit 24 further performs predetermined calculations using the captured image data and performs TTL AWB (auto white balance) processing based on the obtained calculation results.

[0024] The output data from the A / D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15. Alternatively, the output data from the A / D converter 23 is written to the memory 32 via the memory control unit 15 without going through the image processing unit 24. The memory 32 stores image data obtained by the imaging unit 22 and converted into digital data by the A / D converter 23, as well as image data for display on the display unit 28 and EVF 29. The memory 32 has sufficient storage capacity to store a predetermined number of still images, a predetermined amount of video footage, and audio.

[0025] Furthermore, memory 32 also serves as memory for image display (video memory). The D / A converter 19 converts the image data for image display stored in memory 32 into an analog signal and supplies it to the display unit 28 and EVF 29. In this way, the image data for display written to memory 32 is displayed by the display unit 28 and EVF 29 via the D / A converter 19. The display unit 28 and EVF 29 are displays made of LCD, organic EL, etc., and display according to the analog signal from the D / A converter 19. The digital signal that has been A / D converted by the A / D converter 23 and stored in memory 32 is converted into an analog signal by the D / A converter 19 and sequentially transferred to the display unit 28 or EVF 29 for display, thereby enabling live view display (LV). Hereinafter, the image displayed in live view display will be referred to as a live view image (LV image).

[0026] The system control unit 50 is a control unit consisting of at least one processor and / or at least one circuit, and controls the entire digital camera 100. The system control unit 50 is both a processor and a circuit. The system control unit 50 realizes each of the processes of this embodiment, which will be described later, by executing a program recorded in the non-volatile memory 56. The system control unit 50 also performs display control by controlling the memory 32, D / A converter 19, display unit 28, EVF 29, etc.

[0027] System memory 52 is, for example, RAM, and the system control unit 50 loads constants, variables, programs read from non-volatile memory 56, etc., for the operation of the system control unit 50 into system memory 52.

[0028] The non-volatile memory 56 is an electrically erasable and recordable memory, such as an EEPROM. Constants for the operation of the system control unit 50, programs, etc., are stored in the non-volatile memory 56. The program referred to here is a program for executing various flowcharts, which will be described later in this embodiment.

[0029] The system timer 53 is a timekeeping unit that measures the time used for various controls and the time of the built-in clock.

[0030] The communication unit 54 transmits and receives video and audio signals to and from external devices connected wirelessly or via wired cables. The communication unit 54 can also connect to wireless LAN (Local Area Network) and the internet. Furthermore, the communication unit 54 can communicate with external devices using Bluetooth® and Bluetooth Low Energy. The communication unit 54 can transmit images (including LV images) captured by the imaging unit 22 and images recorded on the recording medium 200, and can receive various information from external devices, such as image data and instructions to start video recording. When an instruction to start video recording is received from an external device, the communication unit 54 can notify the user of the instruction by illuminating the light-emitting unit 102 or emitting an electronic sound from the speaker 92. Examples of external devices that can communicate include smartphones, tablet PCs, and desktop PCs.

[0031] The attitude detection unit 55 detects the orientation of the digital camera 100 relative to the direction of gravity. Based on the orientation detected by the attitude detection unit 55, it is possible to determine whether the image captured by the imaging unit 22 was taken with the digital camera 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the orientation detected by the attitude detection unit 55 to the image file of the image captured by the imaging unit 22, or rotate the image before recording. An acceleration sensor or gyro sensor can be used as the attitude detection unit 55. The attitude detection unit 55 can also use the acceleration sensor or gyro sensor to detect the movement of the digital camera 100 (pan, tilt, lift, whether it is stationary or not, etc.).

[0032] The eyepiece detection unit 57 is an eyepiece detection sensor that detects when an eye (object) approaches (approaches) and moves away (away from) the eyepiece section 16 of the eyepiece viewfinder 17 (hereinafter simply referred to as "viewfinder"). The system control unit 50 switches the display / hide status of the display unit 28 and the EVF 29 according to the state detected by the eyepiece detection unit 57. More specifically, at least in the shooting standby state and when the display destination switching setting is set to automatic switching, when not using an eyepiece, the display destination is set to the display unit 28 and the display is turned on, and the EVF 29 is hidden. When using an eyepiece, the display destination is set to the EVF 29 and the display is turned on, and the display unit 28 is hidden. As the eyepiece detection unit 57, for example, an infrared proximity sensor can be used to detect the approach of any object to the eyepiece section 16 of the viewfinder 17 which has a built-in EVF 29. When an object approaches, infrared light emitted from the light-emitting unit (not shown) of the eyepiece detection unit 57 is reflected by the object and received by the light-receiving unit (not shown) of the infrared proximity sensor. The amount of infrared light received can be used to determine how close the object is to the eyepiece unit 16.

[0033] In this way, the eyepiece detection unit 57 performs eyepiece detection to detect the proximity distance of an object to the eyepiece unit 16. When an object is detected approaching the eyepiece unit 16 within a predetermined distance from the non-eyepiece state, it is detected that the user has made eye contact. When the object that was detected approaching moves away from the eyepiece state beyond a predetermined distance, it is detected that the user has moved away. The threshold for detecting eye contact and the threshold for detecting eye separation may be different, for example, by providing hysteresis. After eye contact is detected, the user remains in the eyepiece state until eye separation is detected. After eye separation is detected, the user remains in the non-eyepiece state until eye contact is detected again. Note that the infrared proximity sensor is just one example, and the eyepiece detection unit 57 may use any other sensor that can detect a state that can be considered as eye contact.

[0034] The GPS receiver 119 receives GPS information from GPS satellites to calculate location and time information. The digital camera 100 receives GPS information from the GPS receiver 119 and calculates location and time information based on the received GPS information. The digital camera 100 can add this calculated location and time information to the captured image.

[0035] The hash value generation unit 210 generates (calculates) a hash value by applying a hash function to the image file. Alternatively, the system control unit 50 may generate the hash value instead of the hash value generation unit 210. Details of the hash value generation process will be described later.

[0036] The external viewfinder display unit 43 displays various camera settings, including shutter speed and aperture, via the external viewfinder display unit drive circuit 44.

[0037] The power control unit 80 consists of a battery detection circuit, a DC-DC converter, a switch circuit for switching which blocks are energized, and detects whether a battery is installed, the type of battery, and the remaining battery level. The power control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, supplying the necessary voltage to each part, including the recording medium 200, for the required period. The power supply unit 30 consists of primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries and Li-ion batteries, and an AC adapter.

[0038] The recording medium I / F18 is an interface to the recording medium 200, such as a memory card or hard disk. The recording medium 200 is a recording medium such as a memory card for recording captured images, and is composed of semiconductor memory, magnetic disks, etc.

[0039] The operation unit 70 is an input unit that receives operations from the user and is used to input various operation instructions to the system control unit 50. As shown in Figure 2, the operation unit 70 includes a shutter button 61, a mode selector switch 60, a power switch 72, a touch panel 70a, and other operating members 70b. Other operating members 70b include a main electronic dial 71, a sub electronic dial 73, a four-way key 74, a SET button 75, a video button 76, an AE lock button 77, a zoom button 78, a playback button 79, a menu button 81, a touch bar 82, and the like.

[0040] The shutter button 61 is equipped with a first shutter switch 62 and a second shutter switch 64. The first shutter switch 62 turns ON during the operation of the shutter button 61, specifically when it is half-pressed (indicating preparation for shooting), and generates a first shutter switch signal SW1. The system control unit 50 starts shooting preparation operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing in response to the first shutter switch signal SW1.

[0041] The second shutter switch 64 turns ON when the shutter button 61 is fully pressed (shooting instruction), generating the second shutter switch signal SW2. The system control unit 50 starts a series of shooting operations, from reading the signal from the imaging unit 22 to writing the captured image as an image file to the recording medium 200, in response to the second shutter switch signal SW2.

[0042] The mode switch 60 switches the operating mode of the system control unit 50 to one of the following: still image shooting mode, video shooting mode, playback mode, etc. Modes included in the still image shooting mode include auto shooting mode, auto scene detection mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). There are also various scene modes and custom modes that provide shooting settings for different shooting scenes. The user can switch directly to any of these modes using the mode switch 60. Alternatively, the user can switch to a list screen of shooting modes using the mode switch 60, and then selectively switch to one of the displayed modes using another operating element. Similarly, the video shooting mode may also include multiple modes.

[0043] The touch panel 70a is a touch sensor that detects various touch operations on the display surface of the display unit 28 (the operating surface of the touch panel 70a). The touch panel 70a and the display unit 28 can be configured as an integrated unit. For example, the touch panel 70a is configured such that its light transmittance does not interfere with the display of the display unit 28, and is mounted on the upper layer of the display surface of the display unit 28. Then, the input coordinates on the touch panel 70a are associated with the display coordinates on the display surface of the display unit 28. This makes it possible to provide a GUI (Graphical User Interface) that makes it seem as if the user can directly operate the screen displayed on the display unit 28.

[0044] The system control unit 50 can detect the following operations or states on the touch panel 70a. - A finger or pen that was not previously touching the touch panel 70a now touches the touch panel 70a, i.e., the start of a touch (hereinafter referred to as Touch-Down). • The state in which the touch panel 70a is being touched with a finger or pen (hereinafter referred to as Touch-On). • The finger or pen is moving while touching the touch panel 70a (hereinafter referred to as Touch-Move). - The finger or pen that was touching the touch panel 70a has been removed (released), i.e., the touch has ended (hereinafter referred to as Touch-Up). • The state in which nothing is being touched on the touch panel 70a (hereinafter referred to as Touch-Off) When a touchdown is detected, a touch-on is also detected simultaneously. After a touchdown, touch-ons are usually detected continuously unless a touch-up is detected. Touch-ons are also detected simultaneously if a touch-move is detected. Even if a touch-on is detected, a touch-move will not be detected if the touch position has not moved. After all fingers or pens that were touching have been detected as having touched up, a touch-off occurs.

[0045] These operations and states, as well as the position coordinates of the finger or pen touching the touch panel 70a, are notified to the system control unit 50 via the internal bus. The system control unit 50 then determines what kind of operation (touch operation) was performed on the touch panel 70a based on the notified information.

[0046] Regarding touch movements, the direction of movement of a finger or pen on the touch panel 70a can also be determined for each vertical and horizontal component on the touch panel 70a based on the change in position coordinates. If a touch movement of a predetermined distance or more is detected, it is determined that a slide operation has been performed. An operation in which a finger is touched on the touch panel 70a and then quickly moved a certain distance and released is called a flick. In other words, a flick is an operation in which the finger is quickly traced across the touch panel 70a as if flicking it. If a touch movement of a predetermined distance or more at a predetermined speed or faster is detected, and a touch-up is then detected, it can be determined that a flick has been performed (it can be determined that a flick followed a slide operation). Furthermore, a touch operation in which multiple locations (for example, two points) are touched together (multitouch) and the touch positions are brought closer together is called a pinch-in, and a touch operation in which the touch positions are moved further apart is called a pinch-out. Pinch-out and pinch-in are collectively referred to as a pinch operation (or simply a pinch).

[0047] The touch panel 70a may be of any type from among various types of touch panels, such as resistive, capacitive, surface acoustic wave, infrared, electromagnetic induction, image recognition, and optical sensor types. There are methods that detect a touch when there is contact with the touch panel, and methods that detect a touch when a finger or pen approaches the touch panel, and either method is acceptable.

[0048] Next, Figure 3 is a flowchart showing the main processing steps for the shooting operation in the digital camera 100. Each process in Figure 3 is realized by the system control unit 50 loading the program stored in the non-volatile memory 56 into the system memory 52 and executing it. S represents the step number.

[0049] In S301, the system control unit 50 determines whether or not a setting instruction has been given to change the tamper-proof mode to "on". If a setting instruction to change the tamper-proof mode to "on" has been given, the system control unit 50 proceeds to S302; otherwise, it proceeds to S303.

[0050] Here, the tamper-proof mode is a shooting mode that can detect tampering. Specifically, when shooting with the tamper-proof mode enabled, the image includes provenance information 603 (see Figure 6), and when shooting with the tamper-proof mode disabled, the image does not include provenance information 603. When the tamper-proof mode is enabled, changes can be detected using the hash function and signature value mechanism described later, preventing malicious tampering.

[0051] In S302, the system control unit 50 changes the setting of the tamper-proof mode to "on" and stores it in the memory 32.

[0052] In S303, the system control unit 50 changes the setting of the tamper-proof mode to off and stores it in the memory 32.

[0053] In S304, the system control unit 50 executes the image processing setting process described later using Figure 4.

[0054] In S305, the system control unit 50 determines whether or not a still image capture instruction has been given, such as by pressing the shutter button 61. If a still image capture instruction has been given, the system control unit 50 proceeds to S306; otherwise, it proceeds to S307.

[0055] In S306, the system control unit 50 performs the still image capture process described later using Figure 5.

[0056] In S307, the system control unit 50 determines whether or not an image editing instruction has been given in playback mode or the like. If an image editing instruction has been given, the system control unit 50 proceeds to S308; otherwise, it proceeds to S309.

[0057] In S308, the system control unit 50 performs the image editing process described later using Figure 9.

[0058] In S309, the system control unit 50 determines whether or not an instruction to terminate the main process has been given, such as by turning off the power switch 72. If an instruction to terminate the main process has been given, the system control unit 50 terminates the main process. If it has not been given, the process returns to S301 and repeats the process from S301 to S309.

[0059] The following describes the procedure for setting image processing parameters in S304 of Figure 3, with reference to Figure 4. Figure 4 is a flowchart showing an example of the operation for setting image processing parameters.

[0060] In S401, the system control unit 50 determines whether or not the setting of the glitter effect mode has been instructed. If the glitter effect mode has been instructed, the system control unit 50 proceeds to S402; otherwise, it proceeds to S403.

[0061] Here, the "glitter effect mode" is one of the modes that applies a special effect to an image through image processing. The glitter effect mode may be provided as one of the modes that can be changed using a mode dial, or it may be provided as one of the menu settings. Furthermore, the glitter effect is just one example, and other image processing treatments that change the overall impression of the image may be set as special effects, such as toy camera style, diorama style, oil painting style, watercolor style, fisheye style, monochrome style, or soft focus style.

[0062] In S402, the system control unit 50 changes the sparkle effect mode to "on" and stores it in memory 32.

[0063] In S403, the system control unit 50 determines whether or not the setting of the skin beautifying effect processing has been instructed. If the setting of the skin beautifying effect processing has been instructed, the system control unit 50 proceeds to S404; otherwise, it proceeds to S405.

[0064] In S404, the system control unit 50 changes the setting for the skin beautifying effect processing to one of the instructed weak, medium, or strong settings and stores it in the memory 32.

[0065] Here, "skin beautifying effect processing" refers to image processing that makes a person's skin look more beautiful. Skin beautifying effect processing may be provided as one of the modes that can be changed using a mode dial, or as one of the menu settings. Furthermore, skin beautifying effect processing is just one example; other image processing effects that change the impression of the subject, such as face slimming effect processing, may also be set.

[0066] In S405, the system control unit 50 determines whether or not the HDR processing setting has been instructed. If the HDR processing setting has been instructed, the system control unit 50 proceeds to S406; otherwise, it proceeds to S407.

[0067] Here, HDR (High Dynamic Range) processing is a process that takes two images with different exposures and combines them to obtain an image with an expanded dynamic range. HDR processing may be provided as one of the modes that can be changed using a mode dial, or as one of the menu settings. Furthermore, HDR processing is just one example, and other image processing processes that take multiple images at once and combine them, such as depth stacking, may also be set.

[0068] In S406, the system control unit 50 changes the HDR processing setting to "on" and stores it in the memory 32.

[0069] In S407, the system control unit 50 determines whether or not it has been instructed to set up multiple exposure processing. If it has been instructed to set up multiple exposure processing, the system control unit 50 proceeds to S408; otherwise, it proceeds to S411.

[0070] In S408, the system control unit 50 determines whether or not it has been instructed to read the source image from the recording medium 200. If the system control unit 50 has been instructed to read the source image from the recording medium 200, it proceeds to S410; otherwise, it proceeds to S409.

[0071] In S409, the system control unit 50 changes the setting to multiple exposure, which combines multiple captured images, similar to HDR shooting, and stores it in the memory 32.

[0072] In S410, the system control unit 50 changes the setting to multiple exposure, which reads multiple source images from the recording medium 200 and combines them, and stores this setting in the memory 32. In this case, it is assumed that the memory 32 also stores which images will be used as source images. Furthermore, the multiple exposure process may be provided as one of the modes that can be changed using a mode dial or the like, or as one of the menu setting items.

[0073] In S411, the system control unit 50 determines whether or not it has been instructed to terminate the image processing setting process. If it has been instructed to terminate the image processing setting process, the system control unit 50 terminates the processing of this flow. If it has not been instructed to terminate, it returns to S401 and repeats the processing from S401 to S411.

[0074] The procedure for the still image capture process in S306 will be explained below with reference to Figure 5. Figure 5 is a flowchart showing an example of the operation from exposure during shooting to the generation of a still image file. This process starts when a shooting start operation is received, such as pressing the shutter button 61, and ends when a shooting end operation is received, such as canceling the pressing of the shutter button 61. The still image capture process is a series of processes in which the image data 604 expanded in the system memory 52 is processed, including the addition of shooting information 602 and history information 603, and other processing, until an image file is generated that is untouchable.

[0075] In S501, the system control unit 50 drives the shutter 101, which is located on the subject side of the imaging unit 22, in order to control the exposure time.

[0076] In S502, the system control unit 50 performs imaging processing, which converts the light from the subject received by the imaging unit 22 via the shutter into an electrical signal (analog image signal).

[0077] In S503, the system control unit 50 performs A / D conversion on the electrical signal obtained by the imaging process, and then performs image processing such as development and encoding to generate image data.

[0078] In S504, the system control unit 50 generates metadata 601 for the generated image data 604, including the shooting information 602 shown in Figure 6.

[0079] The shooting information 602 is the settings information of the digital camera 100 when the imaging process for generating the image data 604 is performed. For example, it includes the date and time of shooting, the photographer, the image size, the manufacturer and model of the digital camera 100, various shooting parameters set at the time of shooting, the shooting location, a thumbnail image, etc. The shooting information 602 is generated in accordance with a predetermined technical standard (for example, EXIF ​​(Exchangeable image file format)).

[0080] Provenance information 603 is information used to prove the authenticity of image data 604 and is used when verifying the source and provenance of image data 604. Provenance information 603 is generated in accordance with a predetermined technical standard (for example, C2PA (Coalition for Content Provenance and Authenticity)) and has a prescribed structure. Provenance information 603 includes provenance (Assertion) 613, a hash value 623 to guarantee provenance 613, and a digital signature 633. Provenance 613 stores provenance identification information (Manifest ID) to uniquely identify this provenance, editing history information showing the editing content of image data 604, editing tool information showing the tools used for editing, and creator information of image data 604.

[0081] Here, the image data 604 generated in step S503 is just generated by shooting and has not been edited by an editing application or anything else. Therefore, the editing history stores information indicating "generated," and the editing tool stores information indicating the digital camera 100. Note that when editing is performed, information indicating "edited" is stored.

[0082] In S505, the system control unit 50 executes the history generation process, which will be described later in Figure 7.

[0083] In S506, the system control unit 50 determines whether the image processing setting is enabled or disabled. If the image processing setting is enabled, the system control unit 50 proceeds to S507; otherwise, it proceeds to S508.

[0084] In S507, the system control unit 50 executes the history addition process, which will be described later in Figure 8.

[0085] In S508, the system control unit 50 adds metadata 601 to the image data 604 and generates an image file. Here, the image data 604 is generated according to a still image or video format such as JPEG or MPEG and is stored on the recording medium 200. The user can be notified that writing is in progress by the card access lamp.

[0086] In S509, the system control unit 50 determines whether or not an instruction has been given to end the shooting process. If an instruction has been given to end the shooting process, the system control unit 50 terminates the still image shooting process. If no such instruction has been given, the process returns to S501 and the process from S501 to S509 is repeated.

[0087] The procedure for generating the provenance history in S505 will be explained below with reference to Figure 7. Figure 7 is a flowchart showing an example of the operation from determining whether or not tamper-proof mode is in operation to generating the provenance history.

[0088] In S701, the system control unit 50 determines whether the tamper-proof mode setting is enabled or disabled. If the tamper-proof mode setting is enabled, the system control unit 50 proceeds to S702; otherwise, it terminates the processing of this flow.

[0089] In S702, the system control unit 50 generates the history 613. An example of the image file configuration in this case is shown in Figure 6(a). Figure 6(a) shows an example in which the history of the image data read from the image sensor before image processing is embedded in the image file.

[0090] In step S703, the system control unit 50 applies a hash function to the binary data of the image data 604 and the history data 613 to generate a hash value 623. Alternatively, a hash value may also be generated from the binary data of the shooting information 602.

[0091] In S704, the system control unit 50 generates a digital signature 633 and terminates the provenance generation process.

[0092] Digital signature 633 contains information indicating the signature value, signer, and signing date and time. The signature value is generated by encrypting the generated hash value 623 using a pre-prepared private key. The public key that forms the pair with the private key used here is also stored in digital signature 633. In this case, to prove that the public key is from a trusted manufacturer, information indicating the manufacturer of the digital camera 100 as the signer, and a public key certificate indicating that the public key has been authenticated by a certification authority may also be stored. By attaching such a digital signature 633 containing the signer to the image file 600, it can be shown that the image file is a trusted image file. Note that the model name of the digital camera 100 may be used as the signer instead of the manufacturer name. The signing date and time stores the date and time when the generation of the digital signature was completed.

[0093] The procedure for adding the provenance history in S507 will be explained below with reference to Figure 8. Figure 8 is a flowchart showing an example of the operations from image processing to adding the provenance history.

[0094] In S801, the system control unit 50 determines whether or not there is a history 613 for the image data at the time of generation. If there is a history 613 for the image data at the time of generation, the system control unit 50 proceeds to S802; otherwise, it proceeds to S806.

[0095] In S802, the system control unit 50 determines whether or not still image capture is in progress. Here, "in progress of still image capture" means the period between the shutter drive in S501 and the image file generation process in S508 in the still image capture process shown in Figure 5. If still image capture is in progress, the system control unit 50 proceeds to S803; otherwise, it proceeds to S804. If it is determined that still image capture is not in progress, it is assumed that image editing is in progress, where the image data read from the recording medium 200 is being processed on a playback screen or the like.

[0096] In S803, the system control unit 50 determines whether or not composite shooting is in progress using images read from the recording medium 200 (images within the image file read from the recording medium 200). If composite shooting is in progress using images read from the recording medium 200, the system control unit 50 proceeds to S804; otherwise, it proceeds to S806. In this case, composite shooting is assumed to be a type of shooting such as multiple exposure using images read from the recording medium 200.

[0097] In S804, the system control unit 50 performs tampering verification using hash values ​​and signature values. First, a hash function is applied to the binary data of the image data 604 read from the recording medium 200 to generate a hash value. Then, the generated hash value is compared with the hash value 624 of the image data to be checked. This makes it possible to verify whether or not the image data has been tampered with. Similarly, a hash function is applied to the binary data of the shooting information 602 to generate a hash value. Then, the generated hash value is compared with the hash values ​​of each of the shooting information to be checked. This makes it possible to verify whether or not the shooting date and time, shooting location, photographer, and other data have been tampered with. Whether or not to perform tampering verification depends on whether or not processing is about to be performed on the image read from the recording medium 200. Furthermore, this technology can be applied not only to digital cameras but also to editing applications and the like.

[0098] In S805, the system control unit 50 determines whether or not the data has been tampered with based on the results of the tampering verification. If the data has been tampered with, the system control unit 50 terminates the processing of this flow. If the data has not been tampered with, the process proceeds to S806.

[0099] In S806, the system control unit 50 performs image processing. This applies the special effects shown in Figure 4 and composite processing to the image data generated in S503.

[0100] In S807, the system control unit 50 adds the history 613. An example of the image file configuration in this case is shown in Figure 6(b). Figure 6(b) shows an example in which both the history of the image data before processing read from the image sensor and the history after image processing are embedded in the image file.

[0101] In S808, the system control unit 50 applies a hash function to the binary data of the processed image data 604 and the history data 613 to generate a hash value 623 for the added history data. Alternatively, a hash value may be generated from the binary data of the shooting information 602.

[0102] In S809, the system control unit 50 generates a digital signature 633 and terminates the history addition process.

[0103] Digital signature 633 contains information indicating the signature value, signer, and signing date and time. The signature value is generated by encrypting the appended hash value 623 using a pre-prepared private key. The public key that forms the pair with the private key used here is also stored in digital signature 633. In this case, to prove that the public key is from a trusted manufacturer, information indicating the manufacturer of the digital camera 100 as the signer, and a public key certificate indicating that the public key has been authenticated by a certification authority may also be stored. By attaching such a digital signature 633, including the signer, to the image file 600, it can be shown that the image file is a trusted image file. Note that the model name of the digital camera 100 may be used as the signer instead of the manufacturer's name. The signing date and time stores the date and time when the generation of the digital signature was completed.

[0104] The procedure for the image editing process in S308 will be described below with reference to Figure 9. Figure 9 is a flowchart showing an example of the operation from performing image processing on the object to be edited to adding its history. In this case, the editing process refers to a series of processes from when the image data read from the removable recording medium 200 is expanded into the system memory 52, when processing is performed, and finally when the image file is updated. This process is carried out assuming that the image data read from the removable recording medium 200 is susceptible to tampering.

[0105] In S901, the system control unit 50 determines whether or not an image to be edited has been selected from the images stored on the recording medium 200. If an image to be edited has been selected from the images stored on the recording medium 200, the system control unit 50 proceeds to S902; otherwise, it repeats the process in S902.

[0106] In S902, the system control unit 50 displays the image to be edited, selected from the images stored on the recording medium 200, on the display unit 28.

[0107] In S903, the system control unit 50 determines whether or not processing instructions have been given for the target image. If processing instructions have been given for the target image, the system control unit 50 proceeds to S904; otherwise, it proceeds to S905.

[0108] In S904, the system control unit 50 executes the history addition process described in Figure 8.

[0109] In step S905, the system control unit 50 adds metadata 601 to the image data 604 and generates an image file. Here, the image data 604 is generated according to still image or video formats such as JPEG or MPEG, and is stored on the recording medium 200. The user can be notified that image data is being written by the card access lamp.

[0110] In S906, the system control unit 50 determines whether or not an instruction has been given to terminate the image editing process. If an instruction has been given to terminate the image editing process, the system control unit 50 terminates the image editing process. If no such instruction has been given, the process returns to S901 and the process from S901 to S906 is repeated.

[0111] As explained above, in this embodiment, when a digital camera performs the shooting process, there is no possibility of tampering, so the provenance is added to the captured image data without performing tampering verification. However, when editing is performed on image data that has already been recorded on the recording medium, there is a possibility of tampering, so the provenance is added after tampering verification.

[0112] Furthermore, editing may also be performed by application programs, and if edited using an authorized editing tool and following proper procedures, new provenance information 603 will be generated based on the edited content in accordance with the prescribed technical standards. This information will then be appended to the metadata of the image file and stored. This is equivalent to the processing performed within a digital camera.

[0113] Provenance information 603 is newly generated each time an image file is edited and stored by being appended to the image file's metadata 601. On the other hand, if an image file is edited using an unauthorized editing tool or through fraudulent procedures, the tampering can be detected because the provenance information 603 may not be attached to the image file, or the provenance information attached to the image file may not conform to the prescribed technical standards.

[0114] Furthermore, the series of verification processes to check whether the history data has been tampered with may be performed at a finer level, such as for editing history information, creator information, thumbnail data, and metadata. Also, the signature value can be decrypted using a public key, and if the hash values ​​match, it can be determined that the signature value verification was successful. In this way, it is possible to embed a mechanism for detecting tampering at a finer level into the image file itself.

[0115] As described above, according to this embodiment, user convenience can be improved by performing tampering verification during content editing to maintain authenticity, and by eliminating unnecessary tampering verification processing during shooting that involves image processing.

[0116] (Other embodiments) Furthermore, the present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by a process in which one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0117] The disclosures herein include the following imaging devices and their control methods, programs, and storage media.

[0118] (Item 1) A shooting means that performs a shooting process to capture an image of a subject and generate an image file containing image data, A detection means for detecting tampering with image files, When processing is performed on an image file that has been recorded and played back by the recording means, the detection means performs a process to detect tampering with the played-back image file, and when processing is performed on the image data during the shooting process, the control means controls the shooting means and the detection means so that the detection means does not perform a process to detect tampering with the image data. An imaging device characterized by comprising:

[0119] (Item 2) The imaging apparatus according to item 1, characterized in that, if the control means performs processing on the image data during the shooting process, it performs the processing on the image data before writing the image data to the recording means.

[0120] (Item 3) The imaging device according to item 1 or 2, characterized in that the control means does not add the history of the image file to the image file if tampering is detected by the detection means.

[0121] (Item 4) The imaging apparatus according to any one of items 1 to 3, characterized in that, if the control means performs processing on the image data during the shooting process, it records the history of the image data before the processing and the history of the image data after the processing in the image file.

[0122] (Item 5) The imaging apparatus according to any one of items 1 to 4, characterized in that the processing is an image processing that changes the subject.

[0123] (Item 6) The imaging apparatus according to item 5, characterized in that the processing treatment is a skin beautifying treatment or a face slimming treatment.

[0124] (Item 7) The imaging apparatus according to any one of items 1 to 4, characterized in that the processing is an image processing that changes the impression of the image.

[0125] (Item 8) The imaging device according to item 7, characterized in that the processing is an image processing that includes one of the following: toy camera style, diorama style, oil painting style, watercolor style, fisheye style, monochrome style, soft focus style, or glitter effect.

[0126] (Item 9) The imaging apparatus according to any one of items 1 to 4, characterized in that the processing is a synthesis process that combines multiple images.

[0127] (Item 10) The imaging apparatus according to item 9, characterized in that, when the control means performs the synthesis process on a plurality of image data captured before being recorded in the recording means during the imaging process, the detection means controls the imaging means and the detection means so as not to perform the process of detecting tampering with the plurality of image data.

[0128] (Item 11) The imaging apparatus according to item 9, characterized in that, when the control means performs a synthesis process between the image data captured before being recorded in the recording means and the image data in the reconstructed image file during the imaging process, the detection means controls the imaging means and the detection means to perform a process to detect tampering with the image data in the reconstructed image file.

[0129] (Item 12) The imaging apparatus according to any one of items 9 to 11, characterized in that the synthesis process is an image processing process that includes HDR processing, depth of field processing, or multiple exposure processing.

[0130] (Item 13) The shooting process involves capturing an image of the subject and generating an image file containing image data, A detection process for detecting tampering with image files, When processing is performed on an image file that has been recorded and played back by the recording means, the detection step performs a process to detect tampering with the played-back image file, and when processing is performed on the image data during the shooting process, the control step controls the shooting process and the detection step so that the detection step does not perform a process to detect tampering with the image data. A control method for an imaging device, characterized by having the following features.

[0131] (Item 14) A program to cause a computer to execute each step of the control method for the imaging device described in item 13.

[0132] (Item 15) A computer-readable storage medium containing a program that causes a computer to execute each step of the control method for the imaging device described in item 13.

[0133] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0134] 100: Digital camera, 22: Imaging unit, 24: Image processing unit, 28: Display unit, 50: System control unit, 150: Lens unit

Claims

1. A shooting means that performs a shooting process to capture an image of a subject and generate an image file containing image data, A detection means for detecting tampering with image files, When processing is performed on an image file that has been recorded and played back by the recording means, the detection means performs a process to detect tampering with the played-back image file, and when processing is performed on the image data during the shooting process, the control means controls the shooting means and the detection means so that the detection means does not perform a process to detect tampering with the image data. An imaging device characterized by comprising:

2. The imaging apparatus according to claim 1, characterized in that, if the control means performs processing on the image data during the shooting process, it performs the processing on the image data before writing the image data to the recording means.

3. The imaging apparatus according to claim 1, characterized in that the control means does not add the history of the image file to the image file if tampering is detected by the detection means.

4. The imaging apparatus according to claim 1, characterized in that, if the control means performs processing on the image data during the shooting process, it records the history of the image data before the processing and the history of the image data after the processing in the image file.

5. The imaging apparatus according to claim 1, characterized in that the processing is an image processing that changes the subject.

6. The imaging apparatus according to claim 5, characterized in that the processing treatment is a skin beautifying treatment or a face slimming treatment.

7. The imaging apparatus according to claim 1, characterized in that the processing is an image processing that changes the impression of the image.

8. The imaging apparatus according to claim 7, characterized in that the processing is an image processing that includes any of the following: toy camera style, diorama style, oil painting style, watercolor style, fisheye style, monochrome style, soft focus style, or glitter effect.

9. The imaging apparatus according to claim 1, characterized in that the processing is a synthesis process that combines multiple images.

10. The imaging apparatus according to claim 9, characterized in that, when the control means performs the synthesis process on a plurality of image data captured before being recorded in the recording means during the imaging process, it controls the imaging means and the detection means so that the detection means does not perform the process of detecting tampering with the plurality of image data.

11. The imaging apparatus according to claim 9, characterized in that, when the control means performs a synthesis process between the image data captured before being recorded in the recording means and the image data in the reconstructed image file during the imaging process, the detection means controls the imaging means and the detection means to perform a process to detect tampering with the image data in the reconstructed image file.

12. The imaging apparatus according to claim 9, characterized in that the synthesis process is an image processing process that includes any of HDR processing, depth of field processing, or multiple exposure processing.

13. The shooting process involves capturing an image of the subject and generating an image file containing image data, A detection process for detecting tampering with image files, When processing is performed on an image file that has been recorded and played back by the recording means, the detection step performs a process to detect tampering with the played-back image file, and when processing is performed on the image data during the shooting process, the control step controls the shooting process and the detection step so that the detection step does not perform a process to detect tampering with the image data. A control method for an imaging device, characterized by having the following features.

14. A program for causing a computer to execute each step of the control method for the imaging device described in claim 13.

15. A computer-readable storage medium storing a program for causing a computer to execute each step of the control method for the imaging device described in claim 13.

Images