Image processing apparatus, image processing method, program product, and imaging apparatus

Abstract

An image processing apparatus includes a setting unit that selects metadata related to authenticity verification among metadata containing information of a plurality of items associated with captured image data, and sets mark information in the metadata related to authenticity verification, based on a feature of the captured image data.

Description

TECHNICAL FIELD

[0001] The present technology relates to an image processing apparatus, an image processing method, a program, and an imaging apparatus, and particularly relates to a technology regarding authentication of captured image data. BACKGROUND

[0002] A technology of authenticating a user who took an image of captured image data by adding an electronic signature to the captured image data is known.

[0003] In Patent Literature 1 mentioned below, a technology of transmitting captured image data to which an electronic signature is added to another apparatus via a network or the like, and authenticating a user who took received captured image data based on the added electronic signature when the other apparatus receives the captured image data is described.

[0004] LIST OF CITATIONS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent Application Publication No. 2005-341395 SUMMARY

[0007] PROBLEMS TO BE SOLVED BY THE INVENTION

[0008] Incidentally, in recent years, captured images taken by users themselves by using imaging apparatuses such as digital cameras, or mobile terminals such as smart phones are actively posted to a posting website, a social networking service (SNS), or the like.

[0009] Under the above-described circumstances, some cases of posting captured images as if the captured images are captured images taken by the users themselves by copying captured images taken by other users or the like are often observed.

[0010] Therefore, it is desired to authenticate not only a user who took captured image data, but also that a captured image taken by the user is a captured image based on the user's own creation.

[0011] Thus, in the present disclosure, a technology capable of performing a process of authenticating whether a captured image taken by a user is a captured image based on the user's own creation is proposed.

[0012] SOLUTION TO PROBLEM

[0013] An image processing apparatus according to the present technology includes a setting unit that selects metadata related to authenticity verification in metadata including information of a plurality of items associated with captured image data, based on a feature of the captured image data. Thus, an item reflecting the feature of the captured image data is selected.

[0014] The metadata related to the authenticity verification is, for example, values of "distance to a subject", "focal length of an imaging lens", "position at the time of capturing an image", "sensitivity (ISO sensitivity)", "aperture (F value) setting value", "shutter speed (SS value)", "date and time at the time of capturing an image", "air temperature", "air pressure", and the like.

[0015] Further, the setting unit sets the mark information in the metadata related to the authenticity verification. Thereby, the mark information is set for the item that reflects the characteristics of the captured image data.

[0016] The above-described image processing apparatus according to the present technology includes a hash value calculation unit that calculates a hash value with respect to captured image data and metadata associated with the captured image data, and a merge processing unit that merges signature data generated by using the hash value, the captured image data, and the metadata in which the mark information is set, so that the merged signature data, the captured image data, and the metadata can be recorded.

[0017] Thereby, even if the captured image data is the same, a different hash value is calculated in a case where the metadata associated with the captured image data is different.

[0018] The above-described image processing apparatus according to the present technology includes a generation control unit that generates signature data by using a hash value and a private key in a public key encryption scheme.

[0019] When a user who photographed an image of the captured image data encrypts the signature data using a private key in a public key encryption scheme, and transmits the signature data, and a third party who received the signature data can decrypt the signature data using a public key associated (paired) with the private key, the receiver can verify that the user who photographed the image of the captured image data made the transmission.

[0020] In the above-described image processing apparatus according to the present technology, the generation control unit that generates signature data by using a hash value and a private key in a public key encryption scheme is provided on an attachable and detachable external terminal, and the merge processing unit acquires the signature data from the external terminal.

[0021] Thereby, even if an image is captured using a different imaging apparatus, by connecting the external terminal to the imaging apparatus, signature data can be generated based on a private key in a public key encryption scheme associated with a user who captured the image.

[0022] In the above-described image processing apparatus according to the present technology, it is considered that distance information to an imaging object is included in a plurality of items associated with the captured image data.

[0023] In particular, this is useful in the case where the feature of the captured image is a captured close-up or a captured long shot, in the case where a specific subject such as a person or an animal is the capture target, and in other cases.

[0024] In the above-described image processing apparatus according to the present technology, it is considered that information related to exposure control is included in a plurality of items associated with the captured image data.

[0025] In particular, this is useful in the case where the feature of the captured image is a captured night scene or a captured day scene, or in other cases.

[0026] Here, the information related to exposure control is, for example, shutter speed (SS value) information for obtaining brightness information (EV value) of a shooting environment, sensitivity (ISO sensitivity), aperture (F value) setting value, and the like.

[0027] In the above-described image processing apparatus according to the present technology, it is considered that information related to an imaging date and time of the captured image data is included in a plurality of items associated with the captured image data.

[0028] In particular, this is useful in the case where the feature of the captured image reflects an imaging time of a night scene, a day scene, or the like, in the case where the feature of the captured image reflects an imaging season such as a snow scene, and in other cases.

[0029] It is considered that the above-described image processing apparatus according to the present technology includes a communication unit that outputs file data in which signature data, captured image data, and metadata in which mark information is set are merged.

[0030] Thus, the file data in which the signature data, the captured image data, and the metadata in which the mark information is set are merged is transmitted to an apparatus that verifies the authenticity of the captured image data.

[0031] Further, the image processing apparatus according to the present technology includes:

[0032] an information management unit that acquires information of a selected item from a merged file including captured image data and metadata including information selected as an item related to authenticity verification from a plurality of items associated with the captured image data; and

[0033] an authenticity determination unit that determines authenticity of a feature of the captured image data and the selected item.

[0034] The above-described image processing apparatus according to the present technology includes: an information management unit that acquires information of a selected item from a merged file including captured image data and metadata including an item selected from a plurality of items associated with the captured image data as an item related to authenticity verification; and an authenticity determination unit that determines authenticity of a feature of the captured image data and the selected item.

[0035] Thus, when one or more items of which the degree at which it reflects the feature of the captured image data is high and for which the tag information is set indicate unnatural values, it is determined that the captured image does not have authenticity. For example, when the captured image data indicates that the feature of the image is a long shot, and the distance to the imaged object is too close, it is determined that the image does not have authenticity.

[0036] In the above-described image processing apparatus according to the present technology, the information management unit outputs a result corresponding to authenticity determination by the authenticity determination unit is taken into consideration.

[0037] Thus, for example, the authenticity determination result is transmitted to a terminal to which the merged file is transmitted in order to determine authenticity.

[0038] The above-described image processing apparatus according to the present technology includes: a content registration unit that registers a merged file including captured image data, metadata associated with the captured image data, and signature data generated by using a hash value calculated for the captured image data and the metadata associated with the captured image data; and a signature verification unit that calculates a hash value by using the captured image data in the merged file acquired by the information management unit and the metadata associated with the captured image data, and determines authenticity of the calculated hash value and the hash value used in the signature data of the registered merged file.

[0039] Thus, since a different hash value is generated even when the captured image data in the registered merged file and the captured image data as a verification target are the same as each other, in a case where the metadata associated with the captured image data is different, it is determined that the captured image does not have authenticity.

[0040] In the above-described image processing apparatus according to the present technology, the signature data is generated by using the calculated hash value and a private key in a public key encryption scheme, and the signature verification unit acquires the calculated hash value from the signature data by using a public key.

[0041] When a user who took an image of the captured image data encrypts the signature data using a private key in a public key encryption scheme and transmits the signature data, and a third party who received the signature data can decrypt the signature data using the public key, it can be verified that the hash value acquired from the signature data is the hash value in the captured image data of the user who took the image.

[0042] In the above-described image processing apparatus according to the present technology, the authenticity determining unit determines authenticity in consideration of the presence or absence of other captured image data whose image is captured before or after the imaging time of the captured image data.

[0043] Having captured images taken before or after the imaging time of the captured image data serves as a basis for increasing reliability that the image is captured by the user himself or herself.

[0044] In the above-described image processing apparatus according to the present technology, the authenticity determining unit determines authenticity in consideration of the presence or absence of other captured image data whose angle of view is wider than the angle of view of the captured image data.

[0045] Having other captured image data whose angle of view is wider than the angle of view of the captured image data serves as a basis for increasing reliability that the image is captured by the user himself or herself.

[0046] Further, an imaging apparatus according to the present technology includes an imaging unit, and a setting unit that selects metadata related to authenticity verification in metadata including information on a plurality of items associated with captured image data, based on a feature of the captured image data obtained by the imaging unit.

[0047] Thus, the imaging apparatus realizes a process or another process in which an item reflecting a feature of the captured image data is selected.

[0048] In an image processing method according to the present technology, based on a feature of captured image data, an image processing apparatus performs selection of metadata related to authenticity verification in metadata including information on a plurality of items associated with the captured image data.

[0049] Further, in an image processing method according to the present technology, an image processing apparatus performs

[0050] From a merged file including captured image data and metadata in which an item selected from information on a plurality of items associated with the captured image data is selected as an item related to authenticity verification, information on the selected item is acquired, and authenticity of the feature of the captured image data and the selected item is determined.

[0051] A program according to the present technology causes an image processing apparatus to perform each process corresponding to the above-described image processing method. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is an explanatory diagram of a verification system in an embodiment of the present technology.

[0053] Figure 2 is an explanatory diagram of an apparatus used in an embodiment.

[0054] Figure 3 is a block diagram of an imaging device in an embodiment.

[0055] Figure 4 is a block diagram illustrating an outline of an external terminal and an imaging device in an embodiment.

[0056] Figure 5 is an explanatory diagram showing items corresponding to image features in an embodiment.

[0057] Figure 6 is an explanatory diagram showing a structure of an image file in an embodiment.

[0058] Figure 7 is a block diagram illustrating a computer device in an embodiment.

[0059] Figure 8 is a block diagram showing an outline of a verification device in an embodiment.

[0060] Figure 9 is a flowchart showing a processing example of an analysis unit in an embodiment.

[0061] Figure 10 is a flowchart showing a processing example of a verification device in an embodiment.

[0062] Figure 11 is a diagram illustrating a first display example of a verification result in an embodiment.

[0063] Figure 12 is a diagram illustrating a second display example of a verification result in an embodiment.

[0064] Figure 13 is a diagram illustrating a third display example of a verification result in an embodiment.

[0065] Figure 14 is a diagram illustrating a fourth display example of a verification result in an embodiment.

[0066] Figure 15 is a diagram illustrating a fifth display example of a verification result in an embodiment.

[0067] Figure 16 is a diagram illustrating a sixth display example of a verification result in an embodiment.

[0068] Figure 17 is a flowchart showing a processing example of a terminal on a distribution service side in an embodiment. DETAILED DESCRIPTION

[0069] Hereinafter, the embodiments will be explained in the following order.

[0070] <1. Verification system in this embodiment>

[0071] <2. Configuration of device suitable as image processing device>

[0072] <3. Configuration of imaging device>

[0073] <4. Configuration of verification device>

[0074] <5. Process to realize embodiment>

[0075] <6. Summary and modification>

[0076] Note that in the following, the same reference numerals are used to indicate the same elements and structures as those described above, and the description thereof is omitted.

[0077] <1. Verification system in this embodiment>

[0078] The present technology relates to a process to verify, for example, that captured image data is not a copy of a captured image taken by another user, but captured image data taken by the user himself or herself (hereinafter, also indicated as the captured image is authentic).

[0079] One example of a verification system using the present technology in this embodiment will be described below with reference to Figure 1

[0080] In the present verification system, for example, a user 100, a certificate authority 200, a distribution service 300, and a verification service 400 appear.

[0081] The user 100 is a person who captures an image by an imaging device or the like. The certificate authority 200 is an organization that issues a user identification module (UIM) to certify that a captured image taken by the user 100 is a captured image taken by the user 100 himself or herself.

[0082] The user 100 accesses a website of the certificate authority 200, inputs each item of information such as a name, a company name, an address, and an occupation as personal identification information, and pays a fee such as an annual usage fee, thereby obtaining a UIM issued from the certificate authority 200. The UIM is issued as an external terminal such as a universal serial bus (USB). In the UIM, a key pair of a private key in a public key encryption scheme and a public key associated with the private key is recorded, and a public key certificate on each item of information such as the name of the user 100, that is, user identification information, is recorded.

[0083] The user 100 connects the UIM issued by the certificate authority 200 to an imaging device of the user 100 himself or herself, thereby providing signature data for captured image data taken by the imaging device.

[0084] ​The distribution service 300 is, for example, a provision service of an image posting site or an SNS, and is a service capable of performing posting of captured images. The user 100 posts captured image data taken by the user 100 via the distribution service 300.

[0085] In response to a request from an operator or the like of the distribution service 300, the verification service 400 determines whether or not a posted captured image is not a duplicate but a captured image taken as the user's 100 own creation, and notifies the distribution service 300 of the result.

[0086] In addition, in the verification service 400, the user 100 who took the captured image registers captured image data having signature data provided by the UIM by himself or herself, and based on the registered captured image data, the verification service 400 can determine whether or not a captured image posted to the distribution service 300 is a captured image taken by the user 100 who posted the captured image by himself or herself.

[0087] As described above, the verification service 400 determines whether or not captured image data distributed in the distribution service 300 is a captured image taken based on the user's 100 own creation as a poster, and notifies the distribution service 300 of the determination result, thereby making it possible to implement the present verification system.

[0088] <2. Configuration of a device suitable as an image processing device>

[0089] Hereinafter, an example in which an image processing device in the present technology is implemented in an imaging device used by the user 100 will be described. The image processing device is implemented in various devices.

[0090] Various devices to which the technology of the present disclosure is applicable will be described. Figure 2 Examples of devices that can be an image processing device will be described.

[0091] As a device that can be an image processing device, an imaging device 1 such as a digital camera 1A and a digital video camera IB having an imaging function, a mobile terminal 2 such as a smart phone having an imaging function, and the like are assumed.

[0092] For example, in the imaging device 1, a microcomputer or the like within the imaging device 1 performs various processes on captured image data. The mobile terminal 2 also has an imaging function, and thus can perform various processes on captured image data based on a captured image obtained by imaging.

[0093] The captured image is displayed on a display portion of the imaging device 1 or the mobile terminal 2 in accordance with the output captured image data.

[0094] Note that the device that can be an image processing device is not limited to the imaging device 1 and the mobile terminal 2, and various devices other than these can be considered.

[0095] As described above, although the device that can function as the image processing device in the embodiment is various, hereinafter, an example in which the imaging device 1 as a digital camera 1A is implemented as the image processing device will be described.

[0096] <3. Configuration of the imaging device>

[0097] Reference Figure 3 , a configuration example of the imaging device 1 as the image processing device will be described.

[0098] As Figure 3 illustrated in FIG. 1, the imaging device 1 has an optical system 11, a driver unit 12, an imaging unit 13, a camera signal processing unit 15, a recording control unit 16, a presentation unit 17, a communication unit 18, an operation unit 19, a camera control unit 20, an analysis unit 21, a connector unit 22, a memory unit 23, and a sensor unit 24.

[0099] The optical system 11 includes lenses such as a cover glass, a zoom lens, and a focus lens, an aperture mechanism, and the like. Light (incident light) from a subject is guided and converged to the imaging unit 13 by the optical system 11.

[0100] The driver unit 12 is provided with, for example, a motor driver for a zoom lens driving motor, a motor driver for a focus lens driving motor, a motor driver for an aperture mechanism driving motor, a shutter driver for a shutter driving motor, and the like.

[0101] In response to an instruction from the camera control unit 20 or the camera signal processing unit 15, the driver unit 12 applies a driving current to the corresponding driver, thereby performing movement of the focus lens and the zoom lens, opening / closing of the aperture blade of the aperture mechanism, operation of the shutter, and the like.

[0102] The aperture mechanism is driven by the aperture mechanism driving motor, and controls the amount of light of the incident light into the imaging unit 13 described later. The focus lens is driven by the focus lens driving motor, for focusing. The zoom lens is driven by the zoom lens driving motor, for adjustment of zooming. The shutter mechanism is driven by the shutter driving motor, and operates the shutter.

[0103] The imaging unit 13 is configured to have, for example, an image sensor 14 (image sensor), which is a complementary metal-oxide semiconductor (CMOS) type image sensor, a charge-coupled device (CCD) type image sensor, or the like. The image sensor is constituted by imaging pixels for capturing an image of a subject, and image plane difference pixels for detecting a phase difference between the subject and an optical image.

[0104] In the imaging unit 13, for the electrical signal obtained by subjecting the light received by the image sensor to photoelectric conversion, for example, a correlated double sampling (CDS) process, an automatic gain control (AGC) process, and the like are executed, and furthermore, an analog / digital (A / D) conversion process is executed. The imaging unit 13 outputs the imaging signal as digital data to the camera signal processing unit 15 and the camera control unit 20.

[0105] The imaging pixels of the image sensor 14 store charges corresponding to the intensity of the received light.

[0106] The imaging pixels are covered with a color filter such as a Bayer array, and a plurality of pixel groups are formed in a manner that a pixel having spectral sensitivity of R is arranged at a lower left position, pixels having spectral sensitivity of G are arranged at upper left and lower right positions, and a pixel having spectral sensitivity of B is arranged at an upper right position.

[0107] From the electrical signal obtained by subjecting the light received by these pixel groups to photoelectric conversion, an imaging signal can be read.

[0108] The image sensor 14 outputs the imaging signal to the camera signal processing unit 15 and the camera control unit 20.

[0109] The image surface parallax pixel detects parallax information. The image surface parallax pixel detects a pair of parallax signals, and the imaging unit 13 outputs the pair of parallax signals detected by the image surface parallax pixel. The above parallax signals are used, for example, for a correlation calculation to calculate the distance from the imaging device 1 to the imaging object (subject).

[0110] The imaging unit 13 outputs the parallax signal to the camera signal processing unit 15 and the camera control unit 20.

[0111] The camera signal processing unit 15 is configured as an image processor, for example, by a digital signal processor (DSP) or the like.

[0112] The camera signal processing unit 15 performs various signal processing on the digital signal (captured image signal) from the imaging unit 13. For example, the camera signal processing unit 15 performs a pre-processing, a synchronization processing, a YC generation processing, various correction processing, a resolution conversion processing, and the like.

[0113] In the pre-processing, a clamping processing to clamp the black level of each of R, G, and B to a predetermined level, a correction processing between the color channels of R, G, and B, and the like are performed on the captured image signal from the imaging unit 13.

[0114] In the synchronization processing, a color separation processing to make the image data on each pixel have all the color components of R, G, and B is performed. For example, in the case of an image sensor using a color filter of a Bayer array, as the color separation processing, a demosaicing processing is performed.

[0115] In the YC generation processing, luminance (Y) signals and color (C) signals are generated (separated) from the image data of R, G, and B.

[0116] In the resolution conversion processing, resolution conversion processing is performed on the image data that has undergone various signal processing.

[0117] In the codec processing in the camera signal processing unit 15, encoding processing for recording and communication, and file generation are performed on the image data that has undergone the above-described various processing, for example. For example, an image file MF in MP4 format that records video and audio in conformity with MPEG-4 is generated. In addition, as a still image file, a file of Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), Graphics Interchange Format (GIF), or the like is also generated.

[0118] The recording control unit 16 performs recording and reproduction with respect to a recording medium constituted of a nonvolatile memory, for example. The recording control unit 16 performs processing such as recording of image files such as still image data and moving image data, a thumbnail image, generated defocus map data, and the like into a storage medium described later with respect to the recording medium.

[0119] The actual configuration of the recording control unit 16 can be considered in various ways. For example, the recording control unit 16 can be constituted of a flash memory and a write / read circuit thereof built into the imaging apparatus 1, or can be constituted of a recording medium that can be attached and detached with respect to the imaging apparatus 1, such as a card recording / reproduction unit that performs recording / reproduction access with respect to a memory card (portable flash memory or the like). In addition, as a configuration in which the recording control unit 16 is built into the imaging apparatus 1, in some cases, the recording control unit 16 is realized as a hard disk drive (HDD) or the like.

[0120] The presentation unit 17 has a display portion that performs various displays with respect to the imager, such as a display panel or viewfinder that is located in the housing of the imaging apparatus 1 as a display device such as a liquid crystal display (LCD) and an electroluminescence (EL) display.

[0121] Note that the presentation unit 17 can have an audio output portion such as a speaker, and audio can be output from the audio output portion after a digital audio signal read by the camera control unit 20 is converted into an audio signal by the camera signal processing unit 15, for example.

[0122] The display section of the presentation unit 17 performs various displays on the display screen based on the instruction of the camera control unit 20. For example, the captured image data subjected to resolution conversion by the camera signal processing unit 15 for display is supplied, and the display section performs display based on the captured image data in response to the instruction of the camera control unit 20. Thus, a so-called through image (monitor image of the subject) that is the captured image in the standby state is displayed.

[0123] In addition, the display section displays a reproduction image of the captured image data read from the recording medium in the recording control unit 16.

[0124] The display section performs display of various operation menus, icons, messages, and the like, that is, a graphical user interface (GUI), based on the instruction of the camera control unit 20.

[0125] The communication unit 18 performs data communication and network communication with external devices wired or wirelessly.

[0126] For example, the communication unit 18 transmits and outputs the captured image data (still image file or moving image file) to external display devices, recording devices, reproduction devices, and the like.

[0127] In addition, the communication unit 18 can communicate through various networks such as the Internet, a home network, a local area network (LAN), and the like as a network communication unit, and can transmit and receive various data to and from servers, terminals, and the like on each network.

[0128] The operation unit 19 generally represents an input device with which the user performs various operation inputs. Specifically, the operation unit 19 represents various operation buttons (keys, dials, touch panels, touch pads, and the like) provided on the housing of the imaging device 1.

[0129] The operation performed by the user is detected by the operation unit 19, and a signal corresponding to the input operation is transmitted to the camera control unit 20.

[0130] The camera control unit 20 is constituted by a microprocessor (arithmetic processing unit) including a central processing unit (CPU).

[0131] The memory unit 23 stores information and the like used for processing by the camera control unit 20. The illustrated memory unit 23 generally represents, for example, a read only memory (ROM), a random access memory (RAM), a flash memory, and the like.

[0132] The memory unit 23 can be constituted by a memory area built in a microcomputer chip that is the camera control unit 20, or can be constituted by a separately provided memory chip.

[0133] The camera control unit 20 executes a program stored in a ROM, a flash memory, or the like of the memory unit 23, thereby controlling the entire imaging device 1.

[0134] For example, the camera control unit 20 controls the shutter speed of the imaging unit 13, instructs the processing of each signal in the camera signal processing unit 15 and the analysis unit 21, acquires lens information, performs imaging operation and recording operation in accordance with an operation performed by a user, controls the start / end of moving image recording, performs reproduction operation of a recorded image file, camera operation such as zooming, focusing, exposure adjustment of a lens barrel, user interface operation, and the like, and controls the operation of each unit and portion required.

[0135] In addition, a real-time clock (RTC) is built in the camera control unit 20.

[0136] The analysis unit 21 is constituted by a microcomputer (arithmetic processing unit) including a CPU.

[0137] The analysis unit 21 determines a feature of captured image data, selects a project corresponding to the determined feature in metadata containing information on a plurality of projects associated with the captured image data, and sets mark information of the selected project in the metadata.

[0138] In addition, the analysis unit 21 combines the captured image data and the metadata in which the mark information is set, so that the combined captured image data and metadata can be recorded.

[0139] Note that the analysis unit 21 also generates metadata added to an image file by using information and the like from the camera control unit 20.

[0140] Details of the analysis unit 21 will be described later.

[0141] The connector unit 22 includes a holder of a remote terminal, a USB terminal, or the like, or a jack of an AC power source or the like, and constitutes, for example, an interface with an external terminal described later.

[0142] The RAM of the memory unit 23 serves as a work area when the CPU of the camera control unit 20 processes various data, and is used for temporarily storing data, programs, and the like.

[0143] The ROM and the flash memory (nonvolatile memory) of the memory unit 23 are used for storing an operating system (OS) used by the CPU to control the operation of each unit and portion, a content file such as an image file, and, in addition thereto, an application program, firmware, and the like used for various operations.

[0144] The sensor unit 24 generally represents various sensors installed in the imaging device 1. As the sensor unit 24, for example, a position information sensor such as a global positioning system (GPS), a temperature sensor, an air pressure sensor, an illuminance sensor, an acceleration sensor, and the like are installed.

[0145] The processing to implement the present technology is performed by the imaging device 1 including the above-described functions.

[0146] An outline of the functional configuration of the analysis unit 21 to implement the present technology is described here with reference to Figure 4 , for ease of explanation, Figure 4 Other functional configurations in the imaging device 1 are schematically illustrated in a simplified manner.

[0147] The analysis unit 21 has a feature determination unit 25, an item information acquisition unit 26, a setting unit 27, a hash value calculation unit 28, and a merge processing unit 29.

[0148] The feature determination unit 25 acquires the captured image data from the camera signal processing unit 15 and performs image analysis processing on the acquired captured image data, thereby determining the feature of the captured image data.

[0149] The feature of the captured image data is a feature that implies the content of the captured image data, such as whether the captured image is a “close-up” image, whether the captured image is a “long shot” image, whether the captured image is a “night scene” image, whether the captured image is a “sunny day (day scene)” image, whether the captured image is an image whose imaging subject is “a person”, whether the captured image is an image that causes “blurring”, whether the captured image is a “snow scene” image, whether the captured image is an image captured during “mountain climbing”, whether the captured image is an image in which a flying object is included in the subject, and the like.

[0150] Note that the feature determination unit 25 is not an essential component of the analysis unit 21, and for example, the analysis unit 21 can transmit the captured image data acquired from the camera signal processing unit 15 to an external device. In the case of transmission, the transmission destination is, for example, a specific recording device, an information processing terminal, a cloud server, or the like.

[0151] For example, during the capturing of a moving image using the imaging device 1, the analysis unit 21 transmits the acquired captured image data to a cloud server via a network. On the cloud server side, image analysis processing is performed on the acquired captured image data, thereby determining the feature of the captured image data, and the determination result is transmitted to the imaging device 1 having the analysis unit 21.

[0152] In addition, the feature determination unit 25 generates and acquires metadata containing information of a plurality of items associated with the captured image data. The feature determination unit 25 acquires the information of the plurality of items from the item information acquisition unit 26.

[0153] The item information acquisition unit 26 acquires information of a plurality of items via the camera control unit 20.

[0154] The information of a plurality of items associated with the captured image data is information related to the state of the imaging device 1 at the time of image capture, for example, information related to "distance to subject", "focal length of imaging lens", "position at captured image", "brightness (EV value) of image capture environment", "sensitivity (ISO sensitivity)", "aperture (F value) setting value", "shutter speed (SS value)", "date and time at captured image", "air temperature", "air pressure", and the like. The information of a plurality of items associated with the captured image data is used as metadata related to the authenticity verification described later.

[0155] In addition, as shown in Figure 5 The feature determination unit 25 selects one or more items corresponding to the feature determined based on the image analysis processing from among the metadata including the information of a plurality of items associated with the captured image data.

[0156] In the case where the feature of the captured image is "close-up", for example, "distance to subject" and "focal length of imaging lens" are selected from among the above-described plurality of items.

[0157] In the case where the feature of the captured image is "long shot", for example, "distance to subject" and "position at captured image" are selected from among the above-described plurality of items.

[0158] In the case where the feature of the captured image is "night scene" or "sunny day (day scene)", for example, "brightness (EV value) of image capture environment" and "date and time at captured image" are selected from among the above-described plurality of items.

[0159] In the case where the feature of the captured image is "person" as a subject, for example, "distance to subject" and "focal length of imaging lens" are selected from among the above-described plurality of items.

[0160] In the case where the feature of the captured image is "blurring", for example, "distance to subject", "focal length of imaging lens", and "aperture (F value) setting value" are selected from among the above-described plurality of items.

[0161] In the case where the feature of the captured image is "snow scene", for example, "air temperature", "date and time at captured image", and "position at captured image" are selected from among the above-described plurality of items.

[0162] In the case where the feature of the captured image is that the image is captured during "mountain climbing", for example, "air pressure" and "position at captured image" are selected from among the above-described plurality of items.

[0163] In the case where the characteristic of the captured image is to capture a "moving object", for example, the "shutter speed (SS) value" is selected from the above-described plurality of items.

[0164] As described above, by selecting one or more items in accordance with the characteristic of the captured image, in determining whether the captured image is taken by the user himself or herself, if the value of the selected one or more items is an unnatural value from the characteristic of the captured image, it can be presumed that the image is not captured by the user based on his or her own creation, such as in the case where the user copied an image captured by a third party and other cases.

[0165] In the metadata including information of the plurality of items associated with the captured image data, the setting unit 27 selects metadata related to the authenticity verification, and sets the mark information (setting flag) in the metadata related to the authenticity verification, based on the characteristic of the captured image data.

[0166] The hash value calculation unit 28 calculates a hash value with respect to data in which the captured image data and the metadata associated with the captured image data are merged. The metadata here includes information of the plurality of items associated with the captured image data, and the mark information of one or more items selected in accordance with the characteristic of the captured image data. In addition, the hash value calculation unit 28 transmits the calculated hash value to the generation control unit 30 of the external terminal 3.

[0167] In addition, the metadata used by the hash value calculation unit 28 when calculating the hash value only needs to include information of the items selected (marked) in accordance with the characteristic of the captured image data. In other words, the metadata used when calculating the hash value does not necessarily need to include all information of the plurality of items associated with the captured image data.

[0168] The external terminal 3 is a UIM issued by the certificate authority center 200, and is, for example, a terminal such as a USB that authenticates a user who captures an image using the imaging device 1. The UIM has a structure that does not leak a private key to the outside, thereby making it difficult to be copied.

[0169] The external terminal 3 is connected via the connector unit 22 of the imaging device 1 (see Figure 2 ).

[0170] In the storage unit 31 of the external terminal 3, a private key used for generating signature data, user identification information associated with the private key, and information of a public key associated with the private key are recorded.

[0171] The generation control unit 30 acquires the private key from the storage unit 31, and encrypts the hash value acquired from the hash value calculation unit 28 using the private key, thereby generating signature data. Thus, a third party can acquire the hash value acquired from the hash value calculation unit 28 by decrypting the signature data using the public key.

[0172] The generation control unit 30 transmits the generated signature data and the public key to the hash value calculation unit 28. The hash value calculation unit 28 transmits the acquired signature data and the public key to the merging processing unit 29.

[0173] The merging processing unit 29 merges the signature data generated by using the hash value, the captured image data, and the metadata in which the mark information is set, thereby generating an image file as shown in Figure 6 After that, the generated image file is stored in the storage medium 32 by the recording control unit 16 illustrated in Figure 3

[0174] In order to prove that the image is captured by the user himself / herself, it is necessary to register the generated image file at the verification device 4 side. Therefore, the communication unit 18 transmits the image file generated by the merging processing unit 29 to the verification device 4. At this time, the communication unit 18 acquires the public key and the user identification information from the external terminal 3 and transmits the public key and the user identification information to the verification device 4.

[0175] Note that, although in the present embodiment, the generation control unit 30 and the storage unit 31 are provided in the external terminal 3 provided separately from the imaging device 1, the generation control unit 30 and the storage unit 31 can be provided as a function of the analysis unit 21 of the imaging device 1.

[0176] The present embodiment is implemented by the above-described imaging device 1.

[0177] Although in the above description, an example in which the present technology is implemented by the imaging device 1 is described, the present technology can also be implemented by a mobile terminal 2 having an imaging function such as a smart phone.

[0178] In the case where the imaging control processing is performed by the mobile terminal 2, for example, the present technology can be implemented as a computer device 70 including the configuration illustrated in Figure 7

[0179] Figure 7 According to a program stored in a read only memory (ROM) 72 or a program loaded from a storage unit 79 to a random access memory (RAM) 73, a central processing unit (CPU) 71 of the computer device 70 performs various processing. In the RAM 73, data and the like necessary for the CPU 71 to perform various processing are also appropriately stored. The CPU 71 is provided as the analysis unit 21 of the image processing device.

[0180] The CPU 71, the ROM 72, and the RAM 73 are connected to each other via a bus 74. An input / output interface 75 is also connected to the bus 74.

[0181] ​​Connected to the input / output interface 75 is an input unit 76 constituted by a keyboard, a mouse, a touch panel, and the like; a lens system constituted by an imaging lens, an aperture, a zoom lens, a focus lens, and the like; a drive system for causing the lens system to perform a focus operation and a zoom operation; and an imaging unit 77 constituted by a solid-state image sensor array that detects imaging light obtained through the lens system and performs photoelectric conversion, thereby generating an imaging signal; a display constituted by a liquid crystal display (LCD), a cathode ray tube (CRT), an organic electroluminescence (EL) panel, and the like; an output unit 78 constituted by a speaker and the like; and a hard disk drive (HDD).

[0182] For example, based on an instruction of the CPU 71, the output unit 78 performs display of an image for various image processing, a moving image as a processing target, and the like on a display screen. In addition, based on an instruction of the CPU 71, the output unit 78 performs display of various operation menus, icons, messages, and the like, that is, a graphical user interface (GUI).

[0183] There is also a case where a storage unit 79 constituted by a hard disk, a solid-state memory, and the like and a communication unit 80 constituted by a modem are connected to the input / output interface 75.

[0184] The communication unit 80 performs communication processing via a transmission path such as the Internet, wired / wireless communication with various devices, and communication through bus communication and the like.

[0185] In addition, a drive 81 is connected to the input / output interface 75 as necessary, and a detachable recording medium 82 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted on the drive 81.

[0186] Through the drive 81, a data file such as an image file, various computer programs, and the like can be read from the detachable recording medium 82. The read data file is stored in the storage unit 79, and an image and a sound contained in the data file are output by the output unit 78. In addition, the computer program and the like read from the detachable recording medium 82 are installed in the storage unit 79 as necessary.

[0187] In this computer device 70, for example, software for image processing as an image processing device of the present disclosure can be installed via network communication through the communication unit 80 or the detachable recording medium 82. Alternatively, the above-described software can be stored in the ROM 72, the storage unit 79, and the like in advance.

[0188] Note that the computer device 70 is not limited to being as described above. Figure 7The single device illustrated in the diagram can be constituted in a systematic manner by a plurality of computer devices. The plurality of computer devices can include computer devices as a server group (cloud) that can be used through a cloud computing service.

[0189] The present technology can also be implemented by the mobile terminal 2 having the analysis unit 21 as an image processing device having the above-described functions.

[0190] <4. Configuration of the verification device>

[0191] A configuration example of the verification device 4 as an image processing device will be described below with reference to Figure 8

[0192] As illustrated in the diagram Figure 8 The verification device 4 has a user authentication unit 41, a content registration unit 42, an information management unit 43, a signature verification unit 44, a scene analysis unit 45, a similarity determination unit 46, a reality determination unit 47, and a crawling unit 48. In addition, the verification device 4 has an image database (described below as an image database (DB)) 50 and a feature point DB 51.

[0193] The user authentication unit 41 performs user authentication when a user uses the verification system. In other words, the user authentication unit 41 determines whether the user who is to use the verification system is a user to whom a UIM has been issued from the certificate authority center 200.

[0194] The user authentication unit 41 acquires user identification information (public key certificate) from the external terminal 3 and performs authentication that the user is a correct user by using a challenge response or the like. In addition, the user authentication unit 41 acquires information (expiration information) on whether the user identification information is valid from the certificate authority center 200 and performs cross-checking, thereby performing more accurate user authentication.

[0195] At this time, the user authentication unit 41 acquires the user identification information stored in the external terminal 3 via the image forming device 1. Note that the user authentication unit 41 can directly acquire the user identification information from the external terminal 3.

[0196] When the user authentication performed by the user authentication unit 41 is successful, the content registration unit 42 associates an image file in which signature data acquired from the image forming device 1, captured image data, and metadata in which mark information is set are merged with the user identification information and stores the image file associated therewith in the image DB 50.

[0197] Thus, the captured image taken by the user can be registered in the verification device 4.

[0198] ​When the signature is verified in response to a request from the distribution service 300 or the user, an image file of the captured image data for which verification is expected is input into the information management unit 43.

[0199] At this time, the information management unit 43 determines whether or not there is signature data having the user identification information in the input image file.

[0200] When there is signature data having the user identification information in the image file input into the information management unit 43, the signature verification unit 44 confirms to the certificate authority center 200 whether or not the public key certificate associated with the signature data is not expired. When the public key certificate is not expired, the signature verification unit 44 verifies the signature data.

[0201] When there is no signature data in the image file input into the information management unit 43, the similarity determination unit 46 determines whether or not the registered captured image is similar to the captured image in the input image file.

[0202] The scene analysis unit 45 determines a feature point regarding "what is in the image" for the captured image registered in advance in the image DB 50 by using AI such as deep learning, associates the feature point with the user identification information, and stores the feature point associated therewith in the feature point DB 51. For example, by determining a feature point of "three black dogs are sitting on a green grassland" in advance.

[0203] The scene analysis unit 45 acquires captured image data as a verification object from the similarity determination unit 46, and determines a feature point similarly for the captured image as a verification object.

[0204] The similarity determination unit 46 acquires a feature point determined for the captured image as a verification object, and determines a matching degree (key point matching) between the acquired feature point and a feature point stored in the feature point DB 51.

[0205] Note that the determination result determined by the similarity determination unit 46 can be a determination result such as "a matching probability is 90%" rather than matching / non-matching.

[0206] When there is signature data in the acquired image file, the authenticity determination unit 47 can not only perform signature verification but also determine the content of the captured image data and the metadata associated with the captured image data. This is because when the determination is made using only the user identification information of the signature data, since even if the work is copied, the person who copied the work is provided with the legal signature data of the person, it is not possible to determine the authenticity regarding whether or not the image of the work is captured by the user's own creation (non-copy).

[0207] In this case, first, the authenticity determination unit 47 determines matching between the content of the metadata of the item associated with the captured image data and the feature points. For example, when the image is a landscape of a distant view, but the focus point of the focus is at a close distance, matching cannot be performed, i.e., there is a possibility that the image is a copy (there is a possibility that the image does not have authenticity).

[0208] The information management unit 43 notifies the distribution service 300 and the user of the determination result determined by the signature verification unit 44, the similarity determination unit 46, and the authenticity determination unit 47 in combination.

[0209] The crawling unit 48 can acquire the captured image 60 distributed on the Internet like so-called crawling, not only upon a request of the distribution service 300 and the user. For example, the signature verification unit 44 verifies the signature data for the image data acquired by the crawling unit 48, so that it can determine whether the captured image data is illegally distributed.

[0210] When the crawling unit 48 determines that the captured image data is illegally used, the crawling unit 48 can also issue a warning to the upload destination of the captured image data or the like.

[0211] The present embodiment is implemented by the above-described verification device 4.

[0212] <5. Processing to realize the embodiment>

[0213] The processing performed by the analysis unit 21 as an image processing device in the imaging device 1 in order to realize the above-described embodiment will be described below with reference to Figure 9

[0214] The analysis unit 21 performs, for example, the processing shown in Figure 9 at the timing when the imaging device 1 captures an imaging object.

[0215] Figure 9 In the processing shown in

[0216] Then, in step S102, the analysis unit 21 generates and acquires metadata containing information of a plurality of items associated with the captured image data.

[0217] Here, the information of each item as the plurality of items is information at the timing of image capture, such as "distance to the subject", "focal length of the imaging lens", "position at the captured image", "brightness (EV value) of the image capture environment", "sensitivity (ISO sensitivity)", "aperture (F value) setting value", "shutter speed (SS value)", "date and time at the captured image", "air temperature", and "air pressure".

[0218] ​In step S103, the analysis unit 21 performs image analysis processing on the acquired captured image data, and then in step S104, characteristic determination processing of the captured image is performed in accordance with the image analysis result.

[0219] Then, in step S105, the analysis unit 21 selects from the above items in accordance with the determined characteristic of the captured image, and performs processing in which the selected item is labeled.

[0220] Various examples can be considered for the processing from step S103 to step S105. Here, an example of the above processing will be described.

[0221] (1) Close-up

[0222] For example, in captured image data acquired when a cherry blossom tree is captured as a subject by the imaging device 1, as a result of the image analysis processing, when the cherry blossoms are shown large on the captured image, it can be inferred that the image was captured at close range. Then, the analysis unit 21 determines that the characteristic of the captured image is "close-up".

[0223] In captured image data having the above characteristic, when the distance to the subject is far, or the focal length of the lens is deviated, it can be inferred that the user did not actually capture the image with the cherry blossom as the subject, but the captured image was captured again to make a copy or the like.

[0224] Then, in the case where the characteristic of the captured image is "close-up", the analysis unit 21 selects, for example, "distance to the subject" and "focal length of the imaging lens" from the above plurality of items, and labels them.

[0225] (2) Landscape

[0226] In addition, as a result of the image analysis processing, when it is determined that the image is a landscape of a distant view, the analysis unit 21 determines that the characteristic of the captured image is "landscape".

[0227] In captured image data having the above characteristic, when the distance to the subject is short, it is not a distant view, it can be inferred that the user did not go to the actual place to capture the image, but the user made a copy or the like. In addition, if the name of the place of the "landscape" of the distant view can be recognized, information of the position at which the user captured the image can be acquired, so that it becomes possible to determine whether the user actually visited the place having the name of the place to capture the image.

[0228] Then, in the case where the characteristic of the captured image is "landscape", the analysis unit 21 selects, for example, "distance to the subject" and "position at which the captured image was captured" from the above plurality of items, and labels them.

[0229] (3) Night view or sunny day (day view)

[0230] As a result of the image analysis processing, when it is determined that the captured image is a night view, the analysis unit 21 determines that the characteristic of the captured image is "night view".

[0231] In the captured image data having the above-described characteristic, the brightness (EV value) of the image capturing environment is low, and it is considered that the time of day when the captured image is taken is at night. Thus, when the EV value is high although it is a night view, or the time when the captured image is taken is during the day, it can be presumed that the captured image of the night view is copied. In addition, when it is determined that the characteristic of the captured image is "sunny day (day view)", a similar method is also applicable. In other words, when the brightness (EV value) of the image capturing environment and the time of day when the captured image is taken are inconsistent with the characteristic of the captured image, it can be presumed that copying is performed.

[0232] Thus, in the case where the characteristic of the captured image is "night view" or "sunny day (day view)", the analysis unit 21 selects, for example, "brightness (EV value) of the image capturing environment" and "date and time when the captured image is taken" from among the above-described plurality of items, and marks them.

[0233] (4) Person

[0234] As a result of the image analysis processing, for example, when the entire body of a person is in the image, the analysis unit 21 determines that the characteristic of the captured image is "person".

[0235] In the captured image data having the above-described characteristic, it can be presumed that the focal length of the standard lens is approximately 2 m to 3 m to the distance to the person as the subject. Thus, when the distance to the person as the subject is greater than or less than 2 m to 3 m, it can be presumed that copying or the like is performed.

[0236] Thus, in the case where the characteristic of the captured image is "person" as the subject, the analysis unit 21 selects, for example, "distance to the subject" and "focal length of the imaging lens" from among the above-described plurality of items, and marks them.

[0237] (5) Blur

[0238] As a result of the image analysis processing, the analysis unit 21 can also determine the degree of the characteristic of the captured image to be, for example, "blur" in which the near field is focused and the far field is blurred. In this case, since the near field is focused and the far field is blurred, it can be presumed that the focal length of the imaging lens is relatively long, and the aperture (F value) set value is small.

[0239] In the captured image data having the above-described characteristic, similarly, when the focal length of the imaging lens and the aperture (F value) set value are different from the presumed values, it can be presumed that copying is performed.

[0240] Thus, in the case where the feature of the captured image is "snow", the analysis unit 21 selects, for example, "distance to the subject", "focal length of the imaging lens", and "aperture (F value) setting value" from among the above-described plurality of items and labels them.

[0241] (6) Snow scene

[0242] As a result of the image analysis processing, when there is ice and snow in the captured image, the analysis unit 21 can determine that the feature of the captured image is "snow scene". In this case, since there is ice and snow in the captured image, it can be inferred that the air temperature at the time of capturing the image was low, the season was winter, and the place where the captured image was captured was in a cold region.

[0243] In the captured image data having the above-described feature, when the air temperature at the time of capturing the image was high, the season at the time of capturing the image was not winter, and the image was captured in a warm region, even though there was ice and snow in the captured image, it can be inferred that the captured captured image was recaptured to make a copy or the like.

[0244] Thus, in the case where the feature of the captured image is "snow scene", the analysis unit 21 selects, for example, "air temperature", "date and time at the time of capturing the image", and "location at the captured image" from among the above-described plurality of items and labels them.

[0245] (7) Mountain climbing

[0246] As a result of the image analysis processing, when the image is a mountain climbing scene, the analysis unit 21 can determine that the feature of the captured image is "mountain climbing". Since the mountain climbing scene is captured in a mountainous region, it is assumed that the altitude is high and the air pressure is low. In addition, if the name of the mountain at the place where the mountain climbing scene is captured can be recognized, the location at the time of capturing the image can be recognized.

[0247] In the captured image data having the above-described feature, when it is a mountain climbing scene, but the data of the air pressure indicates high air pressure and the location data indicates that the user is not in the mountain, it can be inferred that the captured image is not an image captured by the user himself or herself, but the possibility that it is a copy is high.

[0248] Thus, in the case where the feature of the captured image is an image captured during "mountain climbing", the analysis unit 21 selects, for example, "air pressure" and "location at the captured image" from among the above-described plurality of items.

[0249] (8) Moving object

[0250] As a result of the image analysis processing, when a flying bird, a jumping cat, or the like appears in a fixed manner in the captured image, the analysis unit 21 can determine that the characteristic of the captured image is "moving object". Since the flying bird, the jumping cat, or the like appears in a fixed manner in the captured image, it can be presumed that the captured image is taken at a high shutter speed.

[0251] In the captured image data having the above-described characteristic, when although an animal in motion appears in a fixed manner in the captured image, the shutter speed at the time of capturing the image is low, it can be presumed that the captured image is more likely to be a duplicate.

[0252] In the case of capturing where the characteristic of the captured image is "moving object", the analysis unit 21 selects, for example, "shutter speed (SS value)" from the above-described plurality of items.

[0253] In addition to the above-described example, labeling of each item corresponding to the characteristic of each captured image is performed.

[0254] In step S105, the analysis unit 21 performs processing in which labeling of each item corresponding to the characteristic of the captured image is performed, and then in step S106, the analysis unit 21 calculates a hash value for the captured image data and the metadata associated with the captured image data.

[0255] By inclusively calculating a hash value not only for the captured image data but also for the metadata associated with the captured image data, in the verification processing on the verification device 4 side, when compared with the captured image data as a verification object, comparison can be performed including the metadata at the time of capturing the image.

[0256] In step S107, the analysis unit 21 issues an electronic signature processing request. In other words, the analysis unit 21 transmits the calculated hash value to the external terminal 3 and requests generation of signature data.

[0257] Here, the external terminal 3 acquires the private key from the storage unit 31 and encrypts the hash value acquired from the analysis unit 21 using the private key, thereby generating signature data. Thus, a third party having the public key decrypts the signature data, thereby making it possible to acquire the hash value provided for the captured image data.

[0258] Subsequently, the external terminal 3 transmits the generated signature data and the public key to the analysis unit 21. Thus, in step S108, the analysis unit 21 acquires the signature data.

[0259] Subsequently, in step S109, the analysis unit 21 performs file merging processing. In other words, the analysis unit 21 merges the signature data generated by using the hash value, the captured image data, and the metadata in which the labeled information is set, and generates an image file as illustrated in FIG. 2. Figure 6 Subsequently, in step S110, the analysis unit 21 transmits the generated image file to the external terminal 3. Thus, in step S111, the external terminal 3 acquires the image file.

[0260] In step S110, the analysis unit 21 performs recording processing of recording the generated image file in the storage medium 32.

[0261] Thereafter, in step S111, the analysis unit 21 transmits the image file to the authentication device 4. Thus, the image file to be recorded in the authentication device 4 is transmitted.

[0262] As described above, the processing in which the marking of each item corresponding to the features of the captured image is performed by the analysis unit 21 on the imaging device 1 side, and the processing in which the image file to be transmitted to the authentication device 4 is generated are realized.

[0263] Note that, although in the present embodiment, the analysis unit 21 performs the image analysis processing on the acquired captured image data, and performs the feature determination processing of the captured image in accordance with the image analysis result, the image analysis processing and the feature determination processing can be performed by an external device (i.e., a specific recording device, an information processing terminal, a cloud server, or the like).

[0264] In this case, for example, in step S103, the analysis unit 21 transmits the captured image data acquired from the camera signal processing unit 15 to the cloud server. On the cloud server side, the image analysis processing and the feature determination processing are performed, and the determination result obtained by the feature determination processing is transmitted to the imaging device 1 having the analysis unit 21.

[0265] In step S104, the analysis unit 21 acquires the determination result transmitted from the cloud server side. Thereafter, in step S105, the analysis unit 21 performs the marking of the metadata in accordance with the features of the captured image acquired from the cloud server.

[0266] In addition, although in the present embodiment, the external terminal 3 generates the signature data, the analysis unit 21 can generate the signature data in the imaging device 1.

[0267] In this case, in step S107, the analysis unit 21 acquires the information on the private key and the public key from the external terminal 3, and then in step S108, the signature data can be generated by using the acquired private key and the hash value calculated in step S106. The private key can be stored in the imaging device 1.

[0268] In addition, although in the processing shown in Figure 9 In the processing shown in

[0269] Further, in step S110, when the analysis unit 21 stores the generated image file in the storage medium 32, Figure 9 The processing shown in FIG. 6 can be completed at a time, and at the timing when the image file is sent to the verification device 4, in step S111, the image file can be read from the storage medium 32 and sent to the verification device 4.

[0270] Also here, although an example in which the analysis unit 21 performs labeling of a plurality of items associated with the captured image data in accordance with the characteristics of the captured image in the processing of step S105 is explained, a processing example in which the analysis unit 21 is not required to perform labeling can also be considered. Figure 9

[0271] For example, the analysis unit 21 selects one or more items in step S105 in accordance with the characteristics of the captured image determined in step S104, and the metadata contains only the selected one or more items.

[0272] Subsequently, in step S106, the analysis unit 21 calculates a hash value for the captured image data and the metadata associated with the captured image data. The metadata used for the calculation of the hash value is information related to the one or more items selected in accordance with the determination result of the characteristics of the captured image.

[0273] Next, with reference to Figure 10 , the verification processing of the captured image in the verification device 4 performed in order to realize the above-described embodiment will be explained.

[0274] The verification device 4, for example, at the timing when an image file is acquired from the distribution service 300 for which verification is desired to be performed, performs the processing shown in FIG. 6. Figure 10

[0275] First, in step S201, the verification device 4 determines whether or not the signature data is contained in the image file acquired from the distribution service 300.

[0276] When the signature data is not contained in the acquired image file, in step S202, the verification device 4 performs image analysis processing on the captured image data contained in the acquired image file, and extracts feature points of the captured image.

[0277] After that, in step S203, the verification device 4 checks whether or not there is captured image data having the feature points of the captured image extracted in step S202, with reference to the feature point DB 51 as illustrated in FIG. 6. Figure 8

[0278] ​​​In step S203, when there is no captured image data with feature points that are the object of verification, in step S204, the verification device 4 determines that "no similar content exists". In other words, it is determined that there is no captured image registered in the verification device 4 that is similar to the image that is the object of verification.

[0279] At this time, in step S205, the verification device 4 displays the determination on the terminal on the distribution service 300 side.

[0280] For example, such as Figure 11 As shown in the diagram, the captured image, which is the object of verification, is displayed in the image display area 91 of the terminal's display screen 90 on the distribution service 300 side, and the judgment result display area 92 displays the judgment result "No similar content found". Afterwards, the verification device 4 ends. Figure 10 The processing shown.

[0281] On the other hand, in step S203, when there is captured image data with feature points that are the object of verification, in step S206, the verification device 4 determines that "similar content exists".

[0282] Additionally, at this point, the similarity of captured images as similar content can be determined, such as "80% similarity to the xxx photo captured by Mr. / Ms. A".

[0283] For example, similarity can be determined by the number of items whose feature points, as the verification object, match the feature points of similar content.

[0284] In step S205, the verification device 4 displays the determination on the terminal on the distribution service 300 side.

[0285] For example, such as Figure 11 As shown in the diagram, in the image display area 91 of the terminal display screen 90 on the distribution service side 300, captured images that are considered similar are displayed, and in the judgment result display area 92, in addition to the judgment result of "similar content exists", the registration information and similarity of the similar content are also displayed, such as "has 80% similarity to the xxx photo captured by Mr. / Ms. A".

[0286] After that, verification device 4 ended. Figure 10 The processing shown.

[0287] In step S201, when the image file obtained from the distribution service 300 contains signature data, the verification device 4 proceeds to step S207 to determine whether the signature data has expired. The verification device 4 sends the signature data information to the terminal on the certificate authority 200 side and obtains information about whether the signature data has expired from the terminal on the certificate authority 200 side, thereby determining the validity of the signature data.

[0288] When the signature data has expired, the verification device 4 proceeds from the process of step S207 to the process of step S208, and determines that "the signature of the content is invalid".

[0289] After that, in step S205, the verification device 4 displays the determination on the terminal on the distribution service 300 side. After that, the verification device 4 ends the process shown in Figure 10 .

[0290] On the other hand, in step S207, when the signature data has not expired (is valid), the verification device 4 proceeds to the process of step S209, and calculates a hash value by using the captured image data recorded in the image file that is the verification target and the metadata associated with the captured image data.

[0291] In step S210, the verification device 4 determines whether the calculated hash value that is the verification target and the hash value in the registered signature data are identical to each other. When the calculated hash value that is the verification target and the hash value in the registered signature data are not identical to each other, it is considered that some form of tampering has been performed on any one of the captured image data that is the verification target and the metadata associated with the captured image data.

[0292] Then, in step S211, the verification device 4 determines that "the content is invalid".

[0293] In other words, it is determined that the captured image data that is the verification target is captured image data obtained by performing some form of tampering in the registered captured image data.

[0294] In step S205, the verification device 4 displays the determination on the terminal on the distribution service 300 side.

[0295] On the terminal on the distribution service 300 side, for example, as illustrated in Figure 11 , in an image display area 91 of a display screen, the captured image that is the verification target is displayed, and in a determination result display area 92, the determination result that "the content is invalid" is displayed. After that, the verification device 4 ends the process shown in Figure 10 .

[0296] When the calculated hash value that is the verification target and the hash value in the signature data are identical to each other, the verification device 4 proceeds from the process of step S210 to the process of step S212. In step S212, when the distribution service 300 side requests only the verification of the signature data, the verification device 4 proceeds from the process of step S212 to the process of step S213, and determines that "the content is valid".

[0297] In other words, it is determined that the captured image data that is the verification target is captured image data in which there is no tampering or the like of the registered captured image data, and is thus authentic.

[0298] In step S205, the verification device 4 displays the determination on the terminal on the distribution service 300 side.

[0299] On the terminal on the distribution service 300 side, for example, as illustrated in FIG. 9, on the image display area 91 of the display screen 90, the captured image that is the verification target is displayed, and in the determination result display area 92, the determination result of "valid content" is displayed. After that, the verification device 4 ends the processing illustrated in FIG. 8. Figure 11 Figure 10 In step S212, when the distribution service 300 side requests a verification such as authenticity verification of the usage metadata and determination of similarity to the registered captured image in addition to the verification of the signature data, the verification device 4 proceeds from the processing of step S212 to the processing of step S214, and performs image analysis processing on the captured image data that is the verification target. The verification device 4 performs image analysis processing on the captured image data contained in the acquired image file, and extracts feature points of the captured image.

[0300] After that, in step S215, the verification device 4 determines the authenticity of the metadata.

[0301] Specifically, for the image file having the signature data, the verification device 4 extracts one or a plurality of items that are marked for the metadata containing information of a plurality of items associated with the captured image data in the image file. Subsequently, the verification device 4 determines whether or not the value of the one or a plurality of items that are marked is within the range of the value reflecting the feature of the captured image data contained in the image file.

[0302] Note, however, that for the example in which the above-described metadata in the image file is constituted only by the selected one or a plurality of items, the marked information does not need to be added, and in this case, the metadata is constituted only by the one or a plurality of items corresponding to the feature of such captured image data. At this time, the verification device 4 only needs to determine whether or not the value of the information of the one or a plurality of items in the metadata is within the range of the value reflecting the feature of the captured image data contained in the image file.

[0303] In step S215, when the value of the one or a plurality of items that are marked is not within the range of the value reflecting the feature of the captured image data, and thus it is determined that the metadata does not have authenticity, in step S216, the verification device 4 checks whether or not there is captured image data having the feature point of the captured image extracted in step S214, with reference to the feature point DB 51 illustrated in FIG. 10.

[0304] Figure 8

[0305] ​​​In step S216, when there is no registered captured image data with feature points that is the object of verification, in step S217, the verification device 4 determines that "there is a question about the content" and "it is suspected that it may be a copy".

[0306] In other words, even though the captured image is not similar to a registered captured image, it is determined that the captured image may be a copy of another unregistered captured image.

[0307] Therefore, in step S205, the verification device 4 displays the determination on the terminal on the distribution service 300 side and ends the process. Figure 10 The processing shown.

[0308] Here, we can consider various examples of displaying the presentation modes of each decision on the terminal on the distribution service 300 side.

[0309] For example, when the item deemed lacking in metadata authenticity is "distance to the subject," such as... Figure 12 As shown in the diagram, the captured image, which is the object of verification, is displayed in the image display area 91 of the display screen 900 of the terminal on the distribution service 300 side, and the judgment result display area 92 displays the judgment results of "questionable content" and "suspected to be a copy". In addition, in area 93, the value of "distance to the subject" that is judged to lack the authenticity of metadata can be displayed.

[0310] Additionally, when the item deemed lacking in metadata authenticity is "brightness (EV value) of the image capture environment," such as... Figure 13 As shown in the diagram, the captured image, which is the object of verification, is displayed in the image display area 91 of the terminal's display screen 900 on the distribution service side 300, and the judgment result display area 92 displays the judgment results of "questionable content" and "suspected to be a copy". In addition, the EV value that is judged to lack the authenticity of metadata can be displayed in area 94.

[0311] In addition to the items mentioned above, various items associated with the captured image data can be displayed, such as "distance to the subject," "focal length of the imaging lens," "position at the point of image capture," "ISO sensitivity," "aperture (F-number) setting," "shutter speed (SS-number)," "date and time of image capture," "temperature," and "barrier pressure." Furthermore, multiple items such as "distance to the subject" and "brightness (EV) of the image capture environment" can be displayed simultaneously.

[0312] By performing the above display, since the person who checks the determination result can visually compare the captured composition together with the verification result and items such as distance information and EV value, a more level determination can be made for the captured image.

[0313] On the other hand, in step S216, when there is registered captured image data having a feature point as a verification object, in step S218, the verification device 4 determines "there is similar content" and "there is 80% similarity with the xxx photo captured by Mr. / Ms. A".

[0314] In other words, it is determined that the similar content is likely to be a copy of the image captured by the registered Mr. / Ms. A.

[0315] Subsequently, in step S205, the verification device 4 displays the determination on the terminal on the distribution service 300 side, and ends the process shown in Figure 10 As the display mode at this time, the above display as illustrated in Figure 12 and Figure 13 can be considered.

[0316] Returning to step S215, when the value of one or more items marked is within the range of the value reflecting the feature of the captured image data, and thus it is determined that the metadata is authentic, in step S219, the verification device 4 refers to the feature point DB 51 as illustrated in Figure 8 , and checks whether there is registered captured image data having the feature point of the captured image extracted in step S214 as a verification object.

[0317] In step S219, when there is no registered captured image data having a feature point as a verification object, the verification device 4 proceeds to the process of step S213, and determines "the content is valid".

[0318] In other words, it is determined that the captured image data as a verification object is captured image data in which there is no tampering or the like of the registered captured image data, and thus is authentic.

[0319] Subsequently, in step S205, the verification device 4 displays the determination on the terminal on the distribution service 300 side as illustrated in Figure 11 , and then ends the process shown in Figure 10 .

[0320] On the other hand, in step S219, when there is registered captured image data having a feature point as a verification object, in step S220, for the registered captured image data having a feature point (captured image data as a verification object), the verification device 4 determines whether captured image data whose image is specially captured is registered.

[0321] For example, when there are other captured image data whose shooting time points are continuous with the captured image data having the registration of the feature point, before and after the captured image data as the verification target, the verification device 4 determines that the captured image data is data of an image obtained by special imaging. Figure 8 When there are captured images whose shooting time points are continuous in the image DB 50 illustrated in FIG. 6, which are shot by continuous shooting, the verification device 4 determines that the captured image data is data of an image obtained by special imaging.

[0322] By comparing a plurality of continuously captured images, since it is possible to check the movement of a moving object, it is possible to verify the authenticity (originality) of the image, that is, the fact that the imager was actually on the scene and shot the image, with a higher probability.

[0323] Note that although the data of the captured images whose shooting time points are continuous, which are stored in the image DB 50, are captured image data before and after the captured image data as the verification target, it is also possible to determine whether the captured image data is data of an image obtained by special imaging, based only on whether there is captured image data before or after the captured image data as the verification target.

[0324] In addition, when there is other captured image data whose angle of view is wider than the angle of view of the captured image data in the image DB 50, the verification device 4 can determine that the captured image data is data of an image obtained by special imaging.

[0325] Here, the angle of view indicates the range of the image to be captured reflected in the image sensor. Thus, when the angle of view is wide, the imaging range is wide, and conversely, when the angle of view is narrow, the imaging range is narrow.

[0326] For example, when a reduced image of the captured image is stored in the image DB 50 along with the captured image data having the registration of the feature point, the verification device 4 determines that the captured image data is data of an image obtained by special imaging.

[0327] In addition, when the captured image data having the registration of the feature point is a cropped image, and the captured image before the cropping is stored in the image DB 50 along with the cropped captured image, the verification device 4 can also determine that the captured image data is data of an image obtained by special imaging.

[0328] By combining an image whose angle of view is wide (has much information) that only the imager can record, it is verified that the imager was definitely on the scene to capture the image, that is, the authenticity (originality) of the image.

[0329] When there is no image associated with the captured image data having the feature point, as described above, registered in step S220, the verification device 4 determines that the captured image data having the registration of the feature point is not data of an image obtained by special imaging, and then proceeds to the processing of step S206.

[0330] In step S206, the verification device 4 determines that "similar content exists".

[0331] Additionally, at this point, the similarity of similar content can be determined, such as "80% similarity to the xxx photo captured by Mr. / Ms. A".

[0332] In other words, although the signature data indicates that the image being verified was captured by the registered user, it is possible that the captured image was taken by the user himself by imitating the captured image taken by Mr. / Ms. A.

[0333] In step S205, the verification device 4 displays the determination on the terminal on the distribution service 300 side.

[0334] For example, such as Figure 11 As shown in the diagram, in the image display area 91 of the terminal display screen 90 on the distribution service 300 side, captured images deemed similar are displayed, and in the judgment result display area 92, judgment results of "Similar content exists" and "80% similarity to the xxx photo captured by Mr. / Ms. A" are displayed. Afterwards, the verification device 4 ends. Figure 10 The processing shown.

[0335] In step S220, when an image associated with the captured image data having feature points as described above is registered, the verification device 4 determines that the registered captured image data having feature points was obtained through special imaging and proceeds to the processing in step S221.

[0336] In step S221, the verification device 4 determines that "similar content exists".

[0337] Additionally, at this point, the similarity of similar content can be determined, such as "It has 95% similarity to the xxx photo captured by Mr. / Ms. A, and has undergone special imaging."

[0338] Here, since the registered captured image data with feature points has an image obtained through special imaging, the captured image taken by Mr. / Ms. A is more likely to be original compared to the captured image used for verification. Therefore, the similarity of the captured image used for verification is determined to be higher than 80%.

[0339] In step S205, the verification device 4 displays the determination on the terminal on the distribution service 300 side.

[0340] Here, depending on the type of special imaging, various display modes can be considered.

[0341] For example, such as Figure 14As illustrated in the middle diagram, when the captured image data with the registered feature point and the continuous photograph taken by the continuous shooting at the continuous photograph time point thereof are stored before and after the captured image data with the registered feature point, the verification device 4 displays the image of the captured image data with the registered feature point on the display screen 90 of the terminal on the distribution service 300 side, and also displays the images of the captured image data before and after the image of the captured image data with the registered feature point on the image display area 91. At this time, the information about the imaging date and time is displayed in the area 95 of the image, so that it is seen that the images are the continuous photographs.

[0342] In addition, for example, as Figure 15 As illustrated in the middle diagram, when the captured image data with the registered feature point and the captured image data in the reduced state are stored, the verification device 4 displays the image of the captured image data in the reduced state in the image display area 91 in addition to the image of the captured image data with the registered feature point on the display screen 90 of the terminal on the distribution service 300 side. At this time, the information about the focal length of the imaging lens is displayed in the area 96 of the image, so that it is seen that the image is in the reduced state.

[0343] In addition, for example, as Figure 16 As illustrated in the middle diagram, when the image of the captured image data with the registered feature point is cropped and the captured image data before the cropping is stored, the verification device 4 can display the image of the captured image data before the cropping in the image display area 91 of the display screen 90 of the terminal on the distribution service 300 side. At this time, the area 97 of the cropped image of the captured image data is displayed surrounded by a broken line, so that it is seen that the cropped image of the captured image data.

[0344] The verification device 4 displays the display mode as described above on the terminal on the distribution service 300 side, and thereafter, ends Figure 10 the processing illustrated in the middle.

[0345] As described above, the verification processing of the captured image by the verification device 4 in the present embodiment is implemented.

[0346] Finally, the verification request processing of the terminal on the distribution service 300 side for implementing the above-described embodiments is described with reference to Figure 17 .

[0347] In step S301, the terminal on the distribution service 300 side transmits the image file containing the captured image data desired to be verified to the verification device 4, and then in step S302, the terminal on the distribution service 300 side waits until the verification result is received from the verification device 4.

[0348] When the terminal on the distribution service 300 side receives the verification result from the verification device 4, the terminal on the distribution service 300 side displays the verification result on the display section at step S303. On the terminal on the distribution service 300 side, the respective determination results are displayed in accordance with the respective determination results, and the respective images illustrated in FIGS. 1 to 3, for example. Figures 11-16

[0349] As described above, the verification request processing of the captured image by the terminal on the distribution service 300 side in the present embodiment is implemented.

[0350] <6. SUMMARY AND MODIFICATIONS>

[0351] The image processing device (analysis unit 21) installed in the imaging device 1 in the embodiment includes a setting unit 27 that selects metadata associated with the authenticity verification among metadata containing information on a plurality of items associated with the captured image data, based on a feature of the captured image data. In addition, the setting unit 27 sets flag information to the metadata associated with the authenticity verification (see Figure 9 ).

[0352] Thus, the flag information is set to the item that reflects the feature of the captured image data. As described above, when it is determined whether the captured image as the verification object is authentic (whether the captured image is not a copy of a work created by another person, or the like), the feature of the captured image is added, so that a determination with high accuracy can be made.

[0353] For example, when the feature of the captured image is a long shot, by using "distance to the subject" for the authenticity determination of the captured image, even if the captured images look the same as each other, that is, if the "distance to the subject" is unnatural, it can be presumed that the captured image is a copy of a work imaged by another person.

[0354] In addition, in the processing of setting the flag information to the metadata associated with the authenticity verification, when the captured image data transmitted from the imaging device 1 is registered, the feature points on the verification device 4 side illustrated in FIG. 4 can be extracted and recorded in the feature point DB 51. Figure 8

[0355] Here, the image processing device (analysis unit 21) in the external terminal performs the processing of setting the flag information, so that the processing load on the verification device 4 side can be reduced.

[0356] The image processing device (analysis unit 21) in the embodiment includes a hash value calculation unit 28 that calculates a hash value with respect to the captured image data and the metadata associated with the captured image data, and a merging processing unit 29 that merges the signature data generated by using the hash value and the captured image data and the metadata in which the flag information is set, so that the merged data can be recorded (see​​Figure 9 S105 to S110 in FIG. 10, etc.).

[0357] Thus, even if the captured image data is the same, a different hash value is calculated in a case where the metadata associated with the captured image data is different.

[0358] As described above, by calculating the hash value that contains not only the captured image data but also information about the state of the imaging device 1 at the time of imaging, such as "distance to the subject", "brightness of the image capturing environment (EV value)", etc., and generating the signature data, even if the captured images look the same as each other, it is possible to determine whether the composition of the images was actually decided by the user on the spot, the images were captured by the user, that is, whether the images are not copies of already captured images, based on whether the hash values match each other or not.

[0359] Thus, by reflecting the information about the state of the imaging device 1, such as "distance to the subject", "brightness of the image capturing environment (EV value)", etc., it is possible to easily determine whether the captured image that is the verification target is original (whether the captured image is not a copy of a work created by another person) or not.

[0360] Note that the hash value calculation unit 28 can calculate the hash value by using the metadata associated with the item to which the marker information is set (see S105, S106, etc. in FIG. 10). Thus, the hash value is calculated by using the metadata of the item in which the marker information is set (that is, the item that reflects the characteristics of the captured image data). Figure 9

[0361] The image processing device (analysis unit 21) in the embodiment includes a generation control unit 30 that generates the signature data by using the hash value and the private key in the public key encryption scheme (see S107, S108, etc. in FIG. 11). Figure 9

[0362] When the user who took the image of the captured image data encrypts the signature data using the private key in the public key encryption scheme, and transmits the encrypted signature data, and a third party who receives the signature data can decrypt the signature data using the public key, the receiver can verify that the user who took the image of the captured image data made the transmission. Thus, it is possible to easily prove that the generation of the signature data by encrypting the hash value was made based on the operation of the user.

[0363] In addition, by providing the generation control unit 30 to the image processing device (analysis unit 21), since the signature data can be generated without using other terminals when the imaging device 1 having the built-in image processing device (analysis unit 21) is available, the convenience of the user is improved.

[0364] ​​In the image processing apparatus (analysis unit 21) in the embodiment, the generation control unit 30 that generates signature data by using a hash value and a private key in a public key encryption scheme is provided for the external terminal 3 that can be attached and detached, and the merging processing unit 29 acquires the signature data from the external terminal 3 (see S107, S108, etc. in Figure 9

[0365] Thus, even in a case where an image is captured using a different imaging apparatus 1, by connecting the external terminal 3, signature data can be generated based on a private key in a public key encryption scheme associated with a user who captured the image. In other words, by having the external terminal 3, for a plurality of imaging apparatuses 1, signature data that proves that the user himself or herself captured the image can be generated. This is convenient, for example, in a case where the imaging apparatus 1 owned by the user himself or herself is broken, and signature data can also be generated in an imaging apparatus 1 that is a replacement.

[0366] In the image processing apparatus (analysis unit 21) in the embodiment, distance information to an imaging subject is contained in a plurality of items associated with captured image data (see Figure 5

[0367] In particular, this is useful in a case where a feature of a captured image is a captured close-up or a captured long shot, in a case where a specific object such as a person or an animal is the imaging target, and in other cases.

[0368] For example, when the image is a circular composition that is substantially at infinity, and the imaging distance is also at infinity, the probability that the long shot is actually imaged is high. In contrast, when the image is a circular composition that is substantially at infinity, but the imaging distance is short, the probability that the image is a copy of a captured image is high.

[0369] Thus, by recording the distance to the imaging subject in a state where authenticity is ensured, it is possible to easily determine whether the image is an image captured by a person who actually captured the image (for example, whether the image is not a copy of a captured image).

[0370] In the image processing apparatus (analysis unit 21) in the embodiment, information related to exposure control is contained in a plurality of items associated with captured image data (see Figure 5

[0371] In particular, this is useful in a case where a feature of a captured image is a captured night scene or a captured day scene, or in other cases.

[0372] ​​​For example, when the image is a composition of a night scene, and the EV value is about 0, the probability that the dark night scene is actually captured is high. In contrast, when the EV value is about 10 and high, the probability that the image is a copy of an image captured in a bright room rather than in a dark environment is high.

[0373] Thus, by recording the EV value of the imaging subject in a state in which authenticity is ensured, it is possible to easily determine whether the image is an image captured by the person who actually captured the image (e.g., whether the image is not a copy of an image that has been captured).

[0374] In the image processing apparatus (analysis unit 21) in the embodiment, information about the imaging date and time at which the image data was captured is included in a plurality of items associated with the captured image data (see Figure 5 and the like).

[0375] In particular, this is useful in cases in which the feature of the captured image reflects the imaging time of a night scene, a day scene, or the like, in cases in which the feature of the captured image reflects the imaging season such as a snow scene, and in other cases.

[0376] For example, when the image is a composition of a snow scene, and the imaging date and time is in December, which is in winter, the probability that the image was actually captured in winter is high. In contrast, when the image is a composition of a snow scene, but the imaging date and time is in July, which is in summer, since it is difficult to imagine that it will snow, the probability that the image is a copy of an image of a snow scene that has been captured is high.

[0377] Thus, by recording the imaging date and time of the imaging subject in a state in which authenticity is ensured, it is possible to easily determine whether the image is an image captured by the person who actually captured the image (e.g., whether the image is not a copy of an image that has been captured).

[0378] The image processing apparatus (analysis unit 21) in the embodiment includes the communication unit 18 that outputs file data in which the signature data, the captured image data, and the metadata in which the flag information is set are merged (see Figure 9 in S111 and the like). Thus, the file data in which the signature data, the captured image data, and the metadata in which the flag information is set are merged can be transmitted to an apparatus that verifies the authenticity of the captured image data.

[0379] Further, the image processing apparatus (verification apparatus 4) in the embodiment includes an information management unit 43 that acquires information on one or more items for which mark information is set from a merged file that includes captured image data and metadata that includes information on a plurality of items associated with the captured image data and for which mark information corresponding to one or more items of features of the captured image data is set, and a reality determination unit 47 that determines reality of the features of the captured image data and the one or more items for which the mark information is set (see S214, S215, etc. in Figure 10

[0380] In other words, when the degree to which it reflects the features of the captured image data is high and one or more items for which mark information is set indicate unnatural values, it is determined that the captured image does not have reality. For example, when the captured image data indicates that the features of the image are long shots, but the distance to the imaged object is too close, it is determined that the image does not have reality. Thus, it can be inferred that the image was captured in an environment different from the original image capturing environment, such as in the case where the captured image was not taken by the user himself or herself, but a copy of an already captured image, and other cases, and thus it can be determined that the captured image was not captured by the imager himself or herself.

[0381] In the image processing apparatus (verification apparatus 4) in the embodiment, the information management unit 43 outputs a result corresponding to the reality determination by the reality determination unit 47 (see S205, etc. in Figure 10

[0382] Thus, for example, the reality determination result is transmitted to the terminal to which the merged file is transmitted in order to determine reality. Thus, the determination result is displayed on the terminal on the distribution service 300 side, and thus the person who requested verification can make a judgment while viewing the captured image. In particular, the information on one or more marked items used for reality determination is displayed together with the captured image of the verification result, and thus the person who requested verification can learn the reason for the verification result.

[0383] The image processing apparatus (verification apparatus 4) in the embodiment includes a content registration unit 42 that registers a merged file including captured image data, metadata associated with the captured image data, and signature data generated by using a hash value calculated for the captured image data and the metadata associated with the captured image data, and a signature verification unit 44 that calculates a hash value by using the captured image data and the metadata associated with the captured image data in the merged file acquired by the information management unit 43, and determines reality of the calculated hash value and the hash value used in the signature data of the registered merged file (see S208, etc. in Figure 10 ​​S209, S210, etc. in FIG. 20).

[0384] Thus, since even if the captured image data in the registered merged file and the captured image data as the verification target are identical to each other, different hash values are generated in the case where the metadata associated with the captured image data are different, and thus it is determined that the captured image does not have authenticity. Thus, it is possible to easily determine that any one of the captured image data as the verification target and the metadata associated with the captured image data has been tampered with in some form.

[0385] In the image processing apparatus (verification apparatus 4) in the embodiment, the signature data is generated by using the calculated hash value and the private key in the public key encryption scheme, and the signature verification unit 44 acquires the calculated hash value from the signature data by using the public key associated with the private key (see Figure 10 S210, etc. in FIG. 20).

[0386] When the user who photographed the image of the captured image data encrypts the signature data using the private key in the public key encryption scheme and transmits the signature data, and a third party who receives the signature data is able to decrypt the signature data using the public key associated (paired) with the private key, it is possible to verify that the hash value acquired from the signature data is the hash value in the captured image data of the user who photographed the image. Thus, in a state where the authenticity of the hash value in the signature data is ensured, it is possible to perform the comparison with the hash value of the captured image as the verification target.

[0387] In the image processing apparatus (verification apparatus 4) in the embodiment, the authenticity determination unit 47 determines the authenticity in accordance with the presence or absence of other captured image data captured before or after the imaging time of the captured image data (see Figure 10 S220, etc. in FIG. 20).

[0388] Having the captured image photographed before or after the imaging time of the captured image data as a basis for increasing the reliability that the image is captured by the user himself or herself. In other words, when the authenticity of one or more images distributed is in question, by distributing a part of the continuously captured images, and storing the other part of the continuously captured images without disclosing the time when the other part is captured, it is expected that the one or more images at the time when the other part is stored become evidence that the one or more images distributed are captured by the imager himself or herself.

[0389] By comparing a plurality of continuously captured images as described above, since it is possible to check the movement of the moving object, it is possible to verify the authenticity (originality) of the image, that is, the fact that the imager was actually on the scene and photographed the image, with a higher probability.

[0390] In the image processing apparatus (verification apparatus 4) in the embodiment, the authenticity determination unit 47 determines authenticity in accordance with the presence or absence of other captured image data whose angle of view is wider than that of the captured image data from the angle of view thereof (see S220 and the like in Figure 10

[0391] The other captured image data whose angle of view is wider than that of the captured image data becomes a basis for increasing the reliability of the image as an image captured by the user himself or herself. In other words, by storing the reduced image secretly, when the authenticity of the distributed image (an image whose angle of view is narrower) is questioned, it is expected that the image becomes evidence that the person himself or herself is the imager and that the image is an image captured by the person himself or herself.

[0392] When determining the authenticity of the captured image as described above, by combining the image whose angle of view is wider (has much information) that only the imager can record, it is possible to verify that the imager undoubtedly captured the image on the scene, that is, the authenticity (originality) of the image.

[0393] In addition, the imaging apparatus 1 in the embodiment includes: an imaging unit 13; a feature determination unit 25 that determines a feature of captured image data obtained by the imaging unit 13, and selects one or a plurality of items corresponding to the determined feature in metadata that contains a plurality of items associated with the captured image data; a setting unit 27 that sets flag information of the one or a plurality of items selected by the feature determination unit 25 in the metadata; and a merge processing unit 29 that merges the captured image data and the metadata in which the flag information is set, so that the merged captured image data and metadata can be recorded (see Figure 9 and the like).

[0394] Thus, the functions of the above-described image processing apparatus (analysis unit 21) are similarly implemented as the imaging apparatus 1, and effects similar to those exhibited by the configuration of the image processing apparatus (analysis unit 21) are achieved.

[0395] In addition, the functions of the above-described image processing apparatus (analysis unit 21) are implemented by the imaging apparatus 1, so that the captured image data can be directly sent from the imaging apparatus 1 to the verification apparatus 4 to be registered in the verification apparatus 4. In other words, the image processing apparatus (analysis unit 21) built in the imaging apparatus 1 performs the process of setting the flag information, so that it is possible to reduce the processing load on the verification apparatus 4 side.

[0396] The program in the embodiment is a program that causes, for example, a CPU, a DSP, or the like or a device including these to execute the processes shown in Figure 9

[0397] ​​In other words, the program in the embodiment is a program that causes the image processing apparatus to execute a feature determination process of determining a feature of captured image data, a selection process of selecting one or more items corresponding to the determined feature in metadata that contains information of a plurality of items associated with the captured image data, a setting process of setting flag information of the one or more items selected in the feature determination process in the metadata, and a merge process of merging the captured image data and the metadata in which the flag information is set, thereby enabling the merged captured image data and metadata to be recorded. Through the above program, the above image processing apparatus can be implemented by, for example, the apparatus such as the imaging apparatus 1 such as the digital camera 1A, the digital video camera IB, or the like, or the mobile terminal 2 such as a smartphone or the like having an imaging function.

[0398] In addition, the program in the embodiment is a program that causes, for example, a CPU, a DSP, or the like, or a device including the same to execute the processes shown in FIG. 8. Figure 10

[0399] In other words, the program in the embodiment is a program that causes the image processing apparatus to execute an acquisition process of acquiring information of one or more items for which flag information is set from a merged file containing captured image data and metadata that contains information of a plurality of items associated with the captured image data, and in which flag information is set for one or more items corresponding to a feature of the captured image data, and a reality determination process of determining a feature of the captured image data and reality of the one or more items for which flag information is set. Through the above program, the above image processing apparatus can be implemented by, for example, an apparatus such as a computer apparatus serving as the verification apparatus 4.

[0400] Each of the above programs can be recorded in advance in, for example, a ROM or the like in a microcomputer having a CPU, or an HDD as a recording medium built into an apparatus such as a computer apparatus.

[0401] Alternatively, each of the programs can be temporarily or permanently stored (recorded) in a detachable recording medium such as a flexible disk, a compact disc read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disc (DVD), a Blu-ray (registered trademark) disk, a magnetic disk, a semiconductor memory, and a memory card. The above detachable recording medium can be provided in the form of a so-called package software.

[0402] In addition, each of the above programs can be installed from the detachable recording medium into a personal computer or the like, and in addition thereto, each of the above programs can be downloaded from a download site via a network such as a local area network (LAN) and the Internet.

[0403] ​In addition, the image processing apparatus according to the embodiments described above is applicable to a wide range. For example, each of the programs is downloaded to a personal computer, a mobile information processing apparatus, a mobile phone, a game apparatus, a video apparatus, a personal digital assistant (PDA), or the like, so that the personal computer or the like can function as the image processing apparatus of the present disclosure.

[0404] Note that the effects described in the present specification are merely illustrative or not restrictive and there can be other effects.

[0405] In addition, the description of the embodiments described in the present specification is merely an example and the present technology is not limited to the above-described embodiments. Thus, various modifications can be made to the above-described embodiments based on design, and the like, without departing from the scope of the technical idea of the present technology.

[0406] The present technology can also adopt the following configuration.

[0407] (1) An image processing apparatus comprising

[0408] a setting unit that selects metadata related to authenticity verification among metadata including information of a plurality of items associated with captured image data, based on a feature of the captured image data.

[0409] (2) The image processing apparatus according to the above (1), wherein

[0410] the setting unit sets flag information in the selected metadata related to authenticity verification.

[0411] (3) The image processing apparatus according to the above (2), comprising:

[0412] a hash value calculation unit that calculates a hash value with respect to the captured image data and the metadata associated with the captured image data; and

[0413] a merge processing unit that merges the signature data generated by using the hash value, the captured image data, and the metadata in which the flag information is set, so that the merged signature data, the captured image data, and the metadata can be recorded.

[0414] (4) The image processing apparatus according to the above (3), comprising

[0415] a generation control unit that generates the signature data by using the hash value and a private key in a public key encryption scheme.

[0416] (5) The image processing apparatus according to the above (3), wherein

[0417] The generation control unit that generates signature data by using a hash value and a private key in a public key encryption scheme is provided on an attachable and detachable external terminal, and

[0418] The merging processing unit acquires signature data from the external terminal.

[0419] (6) The image processing apparatus according to any one of (1) to (5) above, wherein

[0420] Among the plurality of items associated with the captured image data, distance information to the imaged object is included.

[0421] (7) The image processing apparatus according to any one of (1) to (6) above, wherein

[0422] Among the plurality of items associated with the captured image data, information related to exposure control is included.

[0423] (8) The image processing apparatus according to any one of (1) to (7) above, wherein

[0424] Among the plurality of items associated with the captured image data, information related to the imaging date and time of the captured image data is included.

[0425] (9) The image processing apparatus according to any one of (3) to (5) above, comprising

[0426] a communication unit that outputs file data in which the signature data, the captured image data, and the metadata in which the flag information is set are merged.

[0427] (10) An image processing apparatus comprising:

[0428] an information management unit that acquires information of a selected item from a merged file including captured image data and metadata including information of a plurality of items associated with the captured image data, the selected item being selected as an item related to authenticity verification from among the information of the plurality of items, and

[0429] an authenticity determination unit that determines authenticity of the selected item and a feature of the captured image data.

[0430] (11) The image processing apparatus according to (10) above, wherein

[0431] the information management unit acquires information of an item to which flag information is set from a merged file including captured image data and metadata including information of a plurality of items associated with the captured image data, and wherein the flag information is set to an item corresponding to a feature of the captured image data, and

[0432] The authenticity determination unit determines authenticity of a feature of the captured image data and the item to which the tag information is set.

[0433] (12) The image processing apparatus according to any one of (10) or (11), wherein

[0434] The information management unit outputs a result corresponding to the authenticity determination by the authenticity determination unit.

[0435] (13) The image processing apparatus according to any one of (10) to (12), comprising:

[0436] a content registration unit that registers a merge file including the captured image data, the metadata associated with the captured image data, and the signature data generated by using a hash value calculated for the captured image data and the metadata associated with the captured image data; and

[0437] a signature verification unit that calculates a hash value by using the captured image data and the metadata associated with the captured image data in the merge file acquired by the information management unit, and determines authenticity of the calculated hash value and the hash value used in the signature data of the registered merge file.

[0438] (14) The image processing apparatus according to (13), wherein

[0439] The signature data is generated by using the calculated hash value and a private key in a public key encryption scheme, and the signature verification unit acquires the calculated hash value from the signature data by using a public key associated with the private key.

[0440] (15) The image processing apparatus according to any one of (10) to (14), wherein

[0441] The authenticity determination unit determines authenticity in accordance with the presence or absence of other captured image data that captures an image before or after an imaging time of the captured image data.

[0442] (16) The image processing apparatus according to any one of (10) to (15), wherein

[0443] The authenticity determination unit determines authenticity in accordance with the presence or absence of other captured image data that has a wider angle of view than an angle of view of the captured image data.

[0444] (17) An image forming apparatus comprising:

[0445] an imaging unit; and

[0446] a setting unit that selects metadata related to the authenticity verification among metadata containing information of a plurality of items associated with the captured image data, based on a feature of the captured image data obtained by the imaging unit.

[0447] (18) An image processing method in which an image processing apparatus performs selection of metadata related to authenticity verification among metadata containing information of a plurality of items associated with captured image data, based on a feature of the captured image data.

[0448] (19) A program that causes an image processing apparatus to perform processing in which selection of metadata related to authenticity verification among metadata containing information of a plurality of items associated with captured image data is performed, based on a feature of the captured image data.

[0449] (20) An image processing method in which an image processing apparatus performs

[0450] acquiring information of the selected item from a merged file containing the captured image data and metadata containing items selected from information of a plurality of items associated with the captured image data as items related to authenticity verification, and

[0451] determining a feature of the captured image data and authenticity of the selected item.

[0452] (21) A program that causes an image processing apparatus to perform

[0453] acquiring processing in which information of the selected item is acquired from a merged file containing the captured image data and metadata containing items selected from information of a plurality of items associated with the captured image data as items related to authenticity verification, and

[0454] authenticity determining processing in which a feature of the captured image data and authenticity of the selected item are determined.

[0455] List of Reference Signs

[0456] 1 imaging apparatus

[0457] 3 external terminal

[0458] 4 verification apparatus

[0459] 13 imaging unit

[0460] 18 communication unit

[0461] 21 analysis unit

[0462] 25 feature determination unit

[0463] 27 setting unit

[0464] 28 hash value calculation unit

[0465] 29 merging processing unit

[0466] 30 generation control unit

[0467] 42 content registration unit

[0468] 43 information management unit

[0469] 44 signature verification unit

[0470] 47 authenticity determination unit

Claims

1. An image processing apparatus, comprising: The setting unit selects metadata related to authenticity verification from metadata containing information about multiple items associated with the captured image data, based on the characteristics of the captured image data. and The feature determination unit selects one or more items from the metadata that correspond to the features of the captured image data determined based on image analysis processing. These features are those that suggest the content of the captured image data. The setting unit sets tagging information in the selected metadata related to authenticity verification; The authenticity of a captured image is determined by matching the features of the captured image data with the items marked on them; and The image processing device further includes: A hash value calculation unit calculates a hash value for the captured image data and the metadata associated with the captured image data; and The merging processing unit merges signature data generated using hash values, captured image data, and metadata in which the tag information is set, thereby enabling the merged signature data, captured image data, and metadata to be recorded.

2. The image processing apparatus according to claim 1, comprising: A generation control unit generates signature data using a hash value and a private key in a public-key encryption method.

3. The image processing apparatus according to claim 2, wherein, The generation control unit, which generates signature data using a hash value and a private key in a public-key encryption method, is mounted on an attachable and detachable external terminal. The merging processing unit obtains signature data from the external terminal.

4. The image processing apparatus according to claim 1, wherein, Among the multiple items associated with the captured image data is distance information to the imaged object.

5. The image processing apparatus according to claim 1, wherein, Among the multiple items associated with captured image data is information related to exposure control.

6. The image processing apparatus according to claim 1, wherein, The multiple items associated with the captured image data contain information related to the imaging date and time of the captured image data.

7. The image processing apparatus according to claim 1, comprising: A communication unit that outputs file data containing signature data, captured image data, and metadata with set tag information.

8. An image processing apparatus, comprising: An information management unit obtains information about selected items from a merged file containing captured image data and metadata containing metadata of items selected from multiple items associated with the captured image data as items related to authenticity verification. and An authenticity determination unit determines the authenticity of the features of the captured image data and the selected item, wherein... The information management unit retrieves information from a merged file containing captured image data and metadata, specifying items for which tagged information is assigned. The metadata includes information on multiple items associated with the captured image data, and specifies tagged information for items corresponding to characteristics of the captured image data. The authenticity determination unit determines the authenticity of the features of the captured image data and the items for which it sets label information. The items for which the tagging information is set are one or more items selected in the metadata that correspond to the characteristics of the captured image data determined based on image analysis processing. The characteristics of the captured image data are features that suggest the content of the captured image data. The authenticity of a captured image is determined by matching the features of the captured image data with the items for which they are labeled.

9. The image processing apparatus according to claim 8, wherein, The information management unit outputs the result corresponding to the authenticity determination performed by the authenticity determination unit.

10. The image processing apparatus according to claim 8, comprising: A content registration unit registers a merged file, the merged file including captured image data, metadata associated with the captured image data, and signature data generated by using a hash value calculated for the captured image data and the metadata associated with the captured image data; and The signature verification unit calculates a hash value using the captured image data in the merged file obtained by the information management unit and the metadata associated with the captured image data, and determines the authenticity of the calculated hash value and the hash value used in the signature data of the registered merged file.

11. The image processing apparatus according to claim 10, wherein, Signature data is generated by using a calculated hash value and a private key in a public-key encryption method, and the signature verification unit obtains the calculated hash value from the signature data by using the public key associated with the private key.

12. The image processing apparatus according to claim 8, wherein, The authenticity determination unit determines authenticity based on the presence or absence of other captured image data that captured the image before or after the imaging time of the captured image data.

13. The image processing apparatus according to claim 8, wherein, The authenticity determination unit determines authenticity based on the presence or absence of other captured image data whose viewing angle is wider than that of the captured image data.

14. An imaging device, comprising: Imaging unit; and Image processing apparatus according to any one of claims 1 to 7.

15. An image processing method, wherein, The image processing method is performed by an image processing apparatus according to any one of claims 1 to 7.

16. A program product that causes an image processing apparatus to perform the image processing method according to claim 15.

17. An image processing method, wherein, The image processing method is performed by an image processing apparatus according to any one of claims 8 to 13.

18. A program product that causes an image processing apparatus to perform the image processing method according to claim 17.

Images