Image verification system, image verification method, and program

Abstract

To provide an image verification system that enables a user to easily recognize which object area in an image has changed due to editing.SOLUTION: An image verification system 100 acquires post-edited image data 707 with verification data to which just-generated verification data, first-edited verification data, and secondly-edited verification data are appended. The image verification system 100 compares object area information of an object area detected from the post-edited image data 707 with the verification data with object area information of the just-generated verification data and object area information of the first-edited verification data, which are appended to the post-edited image data 707 with the verification data. Based on a comparison result, the image verification system 100 displays a predetermined notification on a verification screen 1200 to allow the user to recognize that a change has occurred due to editing by the image editing device 102.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image verification system, an image verification method, and a program. [Background technology]

[0002] In recent years, with the development of image generation and processing technologies, the authenticity of the origin of an image and the editing process has become an issue. A related technology is proposed in Patent Document 1. In Patent Document 1, a digital signature is added to an image when the image is generated, and an editing history and its digital signature are added to the image every time the image is edited, making it possible to detect tampering with the image and the editing history. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2008-124668 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technique of Patent Document 1 described above, the user cannot easily recognize which object area in the image has changed as a result of editing.

[0005] An object of the present invention is to provide an image verification system, an image verification method, and a program that enable a user to easily recognize which object area in an image has changed due to editing. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the image verification system of the present invention is characterized in that it comprises: a means for adding first object area information regarding an object area detected from the generated image data to the generated image data; a means for editing the image data to which at least the first object area information has been added; a means for adding second object area information regarding the object area detected from the image data after editing by the editing means to the edited image data; a means for comparing third object area information regarding the object area detected from the edited image data with the first object area information and the second object area information added to the edited image data, respectively; and a means for displaying a notification to notify the user that a change has occurred due to editing by the editing means, based on the result of the comparison by the comparing means. Effect of the Invention

[0007] According to the present invention, it is possible for a user to easily recognize which object area in an image has changed as a result of editing. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating a schematic configuration of an image verification system according to an embodiment of the present invention. [Diagram 2] 2 is a block diagram illustrating a schematic hardware configuration of the image generating device in FIG. 1. [Diagram 3] 2 is a block diagram illustrating a schematic hardware configuration of the image editing device in FIG. 1. [Figure 4] 2 is a block diagram illustrating a schematic hardware configuration of the image verification device in FIG. 1. [Diagram 5] 2 is a block diagram showing a schematic configuration of a software module 500 of the image generating device of FIG. 1. [Figure 6] 4 is a flowchart showing the procedure of image data generation processing executed by the image generating device of FIG. [Figure 7] 2 is a diagram showing an example of the configuration of image data generated in the image verification system 100 of FIG. 1. [Figure 8] 2 is a block diagram showing a schematic configuration of image editing software of the image editing device of FIG. 1. [Figure 9] 4 is a flowchart showing the procedure of image data editing processing executed by the image editing device of FIG. 1. [Figure 10] 2 is a block diagram showing an outline of a configuration of image verification software of the image verification device of FIG. 1. FIG. [Figure 11] 4 is a flowchart showing the procedure of image data verification processing executed by the image verification device of FIG. 1. [Figure 12] FIG. 5 is a diagram showing an example of a verification screen displayed on the display in FIG. 4. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals are used to designate the same or similar components, and duplicated descriptions will be omitted.

[0010] 1 is a block diagram showing a schematic configuration of an image verification system 100 according to the present embodiment. In FIG. 1, the image verification system 100 includes an image generating device 101, an image editing device 102, and an image verifying device 103.

[0011] The image generating device 101 is a device that generates image data serving as a starting point and its verification data. The configuration of the verification data will be described later. The image generating device 101 is, for example, a digital camera that captures an image of a subject and generates image data.

[0012] The image editing device 102 is a device that performs arbitrary editing on image data, adds verification data to the edited image data, and outputs the edited image data. The image editing device 102 is, for example, a computer in which image editing software 800 shown in Fig. 8, which will be described later, for editing image data, is installed.

[0013] The image verification device 103 is a device that reads the acquired image data, detects and displays the editing status of the image data based on the verification data added to the image data. The image verification device 103 is, for example, a computer in which image verification software 1000 shown in FIG. 10, which will be described later, for verifying the acquired image data, is installed.

[0014] In the present embodiment, a configuration will be described in which the image editing device 102 that edits image data and the image verification device 103 that verifies the image data are separate devices, but the present invention is not limited to this configuration. For example, a single device in which both the image editing software 800 and the image verification software 1000 are installed may edit image data and verify the edited image data. Also, a configuration in which the process of editing image data and the process of verifying image data are performed on a server or a cloud may be used.

[0015] In the image verification system 100, first, an image generating device 101 generates image data serving as a starting point and its verification data, and outputs the image data to which the verification data has been added to an image editing device .

[0016] The image editing device 102 acquires this image data from the image generating device 101, performs any editing on this image data to generate edited image data, and generates verification data for the image data. The image editing device 102 outputs the edited image data to which the verification data has been added to the image verification device 103. Note that in this embodiment, as an example, it is assumed that the image editing device 102 has edited the image data acquired from the image generating device 101 twice. In other words, the edited image data is assigned with the verification data generated by the image generating device 101 when it was generated, the verification data generated by the image editing device 102 during the first editing, and the verification data generated by the image editing device 102 during the second editing.

[0017] The image verification device 103 obtains the edited image data from the image editing device 102, and verifies the image data based on the verification data added to the edited image data. Note that, in this embodiment, as an example, a case will be described in which the image editing device 102 edits the image data obtained from the image generating device 101 twice, but the basic processing contents are the same even if the number of edits is one, or three or more.

[0018] Fig. 2 is a block diagram showing a schematic hardware configuration of the image generating device 101 in Fig. 1. In Fig. 2, the image generating device 101 includes a CPU 201, a RAM 202, a ROM 203, a storage 204, an operation unit 205, an imaging unit 206, and an I / F 207. These are connected to each other via a system bus 209. Note that I / F is an abbreviation for interface.

[0019] The CPU 201 performs overall control of the above-mentioned devices connected via a system bus 209. The RAM 202 is a memory that can be accessed at high speed, and is used as a temporary storage area such as the main memory and work area of ​​the CPU 201. The ROM 203 stores programs executed by the CPU 201. For example, the ROM 203 stores programs of a software module 500 shown in FIG. 5, which will be described later. The storage 204 stores image data and the like. The operation unit 205 is a user interface that accepts shooting instructions and the like from a user. The imaging unit 206 includes an optical system and a circuit, and converts an image captured according to the shooting instructions received by the operation unit 205 into image data in a format that can be processed by the image generating device 101. The I / F 207 is connected to a network 208, and is used when transmitting image data to the image editing device 102, etc.

[0020] Fig. 3 is a block diagram showing a schematic hardware configuration of the image editing device 102 in Fig. 1. In Fig. 3, the image editing device 102 includes a CPU 301, a RAM 302, a ROM 303, a storage 304, an operation unit 305, a display 306, and an I / F 307. These are connected to each other via a system bus 308.

[0021] The CPU 301 comprehensively controls the above-mentioned devices connected via a system bus 308. The RAM 302 is a memory that can be accessed at high speed, and is used as a temporary storage area such as the main memory and work area of ​​the CPU 301. The ROM 303 stores programs executed by the CPU 301, etc. The storage 304 stores programs of an image editing software 800 in FIG. 8 described later, etc. The operation unit 305 is a device that accepts various operation instructions from a user, and is a keyboard, a mouse, etc. The display 306 is a display device that displays the state and processing status of the image editing device 102. The I / F 307 is connected to the network 208, and is used when receiving image data from the image editing device 102, when transmitting edited image data to the image verification device 103, etc.

[0022] Fig. 4 is a block diagram showing a schematic hardware configuration of the image verification device 103 in Fig. 1. In Fig. 4, the image verification device 103 includes a CPU 401, a RAM 402, a ROM 403, a storage 404, an operation unit 405, a display 406, and an I / F 407. These are connected to each other via a system bus 408. The storage 404 stores a program of the image verification software 1000 in Fig. 10, which will be described later. The hardware configuration of the image verification device 103 is basically the same as that of the image editing device 102. Therefore, the description of the hardware configuration of the image verification device 103 is similar to that of the image editing device 102 described above, and will not be repeated here.

[0023] Next, the software module 500 of the image generating device 101 will be described.

[0024] Fig. 5 is a block diagram showing a schematic configuration of a software module 500 of the image generating device 101 of Fig. 1. In Fig. 5, the software module 500 is composed of an image input unit 501, an object detection unit 502, a verification data generating unit 503, and an image output unit 504. The processing of these modules is realized by the CPU 201 of the image generating device 101 executing the program of the software module 500 read from the ROM 203 to the RAM 202.

[0025] The image input unit 501 inputs image data generated by the imaging unit 206 into the software module 500. The object detection unit 502 detects an object region from the image data input by the image input unit 501, and outputs region data and label information of the detected object region. The region data is data indicating the detected object region, for example, data indicating the position coordinates of this object region. The label information is classification information of the detected object region, for example, information indicating the type and attributes of this object region. The verification data generation unit 503 generates verification data using the region data and label information output by the object detection unit 502, and assigns this verification data to the image data. The image output unit 504 transmits the image data to which the verification data has been assigned to an external device such as the image editing device 102.

[0026] Fig. 6 is a flowchart showing the procedure of image data generation processing executed by image generating device 101 in Fig. 1. The image data generation processing is realized by CPU 201 reading the program of software module 500 stored in ROM 203 into RAM 202 and executing the program read into RAM 202. The image data generation processing is executed when image generating device 101 receives a shooting instruction from a user in a state where it is set to a predetermined mode for outputting image data to which verification data is added.

[0027] In FIG. 6, first, the CPU 201 inputs image data generated by shooting with the imaging unit 206 to the software module 500 (S (step) 601). Next, the CPU 201 detects an object region from this image data (S602). For example, a technique such as R-CNN or YOLO is used for detecting the object region. The CPU 201 outputs region data and label information of the detected object region. If a number of object regions exceeding a preset threshold are detected from the image data, in S602, region data and label information of an aggregate region in which the detected object regions are combined may be output. For example, if a number of person regions exceeding a threshold are detected from the image data, region data indicating the position of an aggregate region in which the detected person regions are combined and label information indicating an attribute of this aggregate region, such as "crowd", are output.

[0028] Next, the CPU 201 generates verification data (S603). Next, the CPU 201 adds this verification data to the image data (S604). By the process of S604, image data 701 with verification data shown in FIG. 7(a) is generated.

[0029] The verification data is composed of object information, algorithm information, and an electronic signature, as shown in 702 of FIG. 7(a). The electronic signature is generated by encrypting a hash value obtained by executing a hash function on data including image data, object information, and algorithm information, using a private key prepared in advance. A public key (certificate) that pairs with this private key is also stored in the electronic signature. As shown in 703, the object information is composed of a plurality of object region information corresponding to a plurality of object regions detected by the object detection unit 502. The object region information includes region data, label information, and a hash value. The region data and label information are data output by the object detection unit 502. The hash value is calculated by executing a hash function on a part of the image data, specifically, on the object region indicated by the region data. The algorithm information includes information indicating the algorithm used by the object detection unit 502 when detecting the object region, and information indicating the above-mentioned hash function. 7(a) and may be other configurations, for example, configurations conforming to a predetermined technical standard (for example, C2PA (Correspondence for Content Provenance and Authenticity)). In addition, when the area data of the aggregate area and the label information of the aggregate area are output in S602, the verification data includes a hash value obtained by applying a hash function to the aggregate area indicated by the area data in the image data, the area data, and object area information including the label information.

[0030] Next, the CPU 201 outputs (transmits) the generated image data 701 with verification data to an external device such as the image editing device 102 (S605), and ends this process.

[0031] Next, the image editing software 800 of the image editing device 102 will be described.

[0032] Fig. 8 is a block diagram showing a schematic configuration of image editing software 800 of image editing device 102 in Fig. 1. In Fig. 8, image editing software 800 is composed of an image input unit 801, a verification data acquisition unit 802, an image editing unit 803, an object detection unit 804, a verification data generation unit 805, and an image output unit 806. The processing of these modules is realized by CPU 301 of image editing device 102 reading the program of image editing software 800 stored in storage 304 or the like into RAM 302 and executing the program read into RAM 302.

[0033] An image input unit 801 inputs image data to be edited to the image editing software 800. The image data to be edited is image data with verification data 701 acquired from the image generating device 101, or edited image data obtained by editing the image data with verification data 701 acquired by the image editing device 102 from the image generating device 101 or the like one or more times. This edited image data is provided with the verification data generated by the image generating device 101 or the verification data generated by the verification data generating unit 805 during the previous editing.

[0034] The verification data acquisition unit 802 judges whether the image data has been tampered with by using the input image data and the digital signature of the verification data added to the image data. The verification data acquisition unit 802 judges whether the image data has been tampered with by a method conforming to a predetermined technical standard (e.g., C2PA). The verification data acquisition unit 802 also acquires algorithm information indicating the algorithm used by the image generating device 101 to detect the object region from the verification data.

[0035] The image editing unit 803 performs image editing on this image data in accordance with an instruction input by the user to the operation unit 305. In image editing, for example, a process of erasing a subject specified by the user from this image data, a process of adding an object specified by the user to this image data, etc. are performed. Furthermore, the image editing unit 803 generates edited image data in accordance with a save instruction from the user, and outputs the generated edited image data to the object detection unit 804.

[0036] The object detection unit 804 uses the algorithm information acquired by the verification data acquisition unit 802 to detect an object area from the edited image data in a manner similar to that of the object detection unit 502 of the image generating device 101, and outputs area data and label information for the detected object area.

[0037] The verification data generation unit 805 generates verification data using the region data, label information, etc. output by the object detection unit 804, and adds this verification data to the edited image data. The image output unit 806 transmits the edited image data to which the verification data has been added by the verification data generation unit 805 to an external device such as the image verification device 103.

[0038] FIG. 9 is a flowchart showing the procedure of the image data editing process executed by the image editing device 102 of FIG. 1. The image data editing process is realized by the CPU 301 reading the program of the image editing software 800 stored in the storage 304 into the RAM 302 and executing the program read into the RAM 302. The image data editing process of FIG. 9 is executed, for example, when the user specifies image data to be edited. In this embodiment, the user can specify the image data with verification data 701 transmitted from the image generating device 101 as the image data to be edited. The user can also specify edited image data obtained by editing the image data with verification data 701 acquired by the image editing device 102 from the image generating device 101 or the like one or more times as the image data to be edited. In the following, as an example, it is assumed that the image data with verification data 701 transmitted from the image generating device 101 is specified as the image data to be edited.

[0039] 9, the CPU 301 inputs the image data with verification data 701 designated as an object to be edited to the image editing software 800 (S901). Next, the CPU 301 uses the verification data 702 acquired from the image data with verification data 701 to determine whether the image data with verification data 701 has been tampered with (S902).

[0040] If it is determined in S902 that the verification data-added image data 701 has been tampered with, the CPU 301 displays an error notification indicating the details of the tampering on the display 306 (S903), and ends this process.

[0041] If it is determined in S902 that the verification data-added image data 701 has not been tampered with, the CPU 301 acquires algorithm information from the verification data-added image data 701 (S904).

[0042] Next, CPU 301 performs image editing on verification data-added image data 701 in accordance with an instruction input by the user to operation unit 305 (S905). Next, CPU 301 generates edited image data in accordance with the save instruction received from the user (S906). Next, CPU 301 detects an object region from the edited image data generated in S906 using the algorithm information acquired in S904 (S907). As a result, region data and label information of the detected object region are output. Note that, if a number of object regions exceeding the threshold value are detected from the edited image data generated in S906, in S907, region data and label information of an aggregate region formed by combining the detected object regions may be output, similarly to S602 described above.

[0043] Next, the CPU 301 generates verification data based on the region data of the detected object region, the label information, etc. (S908). Next, the CPU 301 adds this verification data to the edited image data generated in S906 (S909). By the process of S909, edited image data 704 with verification data shown in FIG. 7(b) is generated.

[0044] The edited image data with verification data 704 is composed of image data and verification data. The image data is edited image data generated in S906. The verification data includes the verification data that was added when acquired from the image generating device 101 (in FIG. 7(b) this is referred to as "verification data at the time of generation") and the verification data 705 generated in S908 (in FIG. 7(b) this is referred to as "verification data for the first edit").

[0045] The verification data 705 of the first edit is composed of object information and an electronic signature. As shown in 706, the object information is composed of a plurality of object region information corresponding to a plurality of object regions detected by the object detection unit 804. The object region information has the same configuration as that described in the image data generation process described above. The electronic signature is generated by encrypting a hash value obtained by executing a hash function on the edited image data generated in S906 and data including the object region information of the object region detected from the edited image data, using a private key prepared in advance. A public key (certificate) that pairs with this private key is also stored in the electronic signature. In addition, when the region data of the aggregate region and the label information of the aggregate region are output in S907, the verification data includes the hash value obtained by executing a hash function on the aggregate region indicated by the region data of the edited image data, the region data, and the object region information including the label information.

[0046] Next, the CPU 301 outputs (transmits) the generated edited image data 704 with verification data to an external device such as the image verification device 103 (S910), and ends this process.

[0047] In addition, when the user designates edited image data as the image data to be edited, the above-mentioned image data editing process is performed in the same manner. For example, when the user designates edited image data 704 with verification data as the image data to be edited, the above-mentioned process of S909 generates edited image data 707 with verification data shown in FIG. 7(c). The verification data of the edited image data 707 with verification data includes the verification data at the time of generation, the verification data of the first edit, and the verification data 708 of the second edit. Furthermore, the edited image data 707 with verification data includes, as metadata, a thumbnail image corresponding to the image data at the time of generation (image data before editing) and a thumbnail image corresponding to the image data generated in the first edit. In addition, this metadata may be configured to include information indicating the storage location of the thumbnail image instead of the thumbnail image. The image verification device 103 can display not only the image data generated in the second edit, but also the image data at the time of generation and the image data generated in the first edit on the display 406 using the received edited image data 707 with verification data.

[0048] Next, the image verification software 1000 of the image verification device 103 will be described.

[0049] Fig. 10 is a block diagram showing a schematic configuration of image verification software 1000 of image verification device 103 of Fig. 1. In Fig. 10, image verification software 1000 is composed of image input unit 1001, verification data acquisition unit 1002, object detection unit 1003, comparison data generation unit 1004, comparison unit 1005, and display control unit 1006. Processing of these modules is realized by CPU 401 of image verification device 103 reading the program of image verification software 1000 stored in storage 404 or the like into RAM 402 and executing the program read into RAM 402.

[0050] An image input unit 1001 inputs edited image data with verification data output from an image editing device 102 to an image verification software 1000 .

[0051] The verification data acquisition unit 1002 uses the input edited image data with verification data and the digital signature of the verification data attached to the edited image data with verification data to determine whether or not the edited image data with verification data has been tampered with. The verification data acquisition unit 1002 also acquires algorithm information and object information from the edited image data with verification data.

[0052] The object detection unit 1003 uses the algorithm information acquired by the verification data acquisition unit 1002 to detect an object area from this edited image data with verification data in a manner similar to that of the object detection units 502, 804 described above, and outputs area data and label information for the detected object area.

[0053] The comparison data generation unit 1004 calculates a hash value of the object region indicated by the region data in the edited image data with verification data by executing a hash function for the object region. In this manner, the comparison data generation unit 1004 calculates a hash function for each of all object regions detected by the object detection unit 1003. The comparison data generation unit 1004 uses the hash function indicated by the algorithm information acquired by the verification data acquisition unit 1002. The comparison data generation unit 1004 also generates comparison data including the calculated hash value, the region data output by the object detection unit 1003, and label information for each detected object region.

[0054] The comparison unit 1005 compares the comparison data with the verification data at the time of generation, the verification data for the first edit, etc., included in the edited image data with the verification data, and outputs the comparison result. The display control unit 1006 uses the comparison result output by the comparison unit 1005 to cause the display 406 to display the change in the object area caused by the editing.

[0055] 11(a) and 11(b) are flowcharts showing the procedure of image data verification processing executed by the image verification device 103 of FIG. 1. The image data verification processing is realized by the CPU 401 reading the program of the image verification software 1000 stored in the storage 404 into the RAM 402 and executing the program read into the RAM 402. The image data verification processing is executed, for example, when the image verification device 103 acquires image data to be verified from the image editing device 102. In this embodiment, as an example, it will be described that the edited image data with verification data 707 generated by performing two edits on the image data generated by the image generating device 101 is acquired from the image editing device 102.

[0056] 11A, first, the CPU 401 inputs the edited image data with verification data 707 to the image verification software 1000 (S1101). The CPU 401 uses the input edited image data with verification data 707 and the digital signature of the verification data added to the edited image data with verification data 707 to determine whether the edited image data with verification data 707 has been tampered with (S1102).

[0057] If it is determined in S1102 that the edited image data with verification data 707 has been tampered with, the CPU 401 displays an error notification indicating the details of the tampering on the display 406 (S1103), and ends this process.

[0058] If it is determined in S1102 that the edited image data with verification data 707 has not been tampered with, the CPU 401 acquires algorithm information and object region information from the edited image data with verification data 707 (S1104). The object region information acquired in S1104 is, for example, a plurality of pieces of object region information included in the verification data at the time of generation, or a plurality of pieces of object region information included in the verification data of the first edit.

[0059] Next, the CPU 401 detects an object region from the edited image data 707 with verification data using the algorithm information acquired in S1104 (S1105). This outputs the detected region data and label information. Note that if a number of object regions exceeding the threshold value are detected from the edited image data 707 with verification data, in S1105, the region data and label information of an aggregate region formed by combining the detected object regions may be output.

[0060] Next, CPU 401 executes a hash function on the object region indicated by the region data in post-edited image data 707 with verification data, and calculates a hash value for this object region. In this way, CPU 401 calculates a hash function for each of all object regions detected in S1105. Next, CPU 401 generates comparison data for each object region, including the calculated hash value, the output region data, and label information (step S1106). Note that, if the region data and label information of the aggregate region are output in S1105, comparison data is generated that includes the hash value obtained by executing the hash function on the aggregate region indicated by this region data in post-edited image data 707 with verification data, this region data, and this label information.

[0061] Next, the CPU 401 displays the verification screen 1200 of FIG. 12(a) on the display 406 based on the edited image data 707 with the verification data (S1107). The verification screen 1200 displays a thumbnail image 1201 corresponding to the image data generated by the image generating device 101. The verification screen 1200 also displays a thumbnail image 1202 corresponding to the edited image data generated by the first editing by the image editing device 102. The verification screen 1200 also displays edited image data 1203 generated by the second editing by the image editing device 102. It is assumed that the thumbnail image 1201 includes a circular region 1204 and a triangular region 1205 as examples of object regions. It is also assumed that the triangular region 1205 is enlarged in the first editing, and a square region 1206 is newly added in the second editing.

[0062] Next, the CPU 401 determines whether or not any object region has been selected by the user on the verification screen 1200 (S1108). The CPU 401 waits until any object region has been selected by the user on the verification screen 1200. When any object region has been selected by the user on the verification screen 1200, the CPU 401 performs a comparison process (S1109) of FIG. 11(b) in which the comparison data corresponding to the object region selected by the user on the verification screen 1200 (hereinafter referred to as the "selected object region") is compared with the object region information acquired in S1104. Note that, hereinafter, the hash value, region data, and label information included in the comparison data corresponding to the selected object region are referred to as the "hash value of the selected object region", the "region data of the selected object region", and the "label information of the selected object region", respectively.

[0063] FIG. 11B is a flowchart showing the procedure of the comparison process in S1109 of FIG.

[0064] In FIG. 11B, the CPU 401 determines whether the hash values ​​of the selected object region match during the editing process (S1110). For example, a circular region 1204 is an object region that has not changed since the image data was generated even after the second editing. A hash value that matches the hash value of such an object region is included in both the verification data at the time of generation of the post-edited image data 707 with verification data and the verification data for the first editing. When a hash value that matches the hash value of the selected object region is included in both the verification data at the time of generation of the post-edited image data 707 with verification data and the verification data for the first editing, it is determined that the hash values ​​of the selected object region match during the editing process. In this case, the comparison process proceeds to S1111.

[0065] In S1111, the CPU 401 outputs the result that the user-selected object region is unchanged. For example, the CPU 401 displays the circular region 1204, which is the selected object region, in a highlighted manner as shown in Fig. 12(a). Then, this process ends.

[0066] On the other hand, triangular region 1205 is an object region enlarged in the first edit, and square region 1206 is an object region added in the second edit. A hash value matching the hash value of such an object region is not included in at least one of the verification data at the time of generation and the verification data of the first edit in the edited image data with verification data 707. When a hash value matching the hash value of the selected object region is not included in at least one of the verification data at the time of generation and the verification data of the first edit, it is determined that the hash values ​​of the selected object region do not match during the editing process. In this case, the comparison process proceeds to S1112.

[0067] In S1112, it is determined whether object region information containing a hash value that matches the hash value of the selected object region is present in the object region information acquired in S1104, and whether the label information included in this object region information matches the label information of the selected object region.

[0068] For example, if the selected object region is a triangular region 1205 enlarged in the first edit, object region information including a hash value matching the hash value of the selected object region is present among the multiple object region information included in the verification data of the first edit acquired in S1104. Also, the label information included in this object region information matches information indicating the type of triangular region, that is, the label information of the selected object region. In this way, if object region information including a hash value matching the hash value of the selected object region is present among the object region information acquired in S1104, and the label information included in this object region information matches the label information of the selected object region, the comparison process proceeds to S1113.

[0069] In S1113, the CPU 401 outputs the result assuming that the same object region has changed. For example, the CPU 401 highlights the triangular region 1205, which is the selected object region, as shown in Fig. 12(b), and further displays a warning mark 1207 indicating that the region has changed in the first edit on the verification screen 1200. Then, this process ends.

[0070] On the other hand, for example, if the selected object region is square region 1206 added in the second edit, a hash value matching the hash value of the selected object region does not exist in the plurality of object region information acquired in S1104. In this way, if no object region information including a hash value matching the hash value of the selected object region exists in the object region information acquired in S1104, in S1112, the comparison process proceeds to S1114.

[0071] In S1114, the CPU 401 outputs the result that an object region has been newly added. For example, the CPU 401 displays the rectangular region 1206, which is the selected object region, in a highlighted manner as shown in Fig. 12(c), and further displays a newly added mark 1208 indicating that the region was added in the second edit on the verification screen 1200. Then, this process ends.

[0072] In addition, in the above-mentioned image data verification process, for example, if the circular area 1204 is deleted in the second editing, a verification screen including a deletion mark indicating that the circular area 1204 has been deleted may be displayed on the display 406 in S1107.

[0073] As described above, according to the embodiment, object region information of the object region detected from the post-edited image data with verification data 707 is compared with object region information of the verification data at the time of generation that was added to the post-edited image data with verification data 707, and object region information of the verification data for the first edit. Based on the comparison result, a predetermined notification that makes the user recognize that a change has occurred due to editing by the image editing device 102 is displayed on the verification screen 1200. This allows the user to easily recognize which object region in the image data has changed due to editing by the image editing device 102.

[0074] Furthermore, in the above-described embodiment, the image data generated by the image generating device 101 is image data generated by imaging using the imaging unit 206. This allows the user to easily recognize which object region in the image data generated by imaging using the imaging unit 206 has changed due to editing using the image editing device 102.

[0075] Furthermore, in the above-described embodiment, algorithm information indicating the algorithm used when the object detection unit 502 of the image generating device 101 detected an object region from the image data is added to the image data with verification data 701, the edited image data with verification data 704, and the edited image data with verification data 707. As a result, in a configuration in which the generation, editing, and verification of image data are performed by separate devices, the detection of an object region required for the above-described notification can be performed using the same algorithm information.

[0076] In the above-described embodiment, the predetermined notification is a deletion mark that notifies the user that an object region has been deleted as a result of editing by the image editing device 102. This allows the user to easily recognize which object region has been deleted from the image data as a result of editing by the image editing device 102.

[0077] In the above-described embodiment, the predetermined notification is a warning mark 1207 that notifies the user that the object area specified by the user has changed as a result of editing by the image editing device 102. This allows the user to easily recognize which object area has changed in the image data as a result of editing by the image editing device 102.

[0078] Furthermore, in the above-described embodiment, when editing is performed multiple times by the image editing device 102, the predetermined notification is a warning mark 1207 that allows the user to recognize how many edits have been performed by the image editing device 102 that have changed the object area specified by the user. This allows the user to easily recognize which object area in the image data has changed through editing by the image editing device 102 and by how many edits.

[0079] Furthermore, in the above-described embodiment, the predetermined notification is a newly added mark 1208 that notifies the user that the object region specified by the user has been newly added as a result of editing by the image editing device 102. This allows the user to easily recognize which object region has been newly added to the image data as a result of editing by the image editing device 102.

[0080] In this embodiment, the configuration in which label information is used for the determination in S1112 has been described, but area data may be used instead of label information. For example, in S1112, object area information including a hash value that matches the hash value of the selected object area is present in the object area information acquired in S1104, and it is determined whether or not the area data included in this object area information approximately matches the area data of the selected object area. In this way, even in the configuration in which area data is used for the determination in S1112, it is possible to determine the change in the object area during the editing process.

[0081] In addition, in this embodiment, feature points of the object region extracted in conjunction with the detection of the object region may be used in the judgment of S1112. In such a configuration, comparison unit 1005 of image verification device 103 performs feature point matching and performs processing to regard object regions exceeding a threshold as the same object region. In this way, even in a configuration in which feature points are used in the judgment of S1112, it is possible to judge changes in the object region during the editing process.

[0082] Furthermore, in this embodiment, a configuration may be adopted in which whether or not to perform the above-mentioned predetermined notification can be controlled by an arbitrary setting of the image generating device 101.

[0083] Furthermore, the above-mentioned predetermined notification is not limited to the above-mentioned warning mark 1207, new addition mark 1208, and deletion mark. For example, a message notifying that the object area has changed due to editing, such as "Changed in the first editing" may be displayed. In such a configuration, the edited image data with verification data 707 may not include, as metadata, a thumbnail image corresponding to the image data at the time of generation (image data before editing) or a thumbnail image corresponding to the image data generated in the first editing.

[0084] The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications are possible based on the spirit of the present invention, and these modifications are not excluded from the scope of the present invention.

[0085] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0086] The disclosure of this embodiment includes the following configurations and methods. (Configuration 1) An image verification system comprising: a means for adding first object area information regarding an object area detected from the generated image data to the generated image data; a means for editing the image data to which at least the first object area information has been added; a means for adding second object area information regarding the object area detected from the image data after editing by the editing means to the edited image data; a means for comparing third object area information regarding the object area detected from the edited image data with the first object area information and the second object area information added to the edited image data, respectively; and a means for displaying a predetermined notification that causes a user to recognize that the object area detected from the generated image data has changed due to editing by the editing means, based on the result of the comparison by the comparing means. (Configuration 2) The image verification system according to configuration 1, further comprising an imaging means, wherein the generated image data is image data generated by imaging using the imaging means. (Configuration 3) The image verification system described in configuration 1 or 2, characterized in that the means for adding to the generated image data further adds, to the generated image data, information indicating a predetermined algorithm used when detecting an object region from the generated image data, and the image data edited by the editing means is added with at least the first object region information and information indicating the predetermined algorithm. (Configuration 4) The image verification system according to any one of configurations 1 to 3, wherein the specified notification is a notification that notifies the user that an object area has been deleted as a result of editing by the editing means. (Configuration 5) An image verification system described in any one of configurations 1 to 4, characterized in that the specified notification is a notification that makes the user recognize that the object area specified by the user has changed as a result of editing by the editing means. (Configuration 6) The image verification system described in Configuration 5, characterized in that when editing is performed multiple times by the editing means, the specified notification is a notification that allows the user to recognize how many edits by the editing means have changed the object area specified by the user. (Configuration 7) An image verification system described in any one of configurations 1 to 6, characterized in that the specified notification is a notification that notifies the user that the specified object area has been newly added as a result of editing by the editing means. (Configuration 8) The first object region information, the second object region information, and the third object region information each include a hash value obtained by applying a hash function to the detected object region, The image verification system of any one of configurations 1 to 7, characterized in that the comparing means compares a hash value contained in the third object area information with a hash value contained in the first object area information and a hash value contained in the second object area information, respectively. (Configuration 9) An image verification system described in any one of configurations 1 to 8, characterized in that the first object area information, the second object area information, and the third object area information each include information indicating attributes of the detected object area, and the comparing means compares the information indicating the attributes contained in the third object area information with the information indicating the attributes contained in the first object area information and the information indicating the attributes contained in the second object area information, respectively. (Configuration 10) An image verification system described in any one of configurations 1 to 9, characterized in that the first object area information, the second object area information, and the third object area information each include feature points of a detected object area, and the comparing means compares the feature points included in the third object area information with the feature points included in the first object area information and the feature points included in the second object area information, respectively. (Configuration 11) An image verification system described in any one of configurations 1 to 10, characterized in that when multiple object regions exceeding a threshold are detected from the generated image data, the means for assigning to the generated image data assigns information of a first aggregate region formed by combining the multiple object regions to the generated image data, and when multiple object regions exceeding the threshold are detected from the edited image data, the means for assigning to the edited image data assigns information of a second aggregate region formed by combining the multiple object regions to the edited image data, and the means for comparing, when multiple object regions exceeding the threshold are detected from the edited image data, compares information of a third aggregate region formed by combining the multiple object regions with the information of the first aggregate region and the information of the second aggregate region assigned to the edited image data, respectively. [Explanation of symbols]

[0087] 100 Image Verification System 101 Image generating device 102 Image editing equipment 103 Image verification device 201, 301, 401 CPU 406 Display 803 Image Editing Department 1005 Comparison section 1006 Display control unit 1207 Warning Mark 1208 Newly Added Mark

Claims

1. means for adding first object region information regarding an object region detected from the generated image data to the generated image data; means for editing the image data to which at least the first object region information has been added; a means for adding second object region information regarding an object region detected from the image data edited by the editing means to the edited image data; means for comparing third object region information relating to an object region detected from the edited image data with the first object region information and the second object region information added to the edited image data; and a means for displaying a predetermined notification that notifies the user that an object area detected from the generated image data has changed due to editing by the editing means, based on the results of the comparison by the comparing means.

2. Further comprising an imaging means, 2. The image verification system according to claim 1, wherein the generated image data is image data generated by imaging using the imaging means.

3. The means for adding to the generated image data further adds, to the generated image data, information indicating a predetermined algorithm used when detecting an object region from the generated image data; 2. The image verification system according to claim 1, wherein the image data edited by said editing means is provided with at least said first object region information and information indicating said predetermined algorithm.

4. 2. The image verification system according to claim 1, wherein the predetermined notification is a notification that notifies the user that an object area has been deleted as a result of editing by the editing means.

5. 2. The image verification system according to claim 1, wherein the predetermined notification is a notification that makes the user aware that an object area designated by the user has changed as a result of editing by the editing means.

6. The image verification system described in claim 5, characterized in that when editing is performed multiple times by the editing means, the specified notification is a notification that allows the user to recognize how many edits by the editing means have changed the object area specified by the user.

7. 2. The image verification system according to claim 1, wherein the predetermined notification is a notification that makes the user aware that the object area specified by the user has been newly added as a result of editing by the editing means.

8. each of the first object region information, the second object region information, and the third object region information includes a hash value obtained by executing a hash function on the detected object region; The image verification system according to claim 1, characterized in that the comparing means compares the hash value contained in the third object area information with the hash value contained in the first object area information and the hash value contained in the second object area information, respectively.

9. each of the first object region information, the second object region information, and the third object region information includes information indicating an attribute of a detected object region; The image verification system described in claim 1, characterized in that the comparing means compares information indicating attributes contained in the third object area information with information indicating attributes contained in the first object area information and information indicating attributes contained in the second object area information, respectively.

10. each of the first object region information, the second object region information, and the third object region information includes feature points of a detected object region; The image verification system according to claim 1, characterized in that the comparing means compares the feature points contained in the third object area information with the feature points contained in the first object area information and the feature points contained in the second object area information, respectively.

11. When a plurality of object regions exceeding a threshold value are detected from the generated image data, the means for adding to the generated image data adds information of a first aggregate region obtained by combining the plurality of object regions to the generated image data, When a plurality of object regions exceeding the threshold value are detected from the edited image data, the means for adding to the edited image data adds information of a second aggregate region obtained by combining the plurality of object regions to the edited image data, The image verification system described in claim 1, characterized in that when multiple object areas exceeding the threshold are detected from the edited image data, the comparing means compares information of a third aggregate area formed by combining the multiple object areas with information of the first aggregate area and information of the second aggregate area added to the edited image data, respectively.

12. adding first object region information to the generated image data, the first object region information relating to an object region detected from the generated image data; editing the image data to which at least the first object region information has been assigned; adding second object region information regarding an object region detected from the image data edited by the editing means to the edited image data; comparing third object region information relating to an object region detected from the edited image data with the first object region information and the second object region information added to the edited image data; and displaying a predetermined notification based on the results of the comparison by the comparing means, to notify the user that the object area detected from the generated image data has changed due to editing by the editing means.

13. A program for causing a computer to execute an image verification method, The image verification method includes: adding first object region information to the generated image data, the first object region information relating to an object region detected from the generated image data; editing the image data to which at least the first object region information has been assigned; adding second object region information regarding an object region detected from the image data edited by the editing means to the edited image data; comparing third object region information relating to an object region detected from the edited image data with the first object region information and the second object region information added to the edited image data; and displaying a predetermined notification based on the results of the comparison by the comparing means, to notify the user that the object area detected from the generated image data has changed as a result of editing by the editing means.

Images