Image processing device and method

Abstract

The present disclosure relates to an image processing device and a method that make it possible to more easily confirm partial manipulation to which an image has been subjected. The present invention displays: an entire post-manipulation image presenting a manipulation detection region; and a magnified post-manipulation image presenting the manipulation detection region with a different method from when presenting the entire post-manipulation image. Moreover, manipulation to which an entire original image has been subjected is detected using the entire original image and the entire post-manipulation image, and manipulation detection region information regarding the manipulation detection region where said manipulation was detected is supplied. The present disclosure can be applied to, for example, an image processing device, an electronic unit, an image processing method, a program, or the like.

Description

Image processing device and method

[0001] The present disclosure relates to an image processing device and method, and more particularly to an image processing device and method that enable partial processing applied to an image to be more easily confirmed.

[0002] In recent years, advances in image generation technology using AI (Artificial Intelligence) have made it easier to create photographic images and videos, which has led to various social issues such as fake news. As a result, there has been a demand for a more accurate way to check the type of editing that has been done. For example, editing can be confirmed by comparing the image with the original.

[0003] When making such a comparison, it is conceivable to detect an edited area and display the edit detection area to assist in and facilitate confirmation of the edit. However, when it is desired to visually check not only whether or not an edit has been performed but also the detailed changes caused by slight edits, simply showing the edited area does not provide sufficient visibility, and there is a risk that checking the edits will require cumbersome work. For example, by enlarging and comparing the images, more detailed changes can be more easily confirmed. One method for enlarging an image is to display an enlarged image of the area around the cursor nearby (see, for example, Patent Document 1).

[0004] Japanese Patent Application Publication No. 10-133639

[0005] However, simply enlarging an image reduces the visibility of the manipulation detection area in the enlarged image, which can lead to the risk of requiring cumbersome work to confirm the manipulation. In particular, it can be difficult to detect partial manipulation, which is manipulation that alters only part of an image, which can lead to the risk of requiring cumbersome work to confirm the manipulation.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to more easily check partial processing applied to an image.

[0007] An image processing device according to one aspect of the present technology is an image processing device that includes a display control unit that displays an entire processed image, which is an image in which partial processing has been applied to an entire original image and which presents a processing detection area in which the partial processing has been detected, and an enlarged processed image, which is an image in which a partial area of ​​the entire processed image has been enlarged and which presents the processing detection area in a manner different from that in the entire processed image, and a display unit that displays the entire processed image and the enlarged processed image in accordance with the control of the display control unit.

[0008] An image processing method according to one aspect of the present technology is an image processing method that includes controlling the display of an entire processed image, which is an image in which partial processing has been applied to an entire original image and which presents a processing detection area in which the partial processing has been detected, and an enlarged processed image, which is an image in which a partial area of ​​the entire processed image has been enlarged and which presents the processing detection area in a manner different from that in the entire processed image, and displaying the entire processed image and the enlarged processed image in accordance with the control.

[0009] Another aspect of the image processing device of the present technology is an image processing device that includes a processing detection unit that detects partial processing performed on an entire original image using an entire original image and an entire processed image in which partial processing has been performed on the entire original image, and a processing detection area information supply unit that supplies processing detection area information regarding the processing detection area in which the partial processing has been detected.

[0010] Another aspect of the image processing method of the present technology is an image processing method that includes using an entire original image and an entire processed image in which partial processing has been applied to the entire original image, detecting the partial processing applied to the entire original image, and supplying processing detection area information regarding the processing detection area in which the partial processing has been detected.

[0011] An image processing device according to yet another aspect of the present technology is an image processing device that includes an alignment estimation image acquisition unit that acquires an alignment estimation image, which is an overall original image that has been estimated to be aligned with an overall processed image that has been subjected to partial processing on the overall original image; a display control unit that displays an enlarged processed image that is an enlarged partial region of the overall processed image and an enlarged original image that is an enlarged partial region of the acquired alignment estimation image at the same positions as each other; and a display unit that displays the enlarged processed image and the enlarged original image in accordance with the control of the display control unit.

[0012] An image processing method according to yet another aspect of the present technology is an image processing method that includes: acquiring an aligned estimated image, which is an overall original image that has been estimated to be aligned with an overall processed image in which partial processing has been performed on the overall original image; controlling an enlarged processed image obtained by enlarging a partial region of the overall processed image and an enlarged original image obtained by enlarging the partial region of the acquired aligned estimated image so that they are displayed in the same positions as each other; and displaying the enlarged processed image and the enlarged original image in accordance with the control.

[0013] In one aspect of the image processing device and method of the present technology, control is performed to display an entire processed image, which is an image in which partial processing has been applied to the entire original image and which presents the processing detection area in which the partial processing has been detected, and an enlarged processed image, which is an image in which a partial area of ​​the entire processed image has been enlarged and which presents the processing detection area in a manner different from that of the entire processed image, and the entire processed image and the enlarged processed image are displayed in accordance with this control.

[0014] In another aspect of the image processing device and method of the present technology, an entire original image and an entire processed image in which partial processing has been applied to the entire original image are used, the partial processing applied to the entire original image is detected, and processing detection area information regarding the processing detection area in which the partial processing has been detected is supplied.

[0015] In yet another aspect of the image processing device and method of the present technology, an estimated alignment image is obtained, which is an overall original image that has been estimated to be aligned with an overall processed image in which partial processing has been performed on the overall original image, and an enlarged processed image in which a partial region of the overall processed image has been enlarged and an enlarged original image in which a partial region of the obtained estimated alignment image has been enlarged are controlled to be displayed in the same positions as each other, and the enlarged processed image and the enlarged original image are displayed in accordance with this control.

[0016] 1 is a diagram illustrating an example of the main configuration of a processing confirmation system. FIG. 1 is a diagram illustrating an example of a displayed image. FIG. 2 is a diagram illustrating an example of a display of an enlarged processed image. FIG. 3 is a diagram illustrating an example of a grid size. FIG. 3 is a diagram illustrating an example of a display of an enlarged original image. FIG. 4 is a diagram illustrating an example of a display of an enlarged processed image. FIG. 5 is a diagram illustrating an example of a display of an enlarged processed image. FIG. 6 is a block diagram illustrating an example of the main configuration of a terminal device. FIG. 7 is a block diagram illustrating an example of the main configuration of a processing confirmation server. FIG. 8 is a flowchart illustrating an example of the flow of a comparison display process. FIG. 9 is a flowchart illustrating an example of the flow of an enlargement display process. FIG. 10 is a flowchart illustrating an example of the flow of a processing confirmation process. FIG. 11 is a diagram illustrating an example of the main configuration of a registration estimation system. FIG. 12 is a block diagram illustrating an example of the main configuration of a registration estimation server. FIG. 13 is a flowchart illustrating an example of the flow of a registration estimation process. FIG. 14 is a flowchart illustrating an example of the flow of a registration estimation process. FIG. 15 is a block diagram illustrating an example of the main configuration of a terminal device. FIG. 16 is a flowchart illustrating an example of the flow of a comparison display process. FIG. 17 is a diagram illustrating an example of a display. FIG. 18 is a diagram illustrating an example of a display. FIG. 19 is a flowchart illustrating an example of the flow of a processing confirmation process.

[0017] The following describes modes for carrying out the present disclosure (hereinafter referred to as embodiments). The description will be given in the following order: 1. Literature, etc. supporting technical content and technical terminology 2. Confirmation of editing 3. Change of editing detection area presentation method 4. Alignment of original image and edited image 5. Supplementary notes

[0018] <1. Literature, etc. supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the content described in the embodiments, but also the content described in the following patent documents and non-patent documents that were publicly known at the time of filing, as well as the content of other documents referenced in the following patent documents and non-patent documents.

[0019] Patent Document 1: (mentioned above) Patent Document 2: (Japanese Patent No. 4807277) Patent Document 3: (Japanese Patent Publication No. 2020-173640)

[0020] In other words, the contents of the above-mentioned patent documents and the contents of other documents referenced in the above-mentioned patent documents are also used as the basis for determining the support requirements.

[0021] <2. Checking for Editing> <Reducing the Visibility of Edited Areas> In recent years, advances in image generation technology using AI (Artificial Intelligence) have made it easier to create photographic images and videos, which has led to various social issues such as fake news. As a result, there has been a demand for a way to more accurately check what kind of editing has been done. For example, editing can be confirmed by comparing it with the original image.

[0022] When comparing, a method for assisting in the confirmation of the editing may be considered, for example, by detecting the edited area and displaying the edited detection area to facilitate the confirmation. However, when it is desired to visually check not only whether or not editing has occurred but also detailed changes due to slight editing, simply showing the edited area may not provide sufficient visibility, and checking the editing may require cumbersome work. For example, by enlarging and comparing the images, more detailed changes can be more easily confirmed. For example, Patent Document 1 discloses a method for enlarging an image by displaying an enlarged image near the cursor.

[0023] However, simply enlarging an image may reduce the visibility of the manipulation detection area in the enlarged image. For example, if the granularity of the manipulation detection area is the same in both the entire image and the enlarged image, it may be difficult to obtain more detailed information about the manipulation detection area in the enlarged image. Furthermore, in the entire image, highlighting the manipulation detection area by masking or the like can help the user understand where the manipulation detection area is located. However, in the manipulated image, the masking reduces the visibility of the manipulation detection area, making it difficult to obtain more detailed information about the manipulation applied within the manipulation detection area. This reduced visibility of the manipulation detection area may require more tedious work to confirm the manipulation. It may be particularly difficult to detect partial manipulation, requiring tedious work to confirm the manipulation.

[0024] In this specification, "partial processing" refers to processing that alters a part of an image. For example, this "partial processing" may include processing such as adding, changing, or deleting a subject object in an image (for example, processing performed using generative AI (Artificial Intelligence)). This "partial processing" may also include processing such as blurring or adding mosaics to a partial area of ​​an image. This "partial processing" may also include processing such as superimposing a synthetic image such as a logo or subtitles. This "partial processing" may also include processing such as skin beautification of the subject's face. This "partial processing" may also include processing such as changing the size or shape of the subject's facial contours or features.

[0025] 3. Changing the Method for Presenting the Editing Detection Area Presentation Presentation of the Editing Area in an Enlarged Image Therefore, the editing detection area is presented in an enlarged image in a manner different from that of the entire image. For example, a first image processing device may include a display control unit that displays an entire edited image, which is an image obtained by partial editing of an entire original image and presents an editing detection area where the partial editing has been detected, and an enlarged edited image, which is an image obtained by enlarging a partial region of the entire edited image and presents the editing detection area in a manner different from that of the entire edited image, and a display unit that displays the entire edited image and the enlarged edited image in accordance with the control of the display control unit. Furthermore, a first image processing method executed by the first image processing device may include controlling the display of an entire edited image, which is an image obtained by partial editing of an entire original image and presents an editing detection area where the partial editing has been detected, and an enlarged edited image, which is an image obtained by enlarging a partial region of the entire edited image and presents the editing detection area in a manner different from that of the entire edited image, and displaying the entire edited image and the enlarged edited image in accordance with the control. The first program also causes the computer to execute processing including controlling the display of an overall processed image, which is an image in which partial processing has been applied to the overall original image and which presents the processing detection area in which the partial processing has been detected, and an enlarged processed image, which is an image in which a partial area of ​​the overall processed image has been enlarged and which presents the processing detection area in a manner different from that of the overall processed image, and displaying the overall processed image and the enlarged processed image in accordance with the control.

[0026] By doing this, the first image processing device, the first image processing device executing the first image processing method, or the computer executing the first program can suppress a reduction in the visibility of the processing detection area, making it easier for users and others to confirm partial processing applied to the image.

[0027] The second image processing device includes an edit detection unit that detects partial edits made to the entire original image using the entire original image and an edited overall image obtained by applying partial edits to the entire original image, and an edit detection area information supply unit that supplies edit detection area information related to the edit detection area where the partial edits are detected. The second image processing method executed by the second image processing device includes detecting partial edits made to the entire original image using the entire original image and the edited overall image obtained by applying partial edits to the entire original image, and supplying edit detection area information related to the edit detection area where the partial edits are detected. The second program causes a computer to execute processes that include detecting partial edits made to the entire original image using the entire original image and the edited overall image obtained by applying partial edits to the entire original image, and supplying edit detection area information related to the edit detection area where the partial edits are detected.

[0028] By providing the processing detection area information in this manner, the second image processing device, the second image processing device executing the second image processing method, or the computer executing the second program can suppress a reduction in the visibility of the processing detection area, making it easier for users and others to confirm partial processing applied to the image.

[0029] <Processing Confirmation System> Fig. 1 is a diagram showing an example of the configuration of a processing confirmation system, which is one aspect of an image processing system to which the present technology is applied. The processing confirmation system 100 shown in Fig. 1 is a system for confirming processing applied to an image.

[0030] 1, the processing confirmation system 100 includes a terminal device 111 and a processing confirmation server 112. The terminal device 111 and the processing confirmation server 112 are communicably connected to each other via a network 110, and can exchange information with each other via this communication. Note that this communication may be wired communication, wireless communication, or both.

[0031] The network 110 is a communication network configured with any communication medium. The communication performed via the network 110 may be wired communication, wireless communication, or both. In other words, the network 110 may be a communication network for wired communication, a communication network for wireless communication, or a communication network configured with both. Furthermore, the network 110 may be configured with a single communication network or multiple communication networks.

[0032] For example, the network 110 may include the Internet. The network 110 may also include a public telephone network. Furthermore, the network 110 may also include a wide-area communication network for wireless mobile devices, such as a so-called 3G line or 4G line. For example, the network 110 may also include a low-power wide-area (LPWA) communication network, such as LTE-M, which is capable of long-distance data communication and consumes low current. The network 110 may also include a wide-area (WAN) network or a local-area (LAN) network. The network 110 may also include a wireless communication network that communicates in accordance with the Bluetooth (registered trademark) standard. The network 110 may also include a short-range wireless communication channel, such as NFC (Near Field Communication). The network 110 may also include an infrared communication channel. The network 110 may also include a wired communication network that complies with standards such as HDMI (High-Definition Multimedia Interface) (registered trademark) and USB (Universal Serial Bus) (registered trademark). In this way, the network 110 may include a communication network or a communication path of any communication standard. Note that the communication network or the communication path may include not only a communication medium such as a cable, but also devices and circuits necessary for communication, such as a communication device or a relay device.

[0033] The terminal device 111 functions as an interface for a user or the like to confirm partial editing (editing that alters part of an image) applied to an image. For example, the terminal device 111 requests the editing confirmation server 112 to confirm partial editing by supplying an original image and an edited image specified by the user or the like to the editing confirmation server 112. Note that, in this specification, an "original image" refers to an image in the generated state (an image that has not been edited since it was generated). For example, an unedited captured image obtained by capturing an image of a subject with a camera or the like can be the original image. Furthermore, an "edited image" refers to an image that has been edited in some way on the original image. In other words, the terminal device 111 selects an edited image for confirming the partial editing applied and an original image corresponding to the edited image. The terminal device 111 supplies the selected original image and edited image to the editing confirmation server 112 via the network 110.

[0034] In this specification, the entire original image and the entire processed image are also referred to as the "whole image." In particular, the entire image of the original image is also referred to as the entire original image. Furthermore, the entire image of the processed image is also referred to as the entire processed image. The terminal device 111 supplies the entire original image and the entire processed image to the processing confirmation server 112.

[0035] The editing confirmation server 112 executes a process for checking partial editing, which is editing that alters a part of an image. For example, the editing confirmation server 112 acquires an entire original image and an entire edited image supplied from the client terminal device 111. The editing confirmation server 112 uses these images to detect partial editing (editing to obtain an edited image from the original image) applied to the entire original image. As a result (response) of the confirmation, the editing confirmation server 112 supplies the terminal device 111 with editing detection area information regarding editing detection areas where partial editing was detected.

[0036] The processing confirmation system 100 may include an authenticity determination server 113. This authenticity determination server 113 may be communicably connected to the terminal device 111 and the processing confirmation server 112 via the network 110. The authenticity determination server 113 may determine the authenticity of an original image and an edited image for which processing is to be confirmed. For example, the authenticity determination server 113 may acquire these images from the terminal device 111 and determine their authenticity. Any method may be used to determine this authenticity. In other words, partial processing may be confirmed using an image whose authenticity has been determined.

[0037] The terminal device 111 acquires the editing detection area information supplied from the editing confirmation server 112, and generates and displays a display image based on the information. In this display image, the entire edited image and the enlarged edited image are displayed, and the editing detection area is presented in a different manner in each image.

[0038] <Full Image Display> For example, the terminal device 111 generates and displays a display image 201 as shown in Fig. 2. In this display image 201, an entire edited image 211 and an entire original image 212 are displayed. At this time, an edit detection area 221 detected by the edit confirmation server 112 is presented in the entire edited image 211 based on the edit detection area information. For example, a predetermined masking process or the like is applied to the edit detection area 221, and the edit detection area 221 is presented in a state that is distinguishable from other areas. In this way, the user can easily visually determine which part of the entire edited image 211 is the edit detection area 221.

[0039] Other information related to the processing may also be displayed in the display image 201. For example, a list of the detected processing details included in the processing detection area information may be displayed in the processing details display area 213. Such a display allows the user to more easily understand what processing has been performed.

[0040] <Enlarged Image Display> When, for example, the user moves the cursor within the overall processed image 211 in this display image 201 (hereinafter also referred to as "mouse over"), a new window 241 is displayed as shown in FIG. 3 , and an enlarged processed image 242 is displayed in the window 241. This enlarged processed image 242 is an image obtained by enlarging a partial region of the overall processed image 211. For example, a region (a region of a predetermined range) 232 around a cursor 231 moved by the user within the overall processed image 211 may be enlarged and displayed as the enlarged processed image 242. Note that this enlargement magnification may be any magnification. For example, it may be a predetermined magnification (fixed value), or the magnification may be variable. For example, the magnification may be set by the user, etc.

[0041] The display position of this enlarged processed image 242 (window 241) is arbitrary. For example, the display position of the enlarged processed image 242 (window 241) may be outside the entire processed image 211. In this way, the user can confirm the processing detection area while comparing the entire image and the enlarged image. For example, the enlarged processed image 242 (window 241) may be displayed superimposed on the entire original image.

[0042] As described above, in this enlarged processed image, the processing detection area is presented in a different manner from that in the case of the entire processed image.

[0043] <Changing the Granularity of the Editing Detection Area> For example, the granularity of the editing detection area may be changed. For example, the display control unit of the first image processing device may present the editing detection area in the enlarged edited image at a finer granularity than in the entire edited image. For example, as shown in FIG. 4, the editing detection area may be displayed in 8x8 grid units in the entire edited image, and the editing detection area may be displayed in 16x16 grid units in the enlarged edited image. In this way, the user or the like can specify the range of the editing detection area in the enlarged edited image more precisely than in the entire edited image.

[0044] In other words, the manipulation detection area may be presented in the overall processed image at a coarser granularity than in the enlarged processed image. For example, as shown in FIG. 4 , the manipulation detection area information indicates the manipulation detection area at the granularity of the enlarged processed image, and in the overall processed image, the information of the 2x2 grid may be combined to present the manipulation detection area as information of a 1x1 grid (i.e., at a coarser granularity). In the example of FIG. 4 , the manipulation detection area information and the enlarged processed image indicate that the lower left area of ​​the 2x2 grid of interest is not a manipulation detection area. However, in the overall processed image, the information of the 2x2 grid is combined, and the entire 2x2 grid is presented as the manipulation detection area. This can prevent problems such as the grid being too small in the overall processed image, making it difficult to see the manipulation detection area, and can make it easier for users to identify areas of the image where partial manipulation has been applied.

[0045] The granularity of the processing detection area may be any granularity as long as it is visible. For example, the granularity may be a predetermined value (fixed value) or may be variable. For example, the granularity may be set by a user or the like. Furthermore, the granularity may be set according to the magnification factor of the image or the like.

[0046] <Changing Masking> The masking applied to the processing detection area, etc. may be changed. For example, the display control unit of the first image processing device may mask the processing detection area of ​​the entire processed image and display it, and mask the non-processing detection area of ​​the enlarged processed image and display it. Here, the "non-processing detection area" refers to an area other than the processing detection area.

[0047] 3, for example, in the case of the display image 201, in the entire processed image 211, the processing detection area 221 is masked (for example, filled with semi-transparent gray), and in the enlarged processed image 242, the area other than the processing detection area 243 (i.e., the non-processing detection area) is masked (for example, filled with semi-transparent gray). In this way, the user can easily grasp the position of the processing area in the entire image, and can easily grasp the state of the processing detection area (the state of partial processing) in the enlarged image.

[0048] <Display of Enlarged Original Image> An enlarged original image, which is an enlarged image of the original image, may be displayed. For example, the display control unit of the first image processing device may further display an enlarged original image, which is an enlarged partial region of the entire original image, at the same position as the enlarged processed image. In this way, the user or the like can compare the original image and the processed image using the enlarged image. Therefore, the user or the like can compare more detailed information about the processing detection area, etc., and can understand the details of the partial processing, etc., in more detail.

[0049] This enlarged original image may be an image obtained by enlarging a partial region of an estimated registration image, which is an entire original image estimated to be registered with respect to the entire processed image. In other words, the processed image and the original image may be registered. By doing so, it is possible to easily display the enlarged original image at a position corresponding to the enlarged processed image (e.g., the same position). Generally, when a user or the like compares an enlarged processed image and an enlarged original image, it is useful to compare enlarged images at corresponding positions (e.g., the same position) with each other. Therefore, it is useful in the comparison to display the enlarged original image at a position corresponding to the enlarged processed image (e.g., the same position) at the same position as the enlarged processed image.

[0050] The first image processing device may further include an alignment estimated image acquisition unit that acquires the alignment estimated image. The display control unit may then display an image obtained by enlarging a partial region of the acquired alignment estimated image as an enlarged original image. In this way, for example, the terminal device 111 can display the enlarged original image using the alignment estimated image supplied from the processing confirmation server 112.

[0051] In this case, the second image processing device may further include an alignment unit that performs alignment estimation to determine the positional correspondence between the entire original image and the entire processed image, and an alignment estimation supply unit that supplies, as a result of the alignment estimation, at least one of alignment estimation information regarding the alignment estimation and an alignment estimation image, which is the entire original image reflecting the processing result of the alignment estimation.

[0052] For example, the alignment estimation may include a position change operation estimation that estimates the type and degree of a position change operation required to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image. The position change operation estimation may also include a process of determining whether the degree of the estimated position change operation is within an acceptable range. The position change operation estimation may also include a process of reflecting the estimated position change operation in the entire original image.

[0053] It should be noted that the repositioning operation may be an operation on the entire image that involves changing the position of elements within the image, for example, the repositioning operation may include one or more of cropping, rotating, resizing, stretching, and deskewing.

[0054] The registration estimation may include a distortion matching estimation for estimating a difference in lens distortion correction between the entire original image and the entire processed image, and the distortion matching estimation may further include a process for reflecting the estimated difference in lens distortion correction in the entire original image.

[0055] For example, in the second image processing device, the alignment unit may perform the distortion alignment estimation using lens information included in the metadata of the overall processed image and a distortion table corresponding to the lens information. Alternatively, the alignment unit may perform the distortion alignment estimation using camera settings at the time of shooting included in the metadata of the overall processed image.

[0056] Furthermore, the above-described alignment estimation information may include position change operation estimation information related to position change operation estimation, which estimates the type and degree of position change operation required to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image. Note that the position change operation may be an operation on the entire image that involves changing the position of an image element. For example, the position change operation estimation information may include information indicating the type and degree of position change operation estimated by the position change operation estimation. Furthermore, the position change operation estimation information may include information indicating a determination result as to whether the degree of position change operation estimated by the position change operation estimation is within an acceptable range.

[0057] Furthermore, the above-described alignment estimation information may include distortion adjustment estimation information related to distortion adjustment estimation that estimates a difference in lens distortion correction between the entire original image and the entire processed image. For example, the distortion adjustment estimation information may include information indicating a difference in lens distortion correction estimated by the distortion adjustment estimation.

[0058] The above-mentioned alignment estimated image may also include a position change operation estimated image in which a position change operation estimated by position change operation estimation that estimates the type and degree of position change operation necessary to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image is reflected in the entire original image.The above-mentioned alignment estimated image may also include a distortion adjustment estimated image in which a difference in lens distortion correction estimated by distortion adjustment estimation that estimates the difference in lens distortion correction between the entire original image and the entire processed image is reflected in the entire original image.

[0059] For example, in the second image processing device, the processing detection unit may detect partial processing using the alignment estimated image and the overall processed image. Also, the processing detection unit may detect partial processing using the alignment estimated information.

[0060] Furthermore, the alignment unit may skip the execution of a predetermined process among the processes performed to generate the entire processed image from the entire original image during the alignment estimation. The predetermined process may be, for example, partial processing of a partial region of the entire original image. The predetermined process may also be, for example, color correction. The color correction may include, for example, one or more processes such as white balance adjustment, color balance adjustment, hue adjustment, saturation adjustment, luminance adjustment, brightness adjustment, or color space conversion.

[0061] The display of such an enlarged original image may be switched with the display of the enlarged image. That is, either the enlarged image or the enlarged original image may be displayed at the same position. For example, in the first image processing device, a display control unit may switch the display of the enlarged image to the display of the enlarged original image. This switching may be performed, for example, based on an instruction from a user or the like. For example, when the enlarged image 242 is displayed as shown in FIG. 3, as shown in FIG. 5, if the user clicks the mouse button, an enlarged original image 262 may be displayed in the window 241 instead of the enlarged image 242. Furthermore, if the user clicks the mouse button again in the state of FIG. 5, the state may return to the state of FIG. 3 (i.e., the enlarged image 242 is displayed in the window 241).

[0062] Furthermore, the enlarged image and the enlarged original image may be superimposed (semi-transparently) and displayed. That is, the display control unit of the first image processing device may superimpose the enlarged original image on the enlarged image. For example, as shown in FIG. 6 , the enlarged image and the enlarged original image may be displayed simultaneously (so that both images can be viewed) at the same position. In the example of FIG. 6 , a superimposed image 282 of the enlarged image and the enlarged original image is displayed in the window 241. The enlarged image and the enlarged original image are semi-transparent, allowing the user to view both images simultaneously. This allows the user to more easily compare the enlarged image and the enlarged original image.

[0063] For example, the display example of Fig. 6 may be applied instead of the display example of Fig. 3. That is, when a user or the like moves cursor 231 into entire processed image 211, superimposed image 282 may be displayed in window 241 instead of enlarged processed image 242, as in the example of Fig. 6. Also, when a user or the like clicks the mouse button again in the state of Fig. 3 (a state in which enlarged processed image 242 is displayed), superimposed image 282 may be displayed in window 241 as in Fig. 6. Also, when a user or the like clicks the mouse button again in the state of Fig. 5 (a state in which enlarged original image 262 is displayed), superimposed image 282 may be displayed in window 241 as in Fig. 6.

[0064] <Display of detection level> The detection level may also be displayed in the enlarged image. Here, the "detection level" is an index indicating the likelihood (reliability) of the detected edit. In other words, the higher the detection level, the more accurately the partial edit was detected (the more reliably the partial edit was performed). For example, the display control unit of the first image processing device may display the edit detection area in the enlarged image together with the detection level of the edit detection area. In this way, the user or the like can also grasp the likelihood of the detected partial edit.

[0065] Any method may be used to present the detection level. For example, the detection level may be indicated by a numerical value. Furthermore, as in the example of FIG. 7 , the detection level may be expressed by the manner in which the granularity unit area is filled (e.g., color, pattern, density, transparency, etc.). That is, the display control unit of the first image processing device may present the detection level by the manner in which the processed detection area is filled for each granularity. In the example of FIG. 7 , in the processed detection area 243 of the enlarged processed image 242, the detection level is expressed by the density of the fill for each granularity.

[0066] 8, the detection level may be expressed by the aspect of the border of the granularity unit area (e.g., color, pattern, density, transparency, etc.). In other words, the display control unit of the first image processing device may present the detection level by the aspect of the border of the processing detection area for each granularity. In the example of FIG. 8, in the processing detection area 243 of the enlarged processed image 242, the detection level is expressed by the density of the border of each granularity.

[0067] <Not displaying the manipulation detection area> Note that, as in the example of Fig. 9, the manipulation detection area does not need to be displayed in the enlarged and processed image. That is, the display control unit of the first image processing device may not display the manipulation detection area in the enlarged and processed image. In the example of Fig. 9, masking is omitted in the enlarged and processed image 242, and the manipulation detection area 243 is not displayed.

[0068] <Terminal Device> Fig. 10 is a block diagram showing an example of the main configuration of the terminal device 111. As shown in Fig. 10 , the terminal device 111 (first image processing device) has an input unit 511, an image supply unit 512, a processing detection area information acquisition unit 513, an aligned estimated image acquisition unit 514, a display control unit 515, and a display unit 516.

[0069] The input unit 511 accepts input of, for example, instructions from a user or the like, images (original image and processed image), etc. The input unit 511 may supply the input images (original image and processed image) to the image supply unit 512. The input unit 511 may also supply the input instructions from the user or the like to the image supply unit 512 or the display control unit 515.

[0070] When the image supply unit 512 acquires the images (original image and processed image) supplied from the input unit 511, it supplies the images to the processing confirmation server 112 based on user instructions, etc., and requests confirmation of partial processing of the images.

[0071] In response to the request, the manipulation detection area information acquisition unit 513 acquires the manipulation detection area information supplied from the manipulation confirmation server 112. For example, the manipulation detection area information acquisition unit 513 may acquire manipulation detection area information regarding the manipulation detection area detected using the entire original image and the entire manipulated image supplied by the image supply unit 512. The manipulation detection area information acquisition unit 513 supplies the acquired manipulation detection area information to the display control unit 515.

[0072] The aligned estimated image acquisition unit 514 acquires the aligned estimated image supplied from the processing confirmation server 112 in response to the request. The aligned estimated image acquisition unit 514 supplies the acquired aligned estimated image to the display control unit 515. Note that the enlarged original image may be an image obtained by enlarging a partial region of the acquired aligned estimated image.

[0073] The display control unit 515 executes processing related to control of the display of the display unit 516. For example, the display control unit 515 may acquire processing detection area information supplied from the processing detection area information acquisition unit 513. Furthermore, the display control unit 515 may acquire an aligned estimated image supplied from the aligned estimated image acquisition unit 514. The display control unit 515 may use the acquired processing detection area information to cause the display unit 516 to present the processing detection area (or may cause the display unit 516 to display an entire processed image, an enlarged processed image, or the like). Furthermore, the display control unit 515 may use the acquired aligned estimated image to cause the display unit 516 to present an enlarged original image.

[0074] For example, the display control unit 515 may display an entire processed image, which is an image obtained by processing the entire original image and presents a processing detection area where the processing has been detected, and an enlarged processed image, which is an image obtained by enlarging a partial region of the entire processed image and presents the processing detection area in a manner different from that of the entire processed image. For example, the display control unit 515 may display the processing detection area in a finer granularity in the enlarged processed image than in the entire processed image. Furthermore, the display control unit 515 may display the processing detection area in the entire processed image by masking it, and the non-processing detection area in the enlarged processed image by masking it. Furthermore, the display control unit 515 may further display an enlarged original image, which is an enlarged partial region of the entire original image, at the same position as the enlarged processed image. Note that the enlarged original image may be an image obtained by enlarging a partial region of an estimated registration image, which is an entire original image that has been estimated to be aligned with the entire processed image. Furthermore, the display control unit 515 may switch the display of the enlarged processed image to the display of the enlarged original image. The display control unit 515 may also superimpose and display the enlarged original image on the enlarged processed image. The display control unit 515 may also display the processing detection area together with the detection level of the processing detection area in the enlarged processed image. The display control unit 515 may also display the detection level by filling in the processing detection area according to the granularity. The display control unit 515 may also display the detection level by framing the processing detection area according to the granularity. The display control unit 515 may also not display the processing detection area in the enlarged processed image. The display control unit 515 may also display the entire processed image and the enlarged processed image that display the processing detection area in which partial processing has been detected. The display control unit 515 may also display the enlarged processed image obtained by enlarging a partial area of ​​the entire processed image specified by a user instruction supplied from the input unit 511. Furthermore, the display control unit 515 may display the entire processed image and the enlarged processed image that present the processing detection area corresponding to the processing detection area information supplied from the processing detection area information acquisition unit 513 .

[0075] The display unit 516 displays the display image in accordance with the control of the display control unit 515. For example, the display unit 516 displays the entire processed image and the enlarged processed image in accordance with the control of the display control unit 515.

[0076] By having this configuration, the terminal device 111 (first image processing device) can suppress a reduction in the visibility of the processing detection area, making it easier for users and others to confirm partial processing applied to the image.

[0077] <Processing Confirmation Server> Fig. 11 is a block diagram showing an example of the main configuration of the processing confirmation server 112. As shown in Fig. 11 , the processing confirmation server 112 (second image processing device) has an image acquisition unit 611, a registration unit 612, a registered estimated image supply unit 613, a processing detection unit 614, and a processing detection area information supply unit 615.

[0078] The image acquisition unit 611 executes processing related to image acquisition. For example, the image acquisition unit 611 may acquire images (the entire original image and the entire processed image) supplied from the terminal device 111. The image acquisition unit 611 may supply the acquired images to the alignment unit 612. The image acquisition unit 611 may supply the acquired images to the processing detection unit 614.

[0079] The registration unit 612 performs processing related to image registration. For example, the registration unit 612 may acquire images (the entire original image and the entire processed image) supplied from the image acquisition unit 611. The registration unit 612 may use the acquired images to perform registration estimation, which determines the positional correspondence between the entire original image and the entire processed image. Any method may be used for this registration estimation. For example, the registration unit 612 may extract feature points from the images and perform matching of the feature points. As a result of the registration estimation, the registration unit 612 may generate at least one of registration estimation information related to the registration estimation and a registration estimation image, which is the entire original image reflecting the processing result of the registration estimation.

[0080] For example, the alignment unit 612 may estimate processing such as rotation and resizing performed on the entire original image based on the feature point matching result, and generate an affine matrix indicating the estimated results of the rotation, resizing, etc. Then, the alignment unit 612 may use the affine matrix corresponding to the estimated results of the rotation, resizing, etc. to generate a rotated / resized estimated image, which is an image in which the estimated processing such as rotation and resizing is reflected on the entire original image. In other words, the alignment unit 612 may generate the rotated / resized estimated image as the aligned estimated image.

[0081] Furthermore, the registration unit 612 may compare the distances from the top, bottom, left, and right edges of the entire original image and the entire processed image to feature points, and estimate processing such as cropping (cutting out a part of the image) that has been performed on the entire original image. Then, based on the comparison result (i.e., the estimated result of processing such as cropping), the registration unit 612 may generate an estimated crop image that is an image in which the estimated processing such as cropping has been reflected on the entire original image. In other words, the registration unit 612 may generate the estimated crop image as the registered estimated image.

[0082] In addition, the alignment estimation can be any process that determines the correspondence between the positions of elements within an image between the entire original image and the entire processed image (also called the submitted image), and is not limited to the examples described above.

[0083] For example, the positioning estimation may include a position change operation estimation that estimates the type and degree of a position change operation required to align the position of an image element in the entire original image with the position of an image element in the entire processed image. The position change operation estimation may further include a process for determining whether the degree of the estimated position change operation is within an acceptable range. The position change operation estimation may further include a process for reflecting the estimated position change operation in the entire original image. The processing result of the position change operation estimation may include at least one of a position change operation estimation image and position change operation estimation information.

[0084] The repositioning operation may be an operation on the entire image, involving changing the position of elements within the image. The repositioning operation may also be referred to as a process of applying a repositioning operation matrix. The repositioning operation may also include one or more of cropping, rotating, resizing, stretching, and deskewing.

[0085] For example, this registration estimation may include distortion adjustment estimation that estimates the difference in lens distortion correction between the entire original image and the entire processed image. Furthermore, the distortion adjustment estimation may further include a process of reflecting the estimated difference in lens distortion correction in the entire original image. The result of this distortion adjustment estimation process may include at least one of a distortion adjustment estimated image and distortion adjustment estimation information. Note that in this specification, the process of reflecting the difference in lens distortion correction is also referred to as distortion adjustment. Distortion adjustment can also be considered a process of applying a lens distortion matrix.

[0086] For example, the alignment unit 612 may perform distortion alignment estimation using lens information included in the metadata of the entire processed image and a distortion table corresponding to the lens information. For example, the alignment unit 612 may perform distortion alignment estimation using lens information stored as metadata in an image file that stores the entire original image or the entire processed image and a distortion table corresponding to the lens information. Furthermore, the alignment unit 612 may perform distortion alignment estimation using camera setting values ​​at the time of capture that are included in the metadata of the entire processed image. For example, the alignment unit 612 may perform distortion alignment estimation using camera setting values ​​at the time of capture that are stored as metadata in an image file that stores the entire original image or the entire processed image.

[0087] The alignment estimation information may include position change operation estimation information related to position change operation estimation, which estimates the type and degree of position change operation required to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image. Note that this position change operation may be an operation on the entire image that involves changing the positions of the image elements. Furthermore, the position change operation estimation information may include information indicating the type and degree of position change operation estimated by the position change operation estimation. Furthermore, the position change operation estimation information may include information indicating a determination result as to whether the degree of position change operation estimated by the position change operation estimation is within an acceptable range.

[0088] The alignment estimation information may also include distortion adjustment estimation information related to distortion adjustment estimation that estimates a difference in lens distortion correction between the entire original image and the entire processed image. The distortion adjustment estimation information may also include information indicating the difference in lens distortion correction estimated by the distortion adjustment estimation.

[0089] The alignment estimated image may also include a position change operation estimated image in which a position change operation estimated by position change operation estimation that estimates the type and degree of position change operation necessary to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image is reflected in the entire original image.The alignment estimated image may also include a distortion adjustment estimated image in which a difference in lens distortion correction estimated by distortion adjustment estimation that estimates the difference in lens distortion correction between the entire original image and the entire processed image is reflected in the entire original image.

[0090] Furthermore, in the registration estimation, the registration unit 612 may skip the execution of a predetermined processing step among the processes performed to generate the entire processed image from the entire original image. The predetermined processing step may be a partial processing step for a partial region of the entire original image. For example, the predetermined processing step may be a color correction step. For example, the color correction step may include one or more of white balance, color balance, hue, saturation, luminance, brightness, and color space conversion.

[0091] The alignment unit 612 may supply at least one of the generated alignment estimation information and the aligned estimation image (for example, the rotated / resized estimated image or the cropped estimated image) to the aligned estimation image supply unit 613. The alignment unit 612 may supply at least one of the alignment estimation information and the aligned estimation image (for example, the rotated / resized estimated image or the cropped estimated image) to the processing detection unit 614.

[0092] The registration estimated image supply unit 613 executes processing related to the supply of the registration estimated image. For example, the registration estimated image supply unit 613 may acquire at least one of the registration estimated information and the registration estimated image supplied from the registration unit 612. The registration estimated image supply unit 613 may supply the acquired registration estimated information and / or the registration estimated image as a result of the registration estimation to the terminal device 111 that has requested the processing confirmation. In other words, the registration estimated image supply unit 613 can also be said to be a registration estimate supply unit.

[0093] The modification detection unit 614 executes processing related to the detection of partial modification. For example, the modification detection unit 614 may acquire images (the entire original image and the entire modified image) supplied from the image acquisition unit 611. The modification detection unit 614 may acquire an aligned estimated image (e.g., a rotated / resized estimated image or a cropped estimated image) supplied from the alignment unit 612. The modification detection unit 614 may detect partial modification applied to the entire original image using the acquired images (the entire original image (or the aligned estimated image) and the entire modified image). The modification detection unit 614 may further detect the partial modification using alignment estimation information. Any method may be used to detect this partial modification. For example, the modification detection unit 614 may compare the entire edited image and the aligned estimated image for each partial region to detect partial modification. Alternatively, the modification detection unit 614 may use the entire original image for the comparison.

[0094] Note that this method of detecting partial manipulation may be a method that distinguishes partial manipulation from partial changes in color or brightness. In other words, in detecting partial manipulation, it is possible to avoid detecting manipulation that merely brightens the sky or changes the color, for example. For example, when detecting manipulation using an algorithm that measures image similarity, such as perceptual hashing, there is a possibility that partial changes in color or brightness, such as those described above, will also be detected. Therefore, the manipulation detection unit 614 may detect partial manipulation without using such an algorithm (by applying a different algorithm). In this way, the manipulation detection unit 614 can detect partial manipulation separately from partial changes in color or brightness. The manipulation detection unit 614 may generate manipulation detection area information regarding the manipulation detection area in which the partial manipulation was detected and supply the manipulation detection area information to the manipulation detection area information supply unit 615.

[0095] The manipulation detection area information supply unit 615 executes processing related to the supply of manipulation detection area information. For example, the manipulation detection area information supply unit 615 may acquire the manipulation detection area information supplied from the manipulation detection unit 614. The manipulation detection area information supply unit 615 may supply the acquired manipulation detection area information to the terminal device 111 that has requested the manipulation confirmation.

[0096] By having such a configuration, the processing confirmation server 112 (second image processing device) can suppress a reduction in the visibility of the processing detection area, making it easier for users and others to check the partial processing applied to the image.

[0097] <Flow of Comparison Display Process> Next, an example of the flow of the comparison display process executed by the terminal device 111 will be described with reference to the flowchart of FIG.

[0098] When the comparison display process starts, the input unit 511 acquires an original image and an edited image in step S501.

[0099] In step S502, the image supply unit 512 supplies the original image and the processed image to the processing confirmation server 112.

[0100] In step S503, the manipulation detection area information acquisition unit 513 acquires the manipulation detection area information supplied from the manipulation confirmation server 112. Furthermore, the aligned estimated image acquisition unit 514 acquires the aligned estimated image supplied from the manipulation confirmation server 112.

[0101] In step S504, the display control unit 515 displays the entire processed image presenting the processing detection area and the entire original image. The display unit 516 displays the entire processed image presenting the processing detection area and the entire original image in accordance with the control.

[0102] In step S505, the display control unit 515 executes an enlargement display process to enlarge and display the image on the display unit 516 in response to a user instruction. For example, by executing this enlargement display process, the display control unit 515 may display an entire processed image, which is an image in which partial processing has been applied to the entire original image and which presents a processing detection area in which the partial processing has been detected, and an enlarged processed image, which is an image in which a partial area of ​​the entire processed image has been enlarged and which presents the processing detection area in a manner different from that of the entire processed image.

[0103] In step S506, the display control unit 515 determines whether or not to end the comparison display process. If it is determined not to end, the process returns to step S505, and the subsequent processes are executed. Also, if it is determined in step S506 that the comparison display process is to end, the comparison display process ends.

[0104] <Flow of Enlarged Display Processing> Next, an example of the flow of the enlarged display processing executed in step S505 of FIG. 12 will be described with reference to the flowchart of FIG.

[0105] When the enlarged display process is started, the display control unit 515 determines whether the cursor has moved into the entire processed image in step S531. If it is determined that the cursor has moved, the process proceeds to step S532.

[0106] In step S532, the display control unit 515 displays an enlarged edited image of a predetermined range around the cursor next to the entire edited image, and presents the edit detection area in a manner different from that used for the entire edited image.

[0107] In step S533, the display control unit 515 determines whether or not the user has clicked the mouse button. If it is determined that the mouse button has been clicked, the process proceeds to step S534.

[0108] In step S534, the display control unit 515 switches the enlarged processed image to the enlarged original image.

[0109] In step S535, the display control unit 515 determines whether or not the user etc. has clicked the mouse button again. If it is determined that the mouse button has been clicked again, the process proceeds to step S536.

[0110] In step S536, the display control unit 515 switches the enlarged original image to the enlarged image. When the processing of step S536 ends, the processing proceeds to step S537. Also, if it is determined in step S535 that a second click has not been performed, the processing proceeds to step S537. Also, if it is determined in step S533 that a second click has not been performed, the processing proceeds to step S537.

[0111] In step S537, the display control unit 515 determines whether the cursor has moved outside the entire processed image. If it is determined that the cursor has not moved outside the entire processed image, the process returns to step S533, and the subsequent processes are executed. Also, if it is determined in step S537 that the cursor has moved outside the entire processed image, the process proceeds to step S538.

[0112] In step S538, the display control unit 515 ends the display of the enlarged image. When the processing of step S538 ends, the enlarged display processing ends, and the processing returns to Fig. 12. Also, if it is determined in step S531 that the cursor has not moved into the entire processed image, the enlarged display processing ends, and the processing returns to Fig. 12.

[0113] By performing each process as described above, the terminal device 111 (first image processing device) can suppress a reduction in the visibility of the processing detection area, making it easier for users and others to confirm the partial processing applied to the image.

[0114] <Flow of Processing Confirmation Process> Next, an example of the flow of processing confirmation process executed by the processing confirmation server 112 will be described with reference to the flowchart of FIG.

[0115] When the processing confirmation process is started, the image acquisition unit 611 acquires the entire original image and the entire processed image supplied from the terminal device 111 in step S561.

[0116] In step S562, the registration unit 612 extracts feature points of the entire original image and the entire processed image, and performs feature point matching.

[0117] In step S563, the registration unit 612 estimates an affine matrix based on the feature point matching result, and generates a rotated and resized estimated image from the entire original image using the affine matrix.

[0118] In step S564, the positioning unit 612 compares the distances from the top, bottom, left, and right edges of the entire original image and the entire processed image to the feature points, and generates an estimated crop image from the entire original image based on the comparison results.

[0119] In step S565, the modification detection unit 614 compares the entire modified image with the aligned estimated image for each partial region to detect modification. For example, the modification detection unit 614 may use the entire original image (or the aligned estimated image) and the entire modified image obtained by performing partial modification on the entire original image to detect partial modification performed on the entire original image.

[0120] In step S566, the manipulation detection area information supply unit 615 supplies manipulation detection area information indicating the manipulation detection result (i.e., manipulation detection area information related to the manipulation detection area in which partial manipulation was detected) to the terminal device 111 that has requested the manipulation confirmation. In addition, the aligned estimated image supply unit 613 supplies the aligned estimated image to the terminal device 111 that has requested the manipulation confirmation.

[0121] When the process of step S566 is completed, the processing confirmation process ends.

[0122] By performing each process as described above, the processing confirmation server 112 (second image processing device) can suppress a reduction in the visibility of the processing detection area, making it easier for users to check the partial processing applied to the image.

[0123] <4. Alignment of original image and processed image> Regardless of the method of presenting the processing detection area, the original image and the processed image may be aligned, and the aligned estimated image may be used to display an enlarged original image corresponding to the enlarged processed image in the same position as the enlarged processed image.

[0124] For example, the first image processing device may include an alignment estimated image acquisition unit that acquires an alignment estimated image, which is an entire original image that has been estimated to be aligned with an entire processed image obtained by performing partial processing on the entire original image, a display control unit that displays an enlarged processed image obtained by enlarging a partial region of the entire processed image and an enlarged original image obtained by enlarging a partial region of the acquired alignment estimated image, at the same positions as each other, and a display unit that displays the enlarged processed image and the enlarged original image in accordance with control of the display control unit.For example, a first image processing method executed by the first image processing device may include acquiring an alignment estimated image, which is an entire original image that has been estimated to be aligned with an entire processed image obtained by performing partial processing on the entire original image, controlling the enlarged processed image obtained by enlarging a partial region of the entire processed image and the enlarged original image obtained by enlarging a partial region of the acquired alignment estimated image to be displayed in the same positions as each other, and displaying the enlarged processed image and the enlarged original image in accordance with the control. Furthermore, the first program may cause a computer to execute processing including acquiring an aligned estimated image, which is an overall original image that has been aligned and estimated for an overall processed image in which partial processing has been applied to the overall original image; controlling the display of an enlarged processed image obtained by enlarging a partial area of ​​the overall processed image and an enlarged original image obtained by enlarging a partial area of ​​the acquired aligned estimated image in the same positions; and displaying the enlarged processed image and the enlarged original image in accordance with the control.

[0125] By doing this, the first image processing device, the first image processing device executing the first image processing method, or the computer executing the first program can suppress a reduction in the visibility of the processing detection area, making it easier for users and others to confirm partial processing applied to the image.

[0126] The display of such an enlarged original image may be switched with the display of the enlarged image. That is, either the enlarged image or the enlarged original image may be displayed at the same position. For example, in the first image processing device, a display control unit may switch the display of the enlarged image to the display of the enlarged original image. This switching may be performed, for example, based on an instruction from a user or the like. For example, when the enlarged image 242 is displayed as shown in FIG. 3, as shown in FIG. 5, if the user clicks the mouse button, an enlarged original image 262 may be displayed in the window 241 instead of the enlarged image 242. Furthermore, if the user clicks the mouse button again in the state of FIG. 5, the state may return to the state of FIG. 3 (i.e., the enlarged image 242 is displayed in the window 241).

[0127] Furthermore, the enlarged image and the enlarged original image may be superimposed (semi-transparently) and displayed. That is, the display control unit of the first image processing device may superimpose the enlarged original image on the enlarged image. For example, as shown in FIG. 6 , the enlarged image and the enlarged original image may be displayed simultaneously (so that both images can be viewed) at the same position. In the example of FIG. 6 , a superimposed image 282 of the enlarged image and the enlarged original image is displayed in the window 241. The enlarged image and the enlarged original image are semi-transparent, allowing the user to view both images simultaneously. This allows the user to more easily compare the enlarged image and the enlarged original image. This allows the user to more easily check the partial processing applied to the image.

[0128] For example, the display example of Fig. 6 may be applied instead of the display example of Fig. 3. That is, when a user or the like moves cursor 231 into entire processed image 211, superimposed image 282 may be displayed in window 241 instead of enlarged processed image 242, as in the example of Fig. 6. Also, when a user or the like clicks the mouse button again in the state of Fig. 3 (a state in which enlarged processed image 242 is displayed), superimposed image 282 may be displayed in window 241 as in Fig. 6. Also, when a user or the like clicks the mouse button again in the state of Fig. 5 (a state in which enlarged original image 262 is displayed), superimposed image 282 may be displayed in window 241 as in Fig. 6.

[0129] The first image processing device may further include an image supply unit that supplies the entire original image and the entire processed image, and the registered estimated image acquisition unit may acquire the registered estimated image derived using the supplied entire original image and the entire processed image.

[0130] The first image processing device may further include a processing detection unit that detects partial processing applied to the entire original image using the entire original image and the entire processed image, and the display control unit may display a processing detection area in which the partial processing has been detected in the enlarged processed image and the enlarged original image that are displayed.

[0131] For example, the second image processing device may include a registration estimation unit that performs registration estimation to determine a positional correspondence between an entire original image and an entire processed image obtained by performing partial processing on the entire original image, and a supply unit that supplies, as a result of the registration estimation, at least one of registration estimation information related to the registration estimation and an estimated registration image, which is the entire original image in which the registration estimation processing result is reflected.For example, the second image processing method performed by the second image processing device may include performing registration estimation to determine a positional correspondence between an entire original image and an entire processed image obtained by performing partial processing on the entire original image, and supplying, as a result of the registration estimation, at least one of registration estimation information related to the registration estimation and an estimated registration image, which is the entire original image in which the registration estimation processing result is reflected. Furthermore, the second program may cause the computer to execute processing including performing alignment estimation to determine the positional correspondence between an entire original image and an entire processed image obtained by performing partial processing on the entire original image, and supplying, as a result of the alignment estimation, at least one of alignment estimation information regarding the alignment estimation and an alignment estimation image, which is the entire original image in which the processing result of the alignment estimation is reflected.

[0132] By providing at least one of the alignment estimation information and the alignment estimation image in this manner, the second image processing device, the second image processing device that executes the second image processing method, or the computer that executes the second program can make it easier for users and others to check the partial processing that has been applied to the image.

[0133] For example, the alignment estimation may include a repositioning operation estimation that estimates the type and degree of repositioning operation required to align the positions of image elements in the overall original image with the positions of image elements in the overall processed image. The repositioning operation may also be an operation on the overall image that involves changing the positions of image elements.

[0134] The estimation of the position change operation may further include a process of determining whether the degree of the estimated position change operation is within an acceptable range, and a process of reflecting the estimated position change operation in the entire original image.

[0135] For example, the repositioning operation may include one or more of cropping, rotating, resizing, stretching, and deskewing.

[0136] The above-described registration estimation may also include distortion alignment estimation that estimates the difference in lens distortion correction between the entire original image and the entire processed image.

[0137] The distortion matching estimation may further include a process of reflecting the difference in the estimated lens distortion correction to the entire original image.

[0138] In the second image processing device, the alignment estimation unit may perform distortion alignment estimation using lens information included in the metadata of the overall processed image and a distortion table corresponding to the lens information. Also, the alignment estimation unit may perform distortion alignment estimation using camera settings at the time of shooting included in the metadata of the overall processed image.

[0139] Furthermore, the above-described alignment estimation information may include position change operation estimation information related to position change operation estimation, which estimates the type and degree of position change operation required to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image. Note that the position change operation may be an operation on the entire image that involves changing the position of an image element. Furthermore, the position change operation estimation information may include information indicating the type and degree of position change operation estimated by the position change operation estimation. Furthermore, the position change operation estimation information may include information indicating a determination result as to whether the degree of the position change operation estimated by the position change operation estimation is within an acceptable range. Furthermore, the alignment estimation information may include distortion adjustment estimation information related to distortion adjustment estimation, which estimates the difference in lens distortion correction between the entire original image and the entire processed image. Furthermore, the distortion adjustment estimation information may include information indicating the difference in lens distortion correction estimated by the distortion adjustment estimation.

[0140] Furthermore, the above-mentioned alignment estimated image may include a position change operation estimated image in which a position change operation estimated by a position change operation estimation that estimates the type and degree of a position change operation necessary to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image is reflected in the entire original image. Furthermore, the alignment estimated image may include a distortion adjustment estimated image in which a difference in lens distortion correction estimated by a distortion adjustment estimation that estimates the difference in lens distortion correction between the entire original image and the entire processed image is reflected in the entire original image.

[0141] Furthermore, the alignment unit may skip the execution of a predetermined process among the processes performed to generate the entire processed image from the entire original image during the alignment estimation. The predetermined process may be, for example, partial processing of a partial region of the entire original image. The predetermined process may also be, for example, color correction. The color correction may include, for example, one or more processes such as white balance adjustment, color balance adjustment, hue adjustment, saturation adjustment, luminance adjustment, brightness adjustment, or color space conversion.

[0142] The second image processing device may further include an image acquisition unit that acquires the entire original image and the entire processed image, and the registration unit may perform registration estimation using the acquired entire original image and the entire processed image.

[0143] <Registration Estimation System> Fig. 15 is a diagram showing an example of the configuration of a registration estimation system, which is one aspect of an image processing system to which the present technology is applied in such a case. The registration estimation system 700 shown in Fig. 15 is a system that performs registration estimation.

[0144] As shown in Fig. 15, the registration estimation system 700 includes a terminal device 711 and a registration estimation server 712. The terminal device 711 and the registration estimation server 712 are communicatively connected to each other via a network 710, and can exchange information with each other via this communication. Note that this communication may be wired communication, wireless communication, or both. The network 710 is a communication network similar to the network 110 in Fig. 1.

[0145] 1 , has basically the same configuration as the terminal device 111, and executes basically the same processing. For example, the terminal device 711 supplies the selected original image and processed image to the alignment estimation server 712 via the network 710. Note that the terminal device 711 may also supply image files storing the original image and processed image to the alignment estimation server 712. Furthermore, the terminal device 711 acquires at least one of the alignment estimation information and the alignment estimation image supplied from the alignment estimation server 712, and generates and displays a display image based on the information.

[0146] The registration estimation server 712 executes processing related to registration estimation. For example, the registration estimation server 712 acquires an entire original image and an entire processed image supplied from the client terminal device 711. The registration estimation server 712 executes registration estimation using these images, generates at least one of registration estimation information and a registration estimation image, and supplies the generated information and image to the terminal device 111.

[0147] The registration estimation system 700 may include an authenticity determination server 713. The authenticity determination server 713 may be communicably connected to the terminal device 711 and the registration estimation server 712 via the network 710. The authenticity determination server 713 may determine the authenticity of the original image and the processed image. For example, the authenticity determination server 713 may acquire these images from the terminal device 711 and determine their authenticity. For example, the registration estimation server 712 may perform registration estimation using an image whose authenticity has been determined by the authenticity determination server 713. The authenticity determination server 713 may also determine authenticity using a registration estimation image generated by the registration estimation server 712. The authenticity determination server 713 may determine authenticity using any method.

[0148] <Registration Estimation Server> Fig. 16 is a block diagram showing an example of the main configuration of the registration estimation server 712. As shown in Fig. 16 , the registration estimation server 712 (second image processing device) has an image acquisition unit 721, a registration estimation unit 722, and a supply unit 723.

[0149] The image acquisition unit 721 executes processing related to image acquisition. For example, the image acquisition unit 721 may acquire images (an entire original image and an entire processed image) supplied from the terminal device 711. The image acquisition unit 721 may acquire an image file storing the entire original image and the entire processed image supplied from the terminal device 711. The image acquisition unit 721 may supply the acquired images to the registration estimation unit 722.

[0150] The registration estimation unit 722 performs processing related to registration estimation. For example, the registration estimation unit 722 may acquire images (the entire original image and the entire processed image) supplied from the image acquisition unit 721. The registration estimation unit 722 may use the acquired images to perform registration estimation, which determines the positional correspondence between the entire original image and the entire processed image. Any method may be used for this registration estimation. For example, the registration estimation unit 722 may extract feature points from the images and perform matching of the feature points. As a result of the registration estimation, the registration estimation unit 722 may generate at least one of registration estimation information related to the registration estimation and a registration estimation image, which is the entire original image reflecting the processing result of the registration estimation.

[0151] 16 , the registration estimation unit 722 may include at least one of a position change operation estimation unit 731 and a distortion adjustment estimation unit 732. The position change operation estimation unit 731 may execute processing related to the estimation of the position change operation. The distortion adjustment estimation unit 732 may execute processing related to the estimation of the distortion adjustment.

[0152] For example, this alignment estimation may include a position change operation estimation that estimates the type and degree of a position change operation required to align the position of an image element in the entire original image with the position of an image element in the entire processed image. The position change operation may also be an operation on the entire image that involves changing the position of an image element. The position change operation estimation may also include a process of determining whether the degree of the estimated position change operation is within an acceptable range. The position change operation estimation may also include a process of reflecting the estimated position change operation on the entire original image. For example, the position change operation may include one or more of cropping, rotating, resizing, stretching, and deskewing. For example, the position change operation estimation unit 731 may perform such a position change operation estimation.

[0153] Furthermore, the above-described alignment estimation may include distortion alignment estimation that estimates a difference in lens distortion correction between the entire original image and the entire processed image. Furthermore, the distortion alignment estimation may further include processing that reflects the estimated difference in lens distortion correction in the entire original image. For example, the distortion alignment estimation unit 732 may perform such distortion alignment estimation.

[0154] The distortion matching estimation unit 732 may perform distortion matching estimation using lens information included in the metadata of the overall processed image and a distortion table corresponding to the lens information. Alternatively, the distortion matching estimation unit 732 may perform distortion matching estimation using camera settings at the time of shooting included in the metadata of the overall processed image.

[0155] Furthermore, the above-described alignment estimation information may include position change operation estimation information related to position change operation estimation, which estimates the type and degree of position change operation required to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image. Note that the position change operation may be an operation on the entire image that involves changing the position of an image element. Furthermore, the position change operation estimation information may include information indicating the type and degree of position change operation estimated by the position change operation estimation. Furthermore, the position change operation estimation information may include information indicating a determination result as to whether the degree of the position change operation estimated by the position change operation estimation is within an acceptable range. For example, the position change operation estimation unit 731 may generate such position change operation estimation information. Furthermore, the alignment estimation information may include distortion adjustment estimation information related to distortion adjustment estimation, which estimates the difference in lens distortion correction between the entire original image and the entire processed image. Furthermore, the distortion adjustment estimation information may include information indicating the difference in lens distortion correction estimated by the distortion adjustment estimation. For example, the distortion adjustment estimation unit 732 may generate such distortion adjustment estimation information.

[0156] Furthermore, the above-described alignment estimated image may include a position change operation estimated image in which a position change operation estimated by a position change operation estimation that estimates the type and degree of a position change operation required to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image is reflected in the entire original image. For example, the position change operation estimation unit 731 may generate such a position change operation estimated image. Furthermore, the alignment estimated image may include a distortion adjustment estimated image in which a difference in lens distortion correction estimated by a distortion adjustment estimation that estimates the difference in lens distortion correction between the entire original image and the entire processed image is reflected in the entire original image. For example, the distortion adjustment estimation unit 732 may generate such a distortion adjustment estimated image.

[0157] Furthermore, the registration estimation unit 722 may skip the execution of a predetermined processing step among the processing steps performed to generate the entire processed image from the entire original image during the registration estimation. The predetermined processing step may be, for example, partial processing of a partial region of the entire original image. The predetermined processing step may also be, for example, color correction. The color correction step may include, for example, one or more of white balance adjustment, color balance adjustment, hue adjustment, saturation adjustment, luminance adjustment, brightness adjustment, or color space conversion.

[0158] The alignment estimation unit 722 may supply at least one of the generated alignment estimation information and the generated alignment estimation image to the supply unit 723. For example, the position change operation estimation unit 731 may supply at least one of the generated position change operation estimation information and the generated position change operation estimation image to the supply unit 723. Furthermore, the distortion alignment estimation unit 732 may supply at least one of the generated distortion alignment estimation information and the generated distortion alignment estimation image to the supply unit 723.

[0159] The supply unit 723 executes processing related to the supply of information related to alignment estimation. For example, the supply unit 723 may acquire at least one of alignment estimation information and alignment estimation image supplied from the alignment estimation unit 722. The supply unit 723 may supply the acquired at least one of the alignment estimation information and alignment estimation image as a result of alignment estimation to the terminal device 711 that has requested the alignment estimation. For example, the supply unit 723 may supply at least one of the position change operation estimation information and the position change operation estimation image supplied from the position change operation estimation unit 731 to the terminal device 711. Furthermore, the supply unit 723 may supply at least one of the distortion alignment estimation information and the distortion alignment estimation image supplied from the distortion alignment estimation unit 732 to the terminal device 711.

[0160] With this configuration, the registration estimation server 712 (second image processing device) can enable the user to more easily check partial processing applied to the image.

[0161] <Flow of Registration Estimation Processing> Next, an example of the flow of registration estimation processing executed by the registration estimation server 712 will be described with reference to the flowcharts of FIGS. 17 to 19. FIG.

[0162] When the registration estimation process is started, the registration estimation unit 722 sets an allowable range for a position change operation in step S701.

[0163] In step S702, the image acquisition unit 721 acquires the original image and the processed image.

[0164] In step S703, the registration estimation unit 722 extracts feature points and performs feature point matching.

[0165] In step S704, the registration estimation unit 722 removes outliers from the feature point matching. After the process of step S704 is completed, the process proceeds to FIGS.

[0166] 18, the position modification operation estimation unit 731 generates a position modification operation matrix based on the positional relationship between feature points in the original image and the processed image. After the process of step S711 ends, the process proceeds to steps S712 and S715.

[0167] In step S712, the position change operation estimation unit 731 estimates the type and degree of the executed position change operation based on the position change operation matrix.

[0168] In step S713, the position change operation estimation unit 731 determines whether the estimated degree of the position change operation is within the allowable range set in step S701.

[0169] In step S714, the supply unit 723 supplies the position change operation estimation information relating to the position change operation estimation described above to the terminal device 711. When the processing of step S714 ends, the registration estimation processing ends.

[0170] In addition, in step S715, the position change operation estimation unit 731 applies the position change operation matrix generated in step S711 to the original image to generate a position change operation estimated image.

[0171] In step S716, the supply unit 723 supplies the position change operation estimated image generated as described above to the terminal device 711. When the processing of step S716 ends, the position adjustment estimation processing ends.

[0172] 18, the distortion matching estimation unit 732 generates a lens distortion matrix based on the positional relationship between feature points in the original image and the processed image. After the process of step S721 ends, the process proceeds to steps S722 and S724.

[0173] In step S722, the distortion matching estimation unit 732 estimates distortion matching based on the lens distortion matrix.

[0174] In step S723, the supply unit 723 supplies distortion alignment estimation information relating to the distortion alignment estimation described above to the terminal device 711. When the process of step S723 ends, the registration estimation process ends.

[0175] In step S724, the distortion matching estimation unit 732 applies the lens distortion matrix generated in step S721 to the original image to generate a distortion matching estimated image.

[0176] In step S725, the supply unit 723 supplies the distortion-aligned estimated image generated as described above to the terminal device 711. When the processing of step S725 ends, the alignment estimation processing ends.

[0177] Note that, if position change operation estimated information is not generated (supplied) in this alignment estimation process, the processes of steps S712 to S714 in Fig. 18 may be omitted. Also, if a position change operation estimated image is not generated (supplied) in this alignment estimation process, the processes of steps S715 and S716 in Fig. 18 may be omitted. Also, if distortion adjustment estimated information is not generated (supplied) in this alignment estimation process, the processes of steps S722 and S723 in Fig. 19 may be omitted. Also, if a distortion adjustment estimated image is not generated (supplied) in this alignment estimation process, the processes of steps S724 and S725 in Fig. 19 may be omitted.

[0178] By performing each process as described above, the registration estimation server 712 (second image processing device) can enable the user to more easily check partial processing applied to the image.

[0179] <Terminal Device Case 1> In this case, the terminal device 711 may have the same configuration as the terminal device 111 described with reference to FIG.

[0180] For example, the terminal device 711 may be equipped with an alignment estimation image acquisition unit 514 that acquires an alignment estimation image, which is an entire original image that has been estimated to be aligned with an entire processed image that has been subjected to partial processing on the entire original image; a display control unit 515 that displays an enlarged processed image that is an enlarged partial area of ​​the entire processed image and an enlarged original image that is an enlarged partial area of ​​the acquired alignment estimation image, at the same positions as each other; and a display unit 516 that displays the enlarged processed image and the enlarged original image according to the control of the display control unit.

[0181] For example, the display control unit 515 may switch the display of the enlarged processed image to the display of the enlarged original image. The display control unit 515 may also superimpose the enlarged original image on the enlarged processed image. The terminal device 711 may further include an image supply unit 512 that supplies the entire original image and the entire processed image. The registered estimated image acquisition unit 514 may then acquire an registered estimated image derived using the supplied entire original image and the entire processed image. The terminal device 711 may also include a processing detection unit 614 that detects partial processing applied to the entire original image using the entire original image and the entire processed image. The display control unit 515 may then present a processing detection area in which the partial processing has been detected in the enlarged processed image and the enlarged original image that are displayed.

[0182] With this configuration, the terminal device 711 can allow the user to more easily check the partial processing applied to the image.

[0183] In this case, the terminal device 711 may also execute the comparison display process described with reference to the flowchart of FIG.

[0184] <Terminal Device Case 2> The terminal device 711 does not have to display an enlarged image. For example, the terminal device 711 may display the alignment estimation information and the aligned estimated image supplied from the alignment estimation server 712.

[0185] For example, the first image processing device may include a registration estimation information acquisition unit that acquires registration estimation information related to registration estimation for determining a positional correspondence between an entire original image and an entire processed image obtained by partially processing the entire original image, a registration estimation image acquisition unit that acquires a registration estimation image that is the entire original image reflecting the registration estimation processing result, and a display control unit that displays the acquired registration estimation information and the acquired registration estimation image on a display unit.Furthermore, a first image processing method executed by the first image processing device may include acquiring registration estimation information related to registration estimation for determining a positional correspondence between an entire original image and an entire processed image obtained by partially processing the entire original image, acquiring a registration estimation image that is the entire original image reflecting the registration estimation processing result, and displaying the acquired registration estimation information and the acquired registration estimation image on a display unit. The first program also causes the computer to execute processing including acquiring alignment estimation information relating to alignment estimation that determines the positional correspondence between an entire original image and an entire processed image that has been subjected to partial processing on the entire original image, acquiring an alignment estimation image that is the entire original image that reflects the processing results of the alignment estimation, and displaying the acquired alignment estimation information and the acquired alignment estimation image on a display unit.

[0186] By doing this, the first image processing device, the first image processing device that executes the first image processing method, or the computer that executes the first program can make it easier for users and others to check the partial processing that has been applied to the image.

[0187] For example, the alignment estimation information may include position change operation estimation information related to a position change operation estimation that estimates the type and degree of a position change operation required to align the position of an image element in the entire original image with the position of an image element in the entire processed image. The position change operation may be an operation on the entire image that involves changing the position of an image element. The position change operation estimation information may also include information indicating the type and degree of a position change operation estimated by the position change operation estimation. The position change operation estimation information may also include information indicating a determination result as to whether the degree of the position change operation estimated by the position change operation estimation is within an acceptable range. The alignment estimation information may also include distortion adjustment estimation information related to a distortion adjustment estimation that estimates a difference in lens distortion correction between the entire original image and the entire processed image. The distortion adjustment estimation information may include information indicating the difference in lens distortion correction estimated by the distortion adjustment estimation.

[0188] For example, the alignment estimated image may include a position change operation estimated image in which a position change operation estimated by a position change operation estimation that estimates the type and degree of a position change operation required to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image is reflected in the entire original image.Furthermore, the alignment estimated image may include a distortion adjustment estimated image in which a difference in lens distortion correction estimated by a distortion adjustment estimation that estimates the difference in lens distortion correction between the entire original image and the entire processed image is reflected in the entire original image.

[0189] <Configuration of Terminal Device> Fig. 20 is a block diagram showing an example of the main configuration of the terminal device 711 in this case. As shown in Fig. 20 , the terminal device 711 (first image processing device) in this case has an input unit 751, an image supply unit 752, an alignment estimated information acquisition unit 753, an alignment estimated image acquisition unit 754, a display control unit 755, and a display unit 756. In addition, the alignment estimated image acquisition unit 754 has at least one of a position change operation estimated image acquisition unit 771 and a distortion alignment estimated image acquisition unit 772.

[0190] The input unit 751 accepts, for example, input of instructions from a user or the like, images (original image and processed image), etc. The input unit 751 may supply the input images (original image and processed image) to the image supply unit 752. The input unit 751 may also supply the input instructions from the user or the like to the image supply unit 752 or the display control unit 755.

[0191] When the image supply unit 752 acquires the images (original image and processed image) supplied from the input unit 751, it supplies the images to the registration estimation server 712 based on user instructions or the like.

[0192] The alignment estimation information acquisition unit 753 executes processing related to the acquisition of alignment estimation information. For example, the alignment estimation information acquisition unit 753 acquires alignment estimation information supplied from the alignment estimation server 712 in response to a request. This alignment estimation information is information related to alignment estimation. Alignment estimation is processing for determining a positional correspondence between an entire original image and an entire processed image obtained by performing partial processing on the entire original image. For example, the alignment estimation information acquisition unit 753 includes at least one of a position change operation estimation information acquisition unit 761 and a distortion alignment estimation information acquisition unit 762.

[0193] The position change operation estimation information acquisition unit 761 may acquire position change operation estimation information related to the position change operation estimation. The position change operation estimation may be a process of estimating the type and degree of a position change operation required to align the positions of image elements in the entire original image with the positions of image elements in the entire processed image. The position change operation may be an operation on the entire image that involves changing the positions of the image elements. Note that this position change operation estimation information may include information indicating the type and degree of the position change operation estimated by the position change operation estimation. Furthermore, this position change operation estimation information may include information indicating a determination result as to whether the degree of the position change operation estimated by the position change operation estimation is within an acceptable range.

[0194] The distortion matching estimation information acquisition unit 762 may acquire distortion matching estimation information related to distortion matching estimation. The distortion matching estimation may be a process of estimating the difference in lens distortion correction between the entire original image and the entire processed image. Note that this distortion matching estimation information may include information indicating the difference in lens distortion correction estimated by the distortion matching estimation.

[0195] The alignment estimation information acquisition unit 753 may supply the acquired alignment estimation information to the display control unit 755. For example, the position change operation estimation information acquisition unit 761 may supply the acquired position change operation estimation information to the display control unit 755. Furthermore, the distortion alignment estimation information acquisition unit 762 may supply the acquired distortion alignment estimation information to the display control unit 755.

[0196] The registration estimated image acquisition unit 754 executes processing related to the acquisition of a registration estimated image. For example, the registration estimated image acquisition unit 754 acquires a registration estimated image provided from the registration estimation server 712 in response to a request. This registration estimated image is an entire original image reflecting the registration estimation processing result. For example, the registration estimated image acquisition unit 754 includes at least one of a position change operation estimated image acquisition unit 771 and a distortion alignment estimated image acquisition unit 772.

[0197] The position change operation estimated image acquisition unit 771 may acquire a position change operation estimated image in which the position change operation estimated by the position change operation estimation is reflected in the entire original image. The distortion adjustment estimated image acquisition unit 772 may acquire a distortion adjustment estimated image in which the difference in lens distortion correction estimated by the distortion adjustment estimation is reflected in the entire original image.

[0198] The aligned estimated image acquisition unit 754 may supply the acquired aligned estimated image to the display control unit 755. For example, the position change operation estimated image acquisition unit 771 may supply the acquired position change operation estimated image to the display control unit 755. Furthermore, the distortion aligned estimated image acquisition unit 772 may supply the acquired distortion aligned estimated image to the display control unit 755.

[0199] The display control unit 755 executes processing related to control of the display of the display unit 756. For example, the display control unit 755 acquires alignment estimation information supplied from the alignment estimation information acquisition unit 753. The display control unit 755 also acquires an alignment estimation image supplied from the alignment estimation image acquisition unit 754. The display control unit 755 also supplies the acquired alignment estimation information and alignment estimation image to the display unit 756, causing it to be displayed.

[0200] The display unit 756 displays the display image under the control of the display control unit 755. For example, the display unit 756 displays the registration estimation information and the registration estimation image under the control of the display control unit 755.

[0201] With this configuration, the terminal device 711 (first image processing device) can allow the user to more easily check the partial processing applied to the image.

[0202] <Flow of Comparison Display Process> Next, an example of the flow of the comparison display process executed by the terminal device 711 will be described with reference to the flowchart of FIG.

[0203] When the comparison display process is started, the input unit 751 acquires an original image and an edited image in step S751.

[0204] In step S752, the image supply unit 752 supplies the original image and the processed image.

[0205] In step S753, the registration estimation information acquisition unit 753 acquires registration estimation information.

[0206] In step S754, the registered estimated image acquisition unit 754 acquires a registered estimated image.

[0207] In step S755, the display control unit 755 causes the display unit 756 to display the entire processed image, the aligned estimated image, and the aligned estimated information.

[0208] When the process of step S755 ends, the comparison display process ends.

[0209] By executing each process as described above, the terminal device 711 (first image processing device) can enable the user to more easily check the partial processing applied to the image.

[0210] <Display Example> For example, the display control unit 755 may cause the display unit 756 to display a display image configured as shown in FIG. 22 . In the example of FIG. 22 , a display image 810 displays an entire processed image 811 and an estimated alignment image 812 side by side. Also, below these images, an estimated alignment information display area 813 (within the dotted-line frame in the figure) is provided, in which estimated alignment information is displayed. In the example of FIG. 22 , the estimated alignment information displays the type and degree of the applied position change operation, such as "crop," "rotate," "stretch," "tilt," or "distort." By displaying the estimated alignment information and estimated alignment image in this way, the terminal device 711 (first image processing device) can enable a user, etc., to more easily confirm the partial processing applied to the image.

[0211] The display control unit 755 may display the eclipsed area of ​​the distortion-matched estimated image as a portion whose authenticity cannot be guaranteed, as in the example of FIG.

[0212] For example, an original whole image is generated by performing so-called development processing on the RAW image of the captured image in the camera that captured the image. At that time, lens distortion correction is performed on the original whole image using the camera's lens distortion correction function. In contrast, when generating an edited whole image, the captured image is output as a RAW image to an external device (such as another image processing device) outside the camera, where it is developed using an image editing application or the like. At that time, lens distortion correction is performed using the lens distortion correction function of the image editing application or the like. The edited whole image is generated by processing the captured image developed in this way.

[0213] When such lens distortion correction is performed, the peripheral portions of the captured image may be removed due to excessive distortion, etc. In this specification, such removed areas are referred to as "shadow areas." For example, differences in the algorithms of the lens distortion correction functions of the camera and the image editing application may result in different lens distortion correction methods being applied to the entire original image and the entire processed image. In such cases, the sizes of these shadow areas may differ, and as a result of alignment, a shadow area may remain in the aligned estimated image 812 (i.e., a shadow area may be displayed), as shown in the example of FIG. 23 .

[0214] Since the authenticity of such a blocked area cannot be guaranteed, it may be clearly indicated that this is the case. For example, as in the example of Fig. 23, the blocked area 821 may be displayed in gray to clearly indicate that the area is an area where the authenticity cannot be guaranteed.

[0215] 24A, the aligned estimated image 831 may be displayed alongside the overall processed image 832. Also, as in the example of FIG. 24B, the aligned estimated image 831 may be displayed superimposed on the overall processed image 832.

[0216] <Flow of Processing Confirmation Process> The alignment estimation information and alignment estimation image generated as described above may be used for processing detection in the processing confirmation server 112. An example of the flow of processing confirmation process executed by the processing confirmation server 112 in this case will be described with reference to the flowchart in FIG.

[0217] When the processing confirmation process is started, the registration unit 612 (FIG. 11) of the processing confirmation server 112 executes registration estimation processing in step S801 to generate registration estimation information and a registered estimated image.

[0218] In step S802, the manipulation detection unit 614 compares the manipulated image with the aligned estimated image for each partial region, taking into account the alignment estimated information generated in step S801, detects manipulation, and generates manipulation detection region information.

[0219] In step S803, the modification detection area information supply unit 815 supplies the modification detection area information generated in step S802 to the terminal device 111. When the processing in step S803 ends, the modification confirmation processing ends.

[0220] By performing each process in this manner, the alignment estimation information and the alignment estimation image can be used for detecting manipulation, and therefore the manipulation confirmation server 112 (second image processing device) can make it easier for users to check partial manipulations applied to an image.

[0221] <5. Supplementary Notes> <Entire Original Image> The entire original image described above may be a still image (for example, a captured image captured as a still image), or may be a still image obtained by extracting one frame from a moving image (for example, a still image obtained by extracting one frame from a captured image captured as a moving image).

[0222] <Combination> Each element of the present technology described above may be applied in combination with any other element, as long as no contradiction occurs. Three or more elements of the present technology may be applied in combination. Furthermore, elements that can be combined may include all elements described in this specification. Furthermore, each element described above may be applied in combination with other elements not described above.

[0223] <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that make up the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0224] FIG. 26 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0225] In a computer 900 shown in FIG. 26, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0226] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0227] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and an input terminal. The output unit 912 includes, for example, a display, a speaker, and an output terminal. The storage unit 913 includes, for example, a hard disk, a RAM disk, and a non-volatile memory. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0228] In a computer configured as described above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0229] The program executed by the computer can be applied by recording it on, for example, a removable medium 921 such as a package medium. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by inserting the removable medium 921 into the drive 915.

[0230] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0231] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0232] <Application of the Present Technology> The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0233] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a set in which other functions are added to a unit (e.g., a video set).

[0234] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0235] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0236] <Fields and uses to which this technology can be applied> Systems, devices, processing units, etc. to which this technology is applied can be used in any field, for example, transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, nature monitoring, etc. In addition, the uses thereof are also arbitrary.

[0237] <Others> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. That is, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be included not only in a bitstream, but also in a bitstream that includes differential information of the identification information relative to certain reference information. Therefore, in this specification, "flag" and "identification information" encompass not only the information itself, but also differential information relative to the reference information.

[0238] Furthermore, various types of information (e.g., metadata) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, the associated pieces of data may be combined into one piece of data or may be separate pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Furthermore, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer not to the entire data, but to only part of the data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0239] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0240] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0241] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0242] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0243] Also, for example, each step of a single flowchart may be executed by a single device, or may be shared and executed by multiple devices. Furthermore, when a single step includes multiple processes, the multiple processes may be executed by a single device, or may be shared and executed by multiple devices. In other words, multiple processes included in a single step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0244] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0245] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0246] The present technology can also be configured as follows. (1) An image processing device including a display control unit that displays an entire processed image, which is an image obtained by partially processing an entire original image and which presents a processing detection area in which the partial processing has been detected, and an enlarged processed image, which is an image obtained by enlarging a partial area of ​​the entire processed image and which presents the processing detection area in a manner different from that in the entire processed image, and a display unit that displays the entire processed image and the enlarged processed image under control of the display control unit. (2) The image processing device described in (1), in which the display control unit presents the processing detection area in the enlarged processed image at a finer granularity than in the entire processed image. (3) The image processing device described in (1) or (2), in which the display control unit displays the processing detection area in the entire processed image after masking, and displays the non-processing detection area in the enlarged processed image after masking. (4) The image processing device described in any of (1) to (3), in which the display control unit further displays an enlarged original image, which is an enlarged partial area of ​​the entire original image, at the same position as the enlarged processed image. (5) The image processing device according to (4), wherein the enlarged original image is an image obtained by enlarging the partial region of an estimated registration image, which is the entire original image estimated to be aligned with the entire processed image. (6) The image processing device according to (5), further comprising an estimated registration image acquisition unit that acquires the estimated registration image, wherein the enlarged original image is an image obtained by enlarging the partial region of the acquired estimated registration image. (7) The image processing device according to any of (4) to (6), wherein the display control unit switches the display of the enlarged processed image to the display of the enlarged original image. (8) The image processing device according to any of (4) to (7), wherein the display control unit displays the enlarged original image superimposed on the enlarged processed image. (9) The image processing device according to any of (1) to (8), wherein the display control unit displays the manipulation detection region in the enlarged processed image together with a detection level of the manipulation detection region.(10) The image processing device according to (9), wherein the display control unit presents the detection level by a filling pattern for each granularity of the manipulation detection area. (11) The image processing device according to (9) or (10), wherein the display control unit presents the detection level by a frame pattern for each granularity of the manipulation detection area. (12) The image processing device according to any of (1) to (11), wherein the display control unit does not present the manipulation detection area in the enlarged, processed image. (13) The image processing device according to any of (1) to (12), further comprising an input unit for inputting a user instruction, wherein the display control unit is configured to display the enlarged, processed image obtained by enlarging the partial area of ​​the entire processed image specified by the user instruction. (14) The image processing device according to any of (1) to (13), further comprising a manipulation detection area information acquisition unit for acquiring manipulation detection area information related to the manipulation detection area, wherein the display control unit is configured to display the entire processed image and the enlarged, processed image presenting the manipulation detection area corresponding to the manipulation detection area information. (15) The image processing device according to (14), further comprising an image supply unit that supplies the entire original image and the entire edited image, wherein the edit detection area information acquisition unit is configured to acquire the edit detection area information related to the edit detection area detected using the supplied entire original image and the entire edited image. (16) The image processing device according to any of (1) to (15), further comprising a edit detection unit that detects the partial edit applied to the entire original image using the entire original image and the entire edited image, wherein the display control unit is configured to display the entire edited image and the enlarged edited image presenting the edit detection area in which the partial edit was detected. (17) The image processing device according to any of (1) to (16), wherein the partial edit is edit that falsifies a part of an image. (18) The image processing device according to (17), wherein the partial edit includes one or more of adding, changing, and deleting a subject object in an image.(19) The image processing device according to (17) or (18), wherein the partial processing includes adding blurring or mosaic to a partial region of the image. (20) The image processing device according to any one of (17) to (19), wherein the partial processing includes superimposing a composite image. (21) The image processing device according to any one of (17) to (20), wherein the partial processing includes skin beautification processing on the face of the subject. (22) The image processing device according to any one of (17) to (21), wherein the partial processing includes changing the size or shape of the contours or features of the face of the subject. (23) The image processing device according to any one of (1) to (22), wherein the entire original image is a still image obtained by extracting one frame from a moving image. (24) An image processing method comprising: controlling to display an entire original image subjected to partial processing, the entire processed image presenting a processing detection area in which the partial processing was detected, and an enlarged processed image, an image obtained by enlarging a partial area of ​​the entire processed image and presenting the processing detection area in a manner different from that of the entire processed image; and displaying the entire processed image and the enlarged processed image in accordance with the control. (25) A program for causing a computer to execute processing including controlling to display an entire original image subjected to partial processing, the entire processed image presenting a processing detection area in which the partial processing was detected, and an enlarged processed image, an image obtained by enlarging a partial area of ​​the entire processed image and presenting the processing detection area in a manner different from that of the entire processed image; and displaying the entire processed image and the enlarged processed image in accordance with the control.

[0247] (31) An image processing device comprising: a processing detection unit that detects partial processing performed on an entire original image by using an entire original image and an entire processed image obtained by performing partial processing on the entire original image, and a processing detection area information supply unit that supplies processing detection area information related to the processing detection area in which the partial processing is detected. (32) The image processing device according to (31), further comprising: a registration unit that performs registration estimation that determines a positional correspondence between the entire original image and the entire processed image, and a registration estimation supply unit that supplies, as a result of the registration estimation, at least one of registration estimation information related to the registration estimation and a registration estimation image, which is the entire original image reflecting the processing result of the registration estimation. (33) The image processing device according to (32), wherein the registration estimation includes a position change operation estimation that estimates a type and degree of a position change operation required to align positions of image elements of the entire original image with positions of image elements of the entire processed image, and the position change operation is an operation on the entire image that involves changing the position of an image element. (34) The image processing device according to (33), wherein the position change operation estimation further includes processing for determining whether the degree of the estimated position change operation is within an allowable range. (35) The image processing device according to (33) or (34), wherein the position change operation estimation further includes processing for reflecting the estimated position change operation in the entire original image. (36) The image processing device according to any of (33) to (35), wherein the position change operation includes one or more of cropping, rotating, resizing, stretching, and tilt correction. (37) The image processing device according to any of (32) to (36), wherein the registration estimation includes distortion adjustment estimation for estimating a difference in lens distortion correction between the entire original image and the entire processed image. (38) The image processing device according to (37), wherein the distortion adjustment estimation further includes processing for reflecting the estimated difference in lens distortion correction in the entire original image.(39) The image processing device according to (37) or (38), wherein the alignment unit performs the distortion alignment estimation using lens information included in metadata of the overall processed image and a distortion table corresponding to the lens information. (40) The image processing device according to any of (37) to (39), wherein the alignment unit performs the distortion alignment estimation using camera setting values ​​at the time of shooting included in metadata of the overall processed image. (41) The image processing device according to any of (32) to (40), wherein the alignment estimation information includes position change operation estimation information related to position change operation estimation that estimates the type and degree of position change operation necessary to align the positions of image elements of the overall original image with the positions of image elements of the overall processed image, and the position change operation is an operation on the overall image that involves changing the positions of elements in the image. (42) The image processing device according to (41), wherein the position change operation estimation information includes information indicating the type and degree of the position change operation estimated by the position change operation estimation. (43) The image processing device according to (41) or (42), wherein the position change operation estimation information includes information indicating a determination result as to whether the degree of the position change operation estimated by the position change operation estimation is within an allowable range. (44) The image processing device according to any of (32) to (43), wherein the alignment estimation information includes distortion adjustment estimation information related to distortion adjustment estimation that estimates a difference in lens distortion correction between the entire original image and the entire processed image. (45) The image processing device according to (44), wherein the distortion adjustment estimation information includes information indicating the difference in lens distortion correction estimated by the distortion adjustment estimation. (46) The image processing device according to any of (32) to (45), wherein the alignment estimated image includes a position change operation estimation image in which the position change operation estimated by position change operation estimation that estimates a type and degree of position change operation necessary to align the position of an element in the entire original image with the position of an element in the entire processed image is reflected in the entire original image.(47) The image processing device according to any one of (32) to (46), wherein the aligned estimated image includes a distortion adjustment estimated image in which a difference in lens distortion correction estimated by distortion adjustment estimation that estimates a difference in lens distortion correction between the entire original image and the entire processed image is reflected in the entire original image. (48) The image processing device according to any one of (32) to (47), wherein the processing detection unit detects the partial processing using the aligned estimated image and the entire processed image. (49) The image processing device according to (48), wherein the processing detection unit further detects the partial processing using the alignment estimation information. (50) The image processing device according to any one of (32) to (49), wherein the alignment unit skips execution of a predetermined processing among processes performed to generate the entire processed image from the entire original image in the alignment estimation. (51) The image processing device according to (50), wherein the predetermined processing is the partial processing on a partial region of the entire original image. (52) The image processing device according to (50) or (51), wherein the predetermined processing is color correction. (53) The image processing device according to any of (31) to (52), further comprising an image acquisition unit that acquires the entire original image and the entire processed image, wherein the processing detection unit is configured to detect the partial processing applied to the entire original image using the acquired entire original image and the entire processed image. (54) The image processing device according to any of (31) to (53), wherein the partial processing is processing that falsifies a part of the image. (55) The image processing device according to (54), wherein the partial processing includes one or more of adding, changing, and deleting a subject object in an image. (56) The image processing device according to (54) or (55), wherein the partial processing includes blurring or adding a mosaic to a partial region of the image. (57) The image processing device according to any of (54) to (56), wherein the partial processing includes superimposing a composite image. (58) The image processing device according to any one of (54) to (57), wherein the partial processing includes skin beautification processing for the face of the subject.(59) The image processing device according to any one of (54) to (58), wherein the partial processing includes changing the size or shape of the contours or features of the subject's face. (60) The image processing device according to any one of (31) to (59), wherein the entire original image is a still image obtained by extracting one frame from a moving image. (61) An image processing method including: using the entire original image and an entire processed image obtained by applying partial processing to the entire original image, detecting the partial processing applied to the entire original image, and supplying processing detection area information related to the processing detection area in which the partial processing was detected. (62) A program for causing a computer to execute processing including: using the entire original image and an entire processed image obtained by applying partial processing to the entire original image, detecting the partial processing applied to the entire original image, and supplying processing detection area information related to the processing detection area in which the partial processing was detected.

[0248] (81) An image processing device comprising: an alignment estimated image acquisition unit that acquires an aligned estimated image, which is an entire original image estimated to be aligned with an entire processed image obtained by performing partial processing on the entire original image; a display control unit that displays an enlarged processed image obtained by enlarging a partial region of the entire processed image and an enlarged original image obtained by enlarging the partial region of the acquired aligned estimated image, at the same positions as each other; and a display unit that displays the enlarged processed image and the enlarged original image in accordance with control of the display control unit. (82) The image processing device according to (81), wherein the display control unit switches the display of the enlarged processed image to the display of the enlarged original image. (83) The image processing device according to (81) or (82), wherein the display control unit displays the enlarged original image superimposed on the enlarged processed image. (84) The image processing device according to any one of (81) to (83), further comprising an image supply unit that supplies the entire original image and the entire processed image, wherein the aligned estimated image acquisition unit is configured to acquire the aligned estimated image derived using the supplied entire original image and the entire processed image. (85) The image processing device according to any one of (81) to (84), further comprising a processing detection unit that detects the partial processing applied to the entire original image using the entire original image and the entire processed image, wherein the display control unit is configured to present a processing detection area in which the partial processing is detected in the enlarged processed image and the enlarged original image to be displayed. (86) The image processing device according to any one of (81) to (85), wherein the partial processing is processing that falsifies a part of an image. (87) The image processing device according to (86), wherein the partial processing includes one or more of adding, changing, and deleting a subject object in an image. (88) The image processing device according to (86) or (87), in which the partial processing includes adding blurring or mosaic to a partial region of the image. (89) The image processing device according to any one of (86) to (88), in which the partial processing includes superimposing a composite image.(90) The image processing device according to any one of (86) to (89), wherein the partial processing includes skin beautification processing on the face of the subject. (91) The image processing device according to any one of (86) to (90), wherein the partial processing includes changing the size or shape of the contours or features of the face of the subject. (92) The image processing device according to any one of (81) to (91), wherein the entire original image is a still image obtained by extracting one frame from a moving image. (93) An image processing method including: acquiring an aligned estimated image, which is the entire original image estimated to be aligned with an entire processed image obtained by performing partial processing on the entire original image; controlling an enlarged processed image obtained by enlarging a partial region of the entire processed image and an enlarged original image obtained by enlarging the partial region of the acquired aligned estimated image, to be displayed in the same positions as each other; and displaying the enlarged processed image and the enlarged original image in accordance with the control. (94) A program for causing a computer to execute a process including: acquiring an aligned estimated image, which is an overall original image that has been aligned and estimated with respect to an overall processed image in which partial processing has been performed on the overall original image; controlling the display of an enlarged processed image in which a partial region of the overall processed image has been enlarged, and an enlarged original image in which the partial region of the acquired aligned estimated image has been enlarged, in the same positions as each other; and displaying the enlarged processed image and the enlarged original image in accordance with the control.

[0249] (111) An image processing device comprising: a registration estimation unit that performs registration estimation to determine a positional correspondence between an entire original image and an entire processed image obtained by performing partial processing on the entire original image; and a supply unit that supplies, as a result of the registration estimation, at least one of registration estimation information regarding the registration estimation and a registration estimation image, which is the entire original image reflecting the processing result of the registration estimation. (112) The image processing device according to (111), wherein the registration estimation includes a position change operation estimation that estimates a type and degree of a position change operation required to align positions of image elements of the entire original image with positions of image elements of the entire processed image, and the position change operation is an operation on the entire image that involves a change in position of an element within the image. (113) The image processing device according to (112), wherein the position change operation estimation further includes processing to determine whether the estimated degree of the position change operation is within an allowable range. (114) The image processing device according to (112) or (113), wherein the position change operation estimation further includes processing for reflecting the estimated position change operation in the entire original image. (115) The image processing device according to any of (112) to (114), wherein the position change operation includes one or more of cropping, rotating, resizing, stretching, and tilt correction. (116) The image processing device according to any of (111) to (115), wherein the alignment estimation includes distortion adjustment estimation for estimating a difference in lens distortion correction between the entire original image and the entire processed image. (117) The image processing device according to (116), wherein the distortion adjustment estimation further includes processing for reflecting the estimated difference in lens distortion correction in the entire original image. (118) The image processing device according to (116) or (117), wherein the alignment estimation unit performs the distortion adjustment estimation using lens information included in metadata of the entire processed image and a distortion table corresponding to the lens information. (119) The image processing device according to any one of (116) to (118), wherein the alignment estimation unit executes the distortion alignment estimation using camera setting values ​​at the time of shooting included in metadata of the overall processed image.(120) The image processing device according to any of (111) to (119), wherein the alignment estimation information includes position change operation estimation information related to position change operation estimation that estimates the type and degree of position change operation required to align the positions of image elements of the entire original image with the positions of image elements of the entire processed image, and the position change operation is an operation on the entire image that involves changing the position of an image element. (121) The image processing device according to (120), wherein the position change operation estimation information includes information indicating the type and degree of the position change operation estimated by the position change operation estimation. (122) The image processing device according to (120) or (121), wherein the position change operation estimation information includes information indicating a determination result as to whether the degree of the position change operation estimated by the position change operation estimation is within an allowable range. (123) The image processing device according to any of (111) to (122), wherein the alignment estimation information includes distortion adjustment estimation information related to distortion adjustment estimation that estimates a difference in lens distortion correction between the entire original image and the entire processed image. (124) The image processing device according to (123), wherein the distortion matching estimation information includes information indicating the difference in lens distortion correction estimated by the distortion matching estimation. (125) The image processing device according to any of (111) to (124), wherein the alignment estimated image includes a position change operation estimated image in which the position change operation estimated by position change operation estimation that estimates the type and degree of position change operation necessary to align the position of an image element of the entire original image with the position of an image element of the entire processed image is reflected in the entire original image. (126) The image processing device according to any of (111) to (125), wherein the alignment estimated image includes a distortion matching estimated image in which the difference in lens distortion correction estimated by distortion matching estimation that estimates the difference in lens distortion correction between the entire original image and the entire processed image is reflected in the entire original image. (127) The image processing device according to any one of (111) to (126), wherein the registration estimation unit skips execution of a predetermined processing among the processing performed to generate the entire processed image from the entire original image in the registration estimation.(128) The image processing device according to (127), wherein the predetermined processing is partial processing on a partial region of the entire original image. (129) The image processing device according to (127) or (128), wherein the predetermined processing is color correction. (130) The image processing device according to any of (111) to (129), further comprising an image acquisition unit that acquires the entire original image and the entire processed image, wherein the registration estimation unit is configured to perform the registration estimation using the acquired entire original image and the entire processed image. (131) The image processing device according to any of (111) to (130), wherein the partial processing is processing to falsify a part of an image. (132) The image processing device according to (131), wherein the partial processing includes one or more of adding, changing, and deleting a subject object in an image. (133) The image processing device according to (131) or (132), wherein the partial processing includes blurring or adding a mosaic to a partial region of an image. (134) The image processing device according to any one of (131) to (133), wherein the partial processing includes superimposing a composite image. (135) The image processing device according to any one of (131) to (134), wherein the partial processing includes skin beautification processing on the face of the subject. (136) The image processing device according to any one of (131) to (135), wherein the partial processing includes changing the size or shape of the contours or features of the face of the subject. (137) The image processing device according to any one of (111) to (136), wherein the entire original image is a still image obtained by extracting one frame from a moving image. (138) An image processing method including: performing registration estimation to determine a positional correspondence between the entire original image and an entire processed image obtained by performing partial processing on the entire original image; and supplying, as a result of the registration estimation, at least one of registration estimation information regarding the registration estimation and a registration estimation image, which is the entire original image reflecting the processing result of the registration estimation.(139) A program for causing a computer to execute a process including: performing alignment estimation to determine a positional correspondence between an entire original image and an entire processed image obtained by performing partial processing on the entire original image; and supplying, as a result of the alignment estimation, at least one of alignment estimation information regarding the alignment estimation and an alignment estimation image, which is the entire original image reflecting the processing result of the alignment estimation.

[0250] (151) An image processing device comprising: a registration estimation information acquisition unit that acquires registration estimation information related to registration estimation that finds a positional correspondence between an entire original image and an entire processed image obtained by performing partial processing on the entire original image; a registration estimation image acquisition unit that acquires a registration estimation image that is the entire original image reflecting the processing result of the registration estimation; and a display control unit that displays the acquired registration estimation information and the acquired registration estimation image on a display unit. (152) The image processing device according to (151), wherein the registration estimation information includes position change operation estimation information related to position change operation estimation that estimates a type and degree of position change operation necessary to align positions of image elements of the entire original image with positions of image elements of the entire processed image, and the position change operation is an operation on the entire image that involves changing the positions of elements within the image. (153) The image processing device according to (152), wherein the position change operation estimation information includes information indicating the type and degree of the position change operation estimated by the position change operation estimation. (154) The image processing device according to (152) or (153), wherein the position change operation estimation information includes information indicating a determination result as to whether the degree of the position change operation estimated by the position change operation estimation is within an allowable range. (155) The image processing device according to any of (151) to (154), wherein the alignment estimation information includes distortion adjustment estimation information related to distortion adjustment estimation that estimates a difference in lens distortion correction between the entire original image and the entire processed image. (156) The image processing device according to (155), wherein the distortion adjustment estimation information includes information indicating the difference in lens distortion correction estimated by the distortion adjustment estimation. (157) The image processing device according to any of (151) to (156), wherein the alignment estimated image includes a position change operation estimated image in which the position change operation estimated by position change operation estimation that estimates a type and degree of position change operation necessary to align the position of an element in the entire original image with the position of an element in the entire processed image is reflected in the entire original image.(158) The image processing device according to any one of (151) to (157), wherein the aligned estimated image includes a distortion aligned estimated image in which a difference in lens distortion correction estimated by distortion alignment estimation that estimates a difference in lens distortion correction between the entire original image and the entire processed image is reflected in the entire original image. (159) The image processing device according to (158), wherein the display control unit displays a shadowed area of ​​the distortion aligned estimated image as a portion whose authenticity cannot be guaranteed. (160) The image processing device according to any one of (151) to (159), wherein the display control unit displays the aligned estimated image alongside the entire processed image. (161) The image processing device according to any one of (151) to (160), wherein the display control unit displays the aligned estimated image superimposed on the entire processed image. (162) The image processing device according to any one of (151) to (161), wherein the entire original image is a still image obtained by extracting one frame from a moving image. (163) An image processing method comprising: acquiring alignment estimation information related to alignment estimation that determines a positional correspondence between an entire original image and an entire processed image obtained by performing partial processing on the entire original image, acquiring an alignment estimation image that is the entire original image reflecting a result of the alignment estimation processing, and displaying the acquired alignment estimation information and the acquired alignment estimation image on a display unit. (164) An image processing method comprising: acquiring alignment estimation information related to alignment estimation that determines a positional correspondence between an entire original image and an entire processed image obtained by performing partial processing on the entire original image, acquiring an alignment estimation image that is the entire original image reflecting a result of the alignment estimation processing, and displaying the acquired alignment estimation information and the acquired alignment estimation image on a display unit.

[0251] 100 Editing confirmation system, 110 Network, 111 Terminal device, 112 Editing confirmation server, 113 Authenticity determination server, 511 Input unit, 512 Image supply unit, 513 Editing detection area information acquisition unit, 514 Alignment estimated image acquisition unit, 515 Display control unit, 516 Display unit, 611 Image acquisition unit, 612 Alignment unit, 613 Alignment estimated image supply unit, 614 Editing detection unit, 615 Editing detection area information supply unit, 700 Alignment estimation system, 710 Network, 711 Terminal device, 712 Alignment estimation server, 713 Authenticity determination server, 721 Image acquisition unit, 722 Alignment estimation unit, 723 Supply unit, 731 Position change operation estimation unit, 732 Distortion alignment estimation unit 751 Input unit, 752 Image supply unit, 753 Alignment estimated information acquisition unit, 754 Alignment estimated image acquisition unit, 755 Display control unit, 756 Display unit, 761 Position change operation estimated information acquisition unit, 762 Distortion alignment estimated information acquisition unit, 771 Position change operation estimated image acquisition unit, 772 Distortion alignment estimated image acquisition unit, 900 Computer

Claims

1. An image processing device comprising: a display control unit that displays an entire processed image, which is an image in which partial processing has been applied to an entire original image, and which presents a processing detection area in which the partial processing has been detected; and an enlarged processed image, which is an image in which a partial area of ​​the entire processed image has been enlarged, and which presents the processing detection area in a manner different from that of the entire processed image; and a display unit that displays the entire processed image and the enlarged processed image in accordance with the control of the display control unit.

2. The image processing device according to claim 1, wherein the display control unit causes the processing detection area to be presented in the enlarged processed image at a finer granularity than in the case of the overall processed image.

3. The image processing device according to claim 1, wherein the display control unit masks the processing detection area of ​​the entire processed image and displays it, and masks the non-processing detection area of ​​the enlarged processed image and displays it.

4. The image processing device according to claim 1, wherein the display control unit further displays an enlarged original image obtained by enlarging a partial region of the entire original image at the same position as the enlarged processed image.

5. An image processing device according to claim 4, wherein the enlarged original image is an image obtained by enlarging the partial region of the estimated registration image, which is the entire original image estimated to be registered with respect to the entire processed image.

6. An image processing device according to claim 5, further comprising an alignment estimation image acquisition unit for acquiring the alignment estimation image, wherein the enlarged original image is an image obtained by enlarging the partial region of the acquired alignment estimation image.

7. The image processing device according to claim 4, wherein the display control unit switches the display of the enlarged image to the display of the enlarged original image.

8. The image processing device according to claim 4, wherein the display control unit displays the enlarged original image superimposed on the enlarged processed image.

9. The image processing device according to claim 1, wherein the display control unit displays the manipulation detection area together with a detection level of the manipulation detection area in the enlarged and manipulated image.

10. The image processing device according to claim 1, wherein the display control unit does not display the processing detection area in the enlarged processed image.

11. The image processing device according to claim 1, wherein the entire original image is a still image obtained by extracting one frame from a moving image.

12. An image processing method comprising: controlling the display of an entire processed image, which is an image in which partial processing has been applied to an entire original image, the entire processed image presenting a processing detection area in which the partial processing has been detected; and an enlarged processed image, which is an image in which a partial area of ​​the entire processed image has been enlarged, the processing detection area being presented in a manner different from that in the entire processed image; and displaying the entire processed image and the enlarged processed image in accordance with the control.

13. An image processing device comprising: an alteration detection unit that detects partial alterations made to an entire original image using an entire original image and an overall altered image in which partial alterations have been made to the entire original image; and an alteration detection area information supply unit that supplies alteration detection area information relating to the alteration detection area in which the partial alteration has been detected.

14. An image processing device as described in claim 13, further comprising: a registration unit that performs registration estimation to determine the positional correspondence between the entire original image and the entire processed image; and a registration estimation supply unit that supplies, as a result of the registration estimation, at least one of registration estimation information regarding the registration estimation and a registration estimation image, which is the entire original image reflecting the processing result of the registration estimation.

15. The image processing device of claim 14, wherein the alignment estimation includes at least one of a position change operation estimation that estimates the type and degree of a position change operation required to align the positions of image elements of the entire original image with the positions of image elements of the entire processed image, and a distortion alignment estimation that estimates the difference in lens distortion correction between the entire original image and the entire processed image, and wherein the position change operation is an operation on the entire image that involves changing the positions of image elements.

16. The image processing device according to claim 15, wherein the position change operation estimation further includes a process of determining whether the estimated degree of the position change operation is within an allowable range.

17. The image processing device according to claim 15, wherein the position change operation estimation further includes a process of reflecting the estimated position change operation on the entire original image.

18. The image processing device of claim 15, wherein the repositioning operations include one or more of cropping, rotating, resizing, stretching, and deskewing.

19. The image processing device according to claim 15, wherein the distortion matching estimation further includes a process of reflecting the estimated difference in the lens distortion correction in the entire original image.

20. An image processing device according to claim 15, wherein the alignment unit performs the distortion alignment estimation using lens information included in the metadata of the overall processed image and a distortion table corresponding to the lens information.

21. The image processing device according to claim 15, wherein the alignment unit executes the distortion alignment estimation using camera settings at the time of shooting that are included in the metadata of the overall processed image.

22. The image processing device described in claim 14, wherein the alignment estimation information includes at least one of position change operation estimation information relating to position change operation estimation, which estimates the type and degree of position change operation required to align the positions of image elements of the entire original image with the positions of image elements of the entire processed image, and distortion adjustment estimation information relating to distortion adjustment estimation, which estimates the difference in lens distortion correction between the entire original image and the entire processed image, and the position change operation is an operation on the entire image that involves changing the positions of image elements.

23. An image processing device as described in claim 22, wherein the position change operation estimation information includes at least one of information indicating the type and degree of the position change operation estimated by the position change operation estimation, and information indicating a determination result as to whether the degree of the position change operation estimated by the position change operation estimation is within an acceptable range.

24. The image processing device described in claim 14, wherein the alignment estimated image includes at least one of a position change operation estimated image in which a position change operation estimated by a position change operation estimation that estimates the type and degree of a position change operation required to align the position of an image element of the entire original image with the position of an image element of the entire processed image is reflected in the entire original image, and a distortion adjustment estimated image in which a difference in lens distortion correction estimated by a distortion adjustment estimation that estimates the difference in lens distortion correction between the entire original image and the entire processed image is reflected in the entire original image.

25. An image processing device according to claim 14, wherein the processing detection unit detects the partial processing using the aligned estimated image, the overall processed image, and the aligned estimated information.

26. An image processing device according to claim 14, wherein the alignment unit skips the execution of a predetermined processing step among the processing steps performed to generate the entire processed image from the entire original image during the alignment estimation.

27. The image processing device according to claim 13, wherein the entire original image is a still image obtained by extracting one frame from a moving image.

28. An image processing method comprising: using an entire original image and an entire processed image in which partial processing has been applied to the entire original image, detecting partial processing applied to the entire original image; and supplying processing detection area information regarding the processing detection area in which the partial processing has been detected.

29. An image processing device comprising: an alignment estimated image acquisition unit that acquires an alignment estimated image, which is an overall original image that has been estimated to be aligned with an overall processed image that has been subjected to partial processing on the overall original image; a display control unit that displays an enlarged processed image that is an enlarged partial region of the overall processed image and an enlarged original image that is an enlarged partial region of the acquired alignment estimated image, at the same positions as each other; and a display unit that displays the enlarged processed image and the enlarged original image in accordance with the control of the display control unit.

30. An image processing method comprising: acquiring an estimated aligned image, which is an overall original image that has been estimated to be aligned with an overall processed image obtained by performing partial processing on the overall original image; controlling an enlarged processed image obtained by enlarging a partial region of the overall processed image and an enlarged original image obtained by enlarging the partial region of the acquired estimated aligned image, so that they are displayed in the same positions as each other; and displaying the enlarged processed image and the enlarged original image in accordance with the control.

Images