Information processing device and method

Abstract

The present disclosure pertains to an information processing device and a method that make it possible to more precisely search for related images of a search image. Similarity information regarding data similarity between a search image and a candidate image, which is a candidate of a related image having a processing relationship with the search image, is derived, and the processing relationship between the search image and the candidate image is determined on the basis of the similarity information. Moreover, a display image indicating the processing relationship between the search image and the candidate image is generated, and the display image is displayed. The present disclosure can be applied to, for example, information processing devices, electronic equipment, information processing methods, or programs.

Description

Information processing device and method

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that enable a search for images related to a search image with higher accuracy.

[0002] In recent years, advances in image generation technology using AI (artificial intelligence) have made it easier to create photographic images and videos, resulting in various social issues such as fake news. In response to this, methods have been developed to verify the authenticity of images by verifying that they are not AI-generated or maliciously manipulated. For example, there is a method that enables tamper detection by adding a digital signature to an image when it is captured (see, for example, Patent Document 1). Another method exists that records a history of image creation and editing with a digital signature, enabling tracing of the history (see, for example, Non-Patent Document 1). Another method exists that adds 3D information and a signature to an image, enabling the detection of image tampering and the detection of forgery in which an altered image is projected onto a panel and photographed (see, for example, Patent Document 2).

[0003] However, when checking the authenticity of an edited image, it is necessary not only to check the authenticity of the original image but also to check whether malicious manipulation has been performed during the editing process, and in this case, it is necessary to compare the actually taken image with the edited image. In other words, in order for a user who only owns the image to be determined as the authenticity to use these techniques to determine the authenticity of that image, it is necessary to prepare related images (images that are in a processing relationship with the image to be determined as the authenticity and are confirmed to be authentic) as comparison targets. In other words, images that are in a processing relationship with the image to be determined as the authenticity are searched for from a group of images with confirmed authenticity, and are used as comparison targets when determining the authenticity.

[0004] Conventionally, there have been various image search methods, such as a method for efficiently searching for similar images using hash values ​​of two lengths (see, for example, Patent Document 3).

[0005] International Publication No. WO 2022 / 137798 International Publication No. WO 2023 / 248807 Japanese Patent Application Laid-Open No. 2020-173640

[0006] https: / / c2pa.org /

[0007] However, this method of searching for similar images does not allow for the identification of editing relationships, making it difficult to search for related images that can be used as comparison targets when determining authenticity. In other words, there is a risk that the accuracy of searching for related images of a search image (an image for which related images are searched) may be reduced.

[0008] The present disclosure has been made in consideration of such circumstances, and makes it possible to search for images related to a search image with higher accuracy.

[0009] An information processing device according to one aspect of the present technology is an information processing device that includes a similarity information derivation unit that derives similarity information regarding the data similarity between a search image and a candidate image that is a candidate for a related image that has a processing relationship with the search image, and a processing determination unit that determines the processing relationship between the search image and the candidate image based on the similarity information.

[0010] An information processing method according to one aspect of the present technology is an information processing method that derives similarity information regarding the data similarity between a search image and a candidate image that is a candidate for a related image that has a processing relationship with the search image, and determines the processing relationship between the search image and the candidate image based on the similarity information.

[0011] An information processing device according to another aspect of the present technology is an information processing device including: a display image generation unit that generates a display image indicating the processing relationship between a search image and a candidate image that is a candidate for a related image that has a processing relationship with the search image; and a display unit that displays the display image.

[0012] Another aspect of the present technology is an information processing method for generating a display image indicating the processing relationship between a search image and a candidate image that is a candidate for a related image in a processing relationship, and displaying the display image.

[0013] In an information processing device and method according to one aspect of the present technology, similarity information is derived regarding the data similarity between a search image and a candidate image that is a candidate for a related image that has a processing relationship with the search image, and the processing relationship between the search image and the candidate image is determined based on the similarity information.

[0014] In the information processing device and method according to another aspect of the present technology, a display image indicating a processing relationship between a search image and a candidate image that is a candidate for a related image in a processing relationship with the search image is generated, and the display image is displayed.

[0015] 10 is a diagram showing an example of the main configuration of an image search system. FIG. 11 is a block diagram showing an example of the main configuration of an image search server. FIG. 12 is a diagram showing an example of the main configuration of similarity information and related image filtering information. FIG. 13 is a block diagram showing an example of the main configuration of a terminal device. FIG. 14 is a diagram showing an example of a method for searching for related images. FIG. 15 is a diagram showing an example of the main configuration of an image search server. FIG. 16 is a flowchart showing an example of the flow of a learning process. FIG. 17 is a diagram showing an example of the main configuration of an image search server. FIG. 18 is a flowchart showing an example of the flow of a processing determination process. FIG. 19 is a flowchart continuing from FIG. 9 showing an example of the flow of a processing determination process. FIG. 19 is a flowchart showing an example of the flow of a related image search process. FIG. 10 is a diagram showing an example of a display image. FIG. 11 is a diagram showing an example of a display image. FIG. 12 is a block diagram showing an example of the main configuration of a computer.

[0016] The following describes modes for carrying out the present disclosure (hereinafter referred to as embodiments). The description will be given in the following order: 1. Literature supporting technical content and technical terms 2. Tamper detection and related image search 3. First embodiment (image search system) 4. Supplementary notes

[0017] <1. Literature, etc. supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the content described in the embodiments, but also the content described in the following patent documents and non-patent documents that were publicly known at the time of filing, as well as the content of other documents referenced in the following patent documents and non-patent documents.

[0018] Patent document 1: (described above) Patent document 2: (described above) Patent document 3: (described above) Non-patent document 1: (described above)

[0019] In other words, the contents of the above-mentioned patent documents and non-patent documents, as well as the contents of other documents referenced in the above-mentioned patent documents and non-patent documents, are also used as the basis for determining support requirements.

[0020] <2. Detecting Tampering and Searching for Related Images> <Determining Image Authenticity> In recent years, advances in image generation technology using AI (artificial intelligence) have made it easier to create photographic images and videos, resulting in various social issues such as fake news. In response to this, methods have been devised for determining image authenticity that verify that an image is not generated by AI or has been maliciously altered. For example, Patent Document 1 proposes a method that enables tamper detection by adding a digital signature to an image when it is captured. Furthermore, as disclosed in Non-Patent Document 1, there is a method that records a history of image creation and editing with a digital signature, making the history traceable. Furthermore, Patent Document 2 proposes a method that adds 3D information and a signature to an image, enabling the detection of image tampering and the detection of forgery in which an altered image is projected onto a panel and photographed.

[0021] However, when checking the authenticity of an edited image, it is necessary not only to check the authenticity of the original image but also to check whether malicious manipulation has been performed during the editing process, and in this case, it is necessary to compare the actually taken image with the edited image. In other words, in order for a user who only owns the image to be determined as the authenticity to use these techniques to determine the authenticity of that image, it is necessary to prepare related images (images that are in a processing relationship with the image to be determined as the authenticity and are confirmed to be authentic) as comparison targets. In other words, images that are in a processing relationship with the image to be determined as the authenticity are searched for from a group of images with confirmed authenticity, and are used as comparison targets when determining the authenticity.

[0022] In this specification, an image that has a processing relationship with image A and is authentic is also referred to as a "related image of image A." Furthermore, a relationship in which one image is processed to generate another image is also referred to as a "processing relationship." For example, when image B is generated by applying arbitrary processing to image A, or when image A is generated by applying arbitrary processing to image B, it can be said that "image A is in a processing relationship with image B," "image B is in a processing relationship with image A," or "image A and image B have a processing relationship with each other."

[0023] In this specification, the term "processing" refers to any process performed on image data, including not only processes that change the content of an image (processing that results in a visual change) but also processes that modify the data. This "processing that modifies the content of an image" includes, for example, image cropping, compositing, enlarging or reducing, rotating or transforming, correcting lens aberrations, correcting color or brightness, and adjusting white balance, as well as image replacement (modifying image data). Furthermore, "processing that modifies data" also includes, for example, changing the compression rate, format conversion, and falsifying metadata. In other words, "processing" includes any process that makes the processed image data indistinguishable from the unprocessed image data. In the following, "processing that modifies image content" will be used as an example of processing. However, the present technology can also be applied to "processing that modifies data" in the same way as the "processing that modifies image content" described below. Note that "processing" is also referred to as "editing (or image editing)" or "tampering," etc.

[0024] <Search for Related Images> Conventionally, there have been various image search methods. For example, Patent Document 3 proposes a method for efficiently searching for images similar to a search image using hash values ​​of two lengths. In this specification, the image to be searched for similar images (i.e., the input image A in an image search that inputs image A and outputs image B similar to image A) is also referred to as the "search image."

[0025] However, while such a search method can search for images that are subjectively similar to a search image, it is difficult to ascertain the editing relationship. In other words, it is difficult to use such a search method to search for images that can be used as comparison targets when determining the authenticity of the search image (i.e., related images of the search image). In other words, there is a risk that the accuracy of searching for related images of the search image will be reduced.

[0026] 3. First Embodiment Image Search System Fig. 1 is a diagram showing an example of the configuration of an image search system, which is one aspect of an information processing system to which the present technology is applied. The image search system 100 shown in Fig. 1 is a system that searches for images related to a search image. That is, the image search system 100 receives a search image as input and outputs images related to the search image. As described above, these related images are images that have been processed in relation to the search image and are of confirmed authenticity. That is, the search image is an image whose authenticity is to be determined, and the image search system 100 can also be said to be a system that searches for images that can be used as comparison targets in determining the authenticity of the search image.

[0027] 1, the image search system 100 includes an image search server 111, an authenticity determination server 112, and a terminal device 113. The image search server 111, the authenticity determination server 112, and the terminal device 113 are communicatively connected to each other via a network 110, and can exchange information with each other via this communication. Note that this communication may be wired communication, wireless communication, or both.

[0028] The network 110 is a communication network configured with any communication medium. The communication performed via the network 110 may be wired communication, wireless communication, or both. In other words, the network 110 may be a communication network for wired communication, a communication network for wireless communication, or a communication network configured with both. Furthermore, the network 110 may be configured with a single communication network or multiple communication networks.

[0029] For example, the network 110 may include the Internet. The network 110 may also include a public telephone network. Furthermore, the network 110 may also include a wide-area communication network for wireless mobile devices, such as a so-called 3G line or 4G line. For example, the network 110 may also include a low-power wide-area (LPWA) communication network, such as LTE-M, which is capable of long-distance data communication and consumes low current. The network 110 may also include a wide-area (WAN) network or a local-area (LAN) network. The network 110 may also include a wireless communication network that communicates in accordance with the Bluetooth (registered trademark) standard. The network 110 may also include a short-range wireless communication channel, such as NFC (Near Field Communication). The network 110 may also include an infrared communication channel. The network 110 may also include a wired communication network that complies with standards such as HDMI (High-Definition Multimedia Interface) (registered trademark) and USB (Universal Serial Bus) (registered trademark). In this way, the network 110 may include a communication network or a communication path of any communication standard. Note that the communication network or the communication path may include not only a communication medium such as a cable, but also devices and circuits necessary for communication, such as a communication device or a relay device.

[0030] The image search server 111 receives a search image as input, searches for related images of the search image, and outputs the search results. The image search server 111 may also request a determination of the authenticity of the search image by supplying the search image and its related images to the authenticity determination server 112, and obtain the determination result. The authenticity determination server 112 receives the search image and related images as input, determines the authenticity of the search image using the related images, and outputs the determination result. The terminal device 113 may, for example, supply the search image to the image search server 111, request a search for related images of the search image, and obtain the search results. The terminal device 113 may also generate and display a display image showing the search results.

[0031] <Configuration of Image Search Server> Fig. 2 is a block diagram showing an example of the main configuration of the image search server 111. As shown in Fig. 2, the image search server 111 has a communication unit 201, an authenticity determination result acquisition unit 202, an information generation unit 203, a candidate image filtering unit 204, a modification determination unit 205, and an image search unit 206.

[0032] The communication unit 201 has a communication function and executes processing related to communication with other devices via the network 110. For example, the communication unit 201 may send and receive information to and from other devices via this communication.

[0033] The authenticity determination result acquisition unit 202 executes processing related to acquisition of authenticity determination result information. For example, the authenticity determination result acquisition unit 202 may acquire authenticity determination result information supplied from the authenticity determination server 112 via the communication unit 201. The authenticity determination result acquisition unit 202 may also request authenticity determination result information from the authenticity determination server 112 via the communication unit 201. The authenticity determination result acquisition unit 202 may supply the acquired authenticity determination result information to the information generation unit 203.

[0034] The authenticity determination result information is information about the determination result of the authenticity of the image. For example, the authenticity determination result information may include an unedited image, an edited image, and an AI editing determination result between those images.

[0035] In this specification, the term "unprocessed image" refers to an image before a certain processing is performed. The image in the generated state (an image that has not been processed since it was generated) is also referred to as the original image. The unprocessed image may be the original image, or an image that has been processed in some way on the original image. Furthermore, the term "processed image" refers to an image that has been processed on the unprocessed image for that processing.

[0036] Furthermore, the "AI processing judgment result between those images" is information that indicates the judgment result of whether or not there was AI processing between the unprocessed image and the processed image. In other words, this information indicates whether or not the processed image was generated by applying AI processing to the unprocessed image. "AI processing" refers to malicious processing. The processing relationship of this AI processing is also referred to as the "AI processing relationship."

[0037] "Malicious editing" refers to editing that changes the phenomenon depicted in an image (the meaning of the image), such as a so-called fake image. For example, malicious editing may include replacing, erasing, or adding objects or people included in an image. In contrast, "non-malicious editing" refers to editing that does not substantially change the phenomenon depicted in an image (the meaning of the image), such as editing to improve appearance. For example, non-malicious editing may include cropping to achieve a predetermined aspect ratio, color correction, lens aberration correction, etc.

[0038] The criteria for distinguishing between non-malicious and malicious editing (which classification is determined) may be any criteria and are not limited to the above-mentioned examples. For example, the classification may be determined based on the type of editing, the scale of the editing, the target of the editing, the number of editing operations, the impact on the meaning of the image, etc. For example, cutting and pasting may be determined to be malicious editing, and color correction may be determined to be non-malicious editing. Editing that erases a large figure in a captured image to represent a false event may be determined to be malicious editing, while editing that erases a small figure in a captured image to improve the design may be determined to be non-malicious editing. The criteria for distinguishing between non-malicious and malicious editing (which classification is determined) may be predetermined or may change depending on the situation. For example, even if the editing is identical, editing by the photographer may be determined to be non-malicious editing, while editing by another user may be determined to be malicious editing. Furthermore, the classification result may change depending on any conditions, such as the time or device used to perform the editing. Furthermore, the criteria may be subjective or determined by the user or system that determines the authenticity.

[0039] In recent years, there has been an increase in cases of malicious editing using AI. Therefore, in this specification, malicious editing is also referred to as AI editing. However, malicious editing is possible without using AI, and non-malicious editing can also be performed using AI. Therefore, in this specification, AI editing also includes editing that does not use AI. In other words, in this specification, malicious editing is also referred to as AI editing, regardless of whether AI is used. For example, editing other than cropping, brightness adjustment, and resizing may also be referred to as AI editing.

[0040] In detecting image tampering, malicious editing may be detected as unauthorized tampering, while non-malicious editing may not be detected as unauthorized tampering. Furthermore, in determining authenticity, malicious editing may be distinguished from non-malicious editing, and authenticity may be determined depending on whether or not the editing was malicious. For example, the authenticity of an image that has been non-maliciously edited may be guaranteed. That is, the authenticity of an edited image that has been non-maliciously edited may be guaranteed for an unedited image whose authenticity is guaranteed. Furthermore, the authenticity of an image that has been maliciously edited may not be guaranteed. That is, the authenticity of an edited image that has been maliciously edited may not be guaranteed for an unedited image whose authenticity is guaranteed. In this case, the AI ​​editing determination result (information indicating whether AI editing has been performed) included in the above-mentioned authenticity determination result information can indicate whether or not the authenticity of such an edited image is guaranteed.

[0041] The information generation unit 203 executes processing related to information generation. For example, the information generation unit 203 may acquire authenticity determination result information supplied from the authenticity determination result acquisition unit 202. The information generation unit 203 may derive related image filtering information based on the authenticity determination result information. The information generation unit 203 may derive similarity information based on the authenticity determination result information. The information generation unit 203 may also acquire a search image and candidate images supplied from the image search unit 206. The information generation unit 203 may derive related image filtering information based on the search image and candidate images. The information generation unit 203 may derive similarity information based on the search image and candidate images. The information generation unit 203 may supply the derived related image filtering information to the candidate image filtering unit 204. The information generation unit 203 may supply the unprocessed image and the processed image used to derive the related image filtering information to the candidate image filtering unit 204. The information generating unit 203 may supply the derived similarity information to the manipulation determining unit 205. ...

[0042] The candidate images are images that are candidates for related images of the search image, that is, images whose processing relationship with the search image is to be determined.

[0043] As shown in FIG. 2 , the information generating unit 203 includes a related image filtering information deriving unit 211 and a similarity information deriving unit 212. The related image filtering information deriving unit 211 performs processing related to the derivation of related image filtering information. For example, the related image filtering information deriving unit 211 may derive related image filtering information based on authenticity determination result information. The related image filtering information deriving unit 211 may derive related image filtering information based on a search image and a candidate image. The similarity information deriving unit 212 performs processing related to the derivation of similarity information. For example, the similarity information deriving unit 212 may derive similarity information between an unedited image and an edited image included in the authenticity determination result information based on the authenticity determination result information. The similarity information deriving unit 212 may derive similarity information between the search image and the candidate image based on the search image and the candidate image.

[0044] FIG. 3 is a diagram showing an example of the main configuration of similarity information and related image filtering information. The related image filtering information 302 shown in FIG. 3 is information for narrowing down candidate images. This related image filtering information 302 is derived by the related image filtering information derivation unit 211. This related image filtering information 302 is composed of information derived from a single image. The content of the related image filtering information 302 may be any. For example, as shown in FIG. 3, the related image filtering information 302 may include metadata 321, a short perceptual hash value 322, and an object category tag 323. Of course, the related image filtering information 302 does not have to include these pieces of information, or may include information other than these pieces of information.

[0045] The metadata 321 is metadata of the image corresponding to this related image filtering information 302. The content of this metadata 321 may be any content related to the image. For example, the metadata 321 may include information related to the image, such as the creator, creation date and time, and location (photography location) of the image. For example, the related image filtering information derivation unit 211 may generate this metadata 321 using metadata attached to the unedited image, edited image, etc. included in the authenticity determination result information. Furthermore, the related image filtering information derivation unit 211 may generate this metadata 321 using metadata attached to the search image.

[0046] The short perceptual hash value 322 is a perceptual hash value with a relatively short data length. A perceptual hash value is a value obtained by inputting image data into a hash function, and has the characteristics that the same value is always generated from the same data and that similar values ​​are generated from data that are recognized as similar. Naturally, the shorter the data length of a perceptual hash value, the more likely the values ​​are to match (or be similar). In other words, when comparing two images using a perceptual hash value, the shorter the data length of the perceptual hash value, the easier it is to determine that the two images match (or are more similar). In other words, the data length of the perceptual hash value can be said to represent the accuracy (reliability) of the match (similarity). As described above, because the short perceptual hash value 322 has a relatively short data length, the reliability of a comparison using the short perceptual hash value 322 is relatively low. However, the load of the comparison process is relatively small.

[0047] For example, the related image filtering information derivation unit 211 may generate a short perceptual hash value 322 for an unprocessed image or an processed image included in the authenticity determination result information. The related image filtering information derivation unit 211 may also generate a short perceptual hash value 322 for a search image. Any algorithm may be used to derive the short perceptual hash value 322. For example, the short perceptual hash value 322 may be derived using a difference from the average luminance of the image. The short perceptual hash value 322 may be derived using a difference from the average luminance of a low-frequency region of an image transformed into the frequency domain by a discrete cosine transform, a discrete wavelet transform, or the like. The short perceptual hash value 322 may also be derived using a difference from an adjacent region.

[0048] The object category tag 323 is information indicating the classification result of the content of the image corresponding to the object category tag 323. In other words, the object category tag 323 indicates what kind of image the content of the corresponding image is (what genre it is classified into). For example, the related image filtering information derivation unit 211 may analyze the content of the unedited image and the edited image included in the authenticity determination result information to generate the object category tag 323. The related image filtering information derivation unit 211 may also analyze the content of the search image to generate the object category tag 323.

[0049] The similarity information 301 shown in FIG. 3 is a parameter representing the data similarity between images. This similarity information 301 is derived by the similarity information derivation unit 212. This similarity information 301 is composed of information derived from the multiple images being compared. The content of the similarity information 301 may be any. For example, as shown in FIG. 3, the similarity information 301 may include a difference value 311 of long perceptual hash values, a feature point matching result 312, and an AI processing determination result 313. Of course, the similarity information 301 does not have to include these pieces of information, or may include information other than these.

[0050] The long perceptual hash value difference value 311 is the difference value between perceptual hash values ​​of relatively long data lengths corresponding to each of a plurality of images. In other words, the data length of the perceptual hash value of each image is longer than the short perceptual hash value 322. In other words, the reliability of a comparison using this perceptual hash value is higher than that of a comparison using the short perceptual hash value 322. Furthermore, the load of a comparison process using this perceptual hash value is greater than that of a comparison using the short perceptual hash value 322.

[0051] For example, the difference value 311 of the long perceptual hash values ​​may be the difference between perceptual hash values ​​of relatively long data lengths corresponding to the pre-processed image and the post-processed image included in the authenticity determination result information. That is, the similarity information derivation unit 212 may generate the difference value 311 of the long perceptual hash values ​​indicating the difference between perceptual hash values ​​of relatively long data lengths corresponding to the pre-processed image and the post-processed image included in the authenticity determination result information. Furthermore, the difference value 311 of the long perceptual hash values ​​may be the difference between perceptual hash values ​​of relatively long data lengths corresponding to the search image and the candidate images. That is, the similarity information derivation unit 212 may generate the difference value 311 of the long perceptual hash values ​​indicating the difference between perceptual hash values ​​of relatively long data lengths corresponding to the search image and the candidate images. As in the case of the short perceptual hash value 322 described above, any algorithm may be used to derive the perceptual hash value of each image used to derive the difference value 311 of the long perceptual hash value.

[0052] The feature point matching result 312 is information indicating the result of matching the feature points of each of a plurality of images. For example, the feature point matching result 312 may be a result of matching feature points between the unedited image and the edited image included in the authenticity determination result information. In other words, the similarity information derivation unit 212 may match feature points between the unedited image and the edited image included in the authenticity determination result information, and generate the feature point matching result 312 indicating the result. The feature point matching result 312 may also be a result of matching feature points between the search image and the candidate image. In other words, the similarity information derivation unit 212 may match feature points between the search image and the candidate image, and generate the feature point matching result 312 indicating the result.

[0053] The AI ​​editing judgment result 313 is information indicating the result of an AI judgment as to whether editing has occurred. For example, the AI ​​editing judgment result 313 may indicate (the judgment result as to) whether AI editing has occurred between the unedited image and the edited image included in the authenticity judgment result information. In this case, the AI ​​editing judgment result 313 may be included in the authenticity judgment result information supplied from the authenticity judgment server 112. In other words, the similarity information derivation unit 212 may derive the similarity information 301 using the AI ​​editing judgment result 313 included in the authenticity judgment result information.

[0054] Returning to FIG. 2 , the candidate image filtering unit 204 performs processing related to narrowing down the candidate images. For example, the candidate image filtering unit 204 may acquire the unprocessed image and the processed image supplied from the information generating unit 203 and store them as candidate images. The candidate image filtering unit 204 may acquire and store related image filtering information supplied from the information generating unit 203. The candidate image filtering unit 204 may acquire the related image filtering information supplied from the information generating unit 203 and use the acquired related image filtering information to narrow down the candidate images (i.e., extract candidate images corresponding to the search image). The candidate image filtering unit 204 may supply the narrowed down (extracted) candidate images to the image search unit 206.

[0055] As shown in FIG. 2 , the candidate image filtering unit 204 includes a candidate image storage unit 221, an image filtering information DB (database) 222, and an image filtering unit 223. The candidate image storage unit 221 includes any storage medium and performs processing related to storing candidate images. For example, the candidate image storage unit 221 may store supplied unprocessed images and processed images in the storage medium as candidate images. The image filtering information DB 222 includes a database and performs processing related to managing related image filtering information. For example, the image filtering information DB 222 may store (register) and manage the supplied related image filtering information in the database. The image filtering unit 223 performs processing related to image filtering, i.e., narrowing down the candidate images. For example, the image filtering unit 223 may search for related image filtering information managed by the image filtering information DB 222 that corresponds to the supplied related image filtering information, and select a candidate image corresponding to the searched related image filtering information from among the candidate images stored in the candidate image storage unit 221.

[0056] The editing determination unit 205 executes processing related to editing determination. For example, the editing determination unit 205 may acquire similarity information and determine the editing relationship between the search image and the candidate images based on the similarity information. For example, the editing determination unit 205 may acquire similarity information regarding the unedited image and the edited image supplied from the information generation unit 203 and store it as learning data used for learning (machine learning). The editing determination unit 205 may use the learning data to perform learning (machine learning) related to editing determination, generate a learning model, and store it. That is, the editing determination unit 205 may perform learning using the similarity information between the unedited image and the edited image as training data and derive a learning model. The editing determination unit 205 may perform editing determination using the similarity information regarding the search image and the candidate images supplied from the information generation unit 203 and the learning model (i.e., the learning result), and derive editing relationship information regarding the search image and the candidate images as the editing determination result. That is, the editing determination unit 205 may determine the editing relationship between the search image and the candidate images using a learning model that inputs similarity information and outputs editing relationship information related to the editing relationship determination result. The editing determination unit 205 may supply the derived editing relationship information to the image search unit 206.

[0057] As shown in FIG. 2 , the processing determination unit 205 includes a learning data management DB (database) 231, a learning unit 232, and a processing-related information derivation unit 233. The learning data management DB 231 includes a database and performs processing related to management of learning data. For example, the learning data management DB 231 may store (register) and manage supplied similarity information in the database as learning data. The learning unit 232 performs processing related to learning (machine learning) related to processing determination. For example, the learning unit 232 may perform learning (machine learning) related to processing determination using the learning data managed by the learning data management DB 231 to generate a learning model. The processing-related information derivation unit 233 performs processing related to derivation of processing-related information using the learning model. For example, the processing-related information derivation unit 233 may perform processing determination by inputting supplied similarity information into the learning model, and derive the processing-related information.

[0058] The processing relationship information is information regarding the determination result of the processing relationship between a search image and a candidate image corresponding to the search image (i.e., the searched candidate image). For example, this processing relationship information may include an AI processing determination result, which is a predetermined processing to be detected between the search image and the candidate image. This processing relationship information may also include the similarity between the search image and the candidate image. This processing relationship information may be output from the processing relationship information derivation unit 233 (the learning model thereof) by inputting similarity information (the difference value 311 of the long perceptual hash value and the feature point matching result 312) into the processing relationship information derivation unit 233 (the learning model thereof).

[0059] The image search unit 206 executes processing related to searching for related images corresponding to a search image. For example, the image search unit 206 may acquire a search image supplied from the terminal device 113 via the communication unit 201. The image search unit 206 may supply the search image to the information generation unit 203 and request candidate images corresponding to the search image. In response to the request, the image search unit 206 may acquire candidate images (narrowed candidate images) supplied from the candidate image filtering unit 204. The image search unit 206 may supply the candidate images to the information generation unit 203 and request a determination of the processing relationship between the search image and the candidate images. In response to the request, the image search unit 206 may acquire processing relationship information supplied from the processing determination unit 205. The image search unit 206 may supply the acquired candidate images and processing relationship information as search results to the terminal device 113 via the communication unit 201.

[0060] As shown in FIG. 2 , the image search unit 206 includes a search image acquisition unit 241, a candidate image request unit 242, an editing judgment request unit 243, and a search result provision unit 244. The search image acquisition unit 241 performs processing related to the acquisition of a search image. For example, the search image acquisition unit 241 may acquire a search image supplied from the terminal device 113 via the communication unit 201. The candidate image request unit 242 performs processing related to a request (narrowing down) of candidate images. For example, the candidate image request unit 242 may request candidate images of related images corresponding to the search image by supplying the search image to the information generation unit 203. In response to the request, the candidate image request unit 242 may acquire candidate images (narrowed down candidate images) supplied from the candidate image filtering unit 204. The editing judgment request unit 243 performs processing related to the request for editing judgment. For example, the editing judgment request unit 243 may request a judgment on the editing relationship between the search image and the candidate images by supplying the candidate image to the information generation unit 203. In response to the request, the processing determination requesting unit 243 may acquire processing-related information supplied from the processing determination unit 205. The search result providing unit 244 executes processing related to providing search results. For example, the search result providing unit 244 may supply candidate images corresponding to the search image (i.e., searched candidate images), processing-related information, and the like to the terminal device 113 via the communication unit 201 as search results of candidate images related to the search image.

[0061] <Configuration of Terminal Device> Fig. 4 is a block diagram showing an example of the main configuration of the terminal device 113. As shown in Fig. 4, the terminal device 113 has an input unit 401, a search image selection unit 402, a search image supply unit 403, a communication unit 404, a search result acquisition unit 405, a display image generation unit 406, and a display unit 407.

[0062] The input unit 401 has any input device such as a keyboard, a mouse, a joystick, a button, a touch panel, an input terminal, etc., and executes processing related to input of instructions and information using the input device. For example, the input unit 401 may receive instructions and information related to a search for candidate images that are input from the outside via the input device, and supply the received instructions and information to the search image selection unit 402, etc.

[0063] The search image selection unit 402 executes processing related to the selection of a search image. For example, the search image selection unit 402 may select, as the search image, an image selected in response to an instruction supplied from the input unit 401. For example, the search image selection unit 402 may cause the display image generation unit 406 to generate a GUI (Graphical User Interface) for selecting a search image, display the GUI on the display unit 407, and select a search image based on an instruction input to the input unit 401 based on the GUI. The search image selection unit 402 may supply the image selected in this manner to the search image supply unit 403 as the search image.

[0064] The search image supply unit 403 executes processing related to the supply of a search image. For example, the search image supply unit 403 may supply the search image supplied from the search image selection unit 402 to the image search server 111 via the communication unit 404, thereby requesting a search for related images (candidate images) corresponding to the search image.

[0065] The communication unit 404 has a communication function and executes processing related to communication with other devices via the network 110. For example, the communication unit 404 may send and receive information to and from other devices via this communication.

[0066] The search result acquisition unit 405 executes processing related to acquisition of search results. For example, the search result acquisition unit 405 may acquire search results supplied from the image search server 111 via the communication unit 404. The search results may include candidate images (i.e., searched candidate images) corresponding to the search image supplied by the search image supply unit 403, processing-related information, etc. The search result acquisition unit 405 may supply the acquired search results to the display image generation unit 406.

[0067] The display image generation unit 406 executes processing related to the generation of a display image. For example, the display image generation unit 406 may acquire search results supplied from the search result acquisition unit 405. The display image generation unit 406 may generate a display image including the contents of the search results (candidate images, processing-related information, etc.). In other words, the display image may be an image indicating the processing relationship between the search image and candidate images that are candidates for related images in a processing relationship. The display image generation unit 406 may supply the generated display image to the display unit 407. Note that the display image generation unit 406 may generate a GUI requested by the search image selection unit 402 as a display image and supply it to the display unit 407. The display image generation unit 406 may also generate other display images and supply them to the display unit 407.

[0068] The display unit 407 has an arbitrary display device and executes processing related to displaying a display image using the display device. For example, the display unit 407 may acquire a display image supplied from the display image generation unit 406. The display unit 407 may display the acquired display image using the display device.

[0069] <Method 1> In such an image search system 100, for example, as shown in the top row of the table in FIG. 5, related images (candidate images) of a search image are searched for (Method 1). In addition to searching, the presence or absence of a processing relationship between the search image and the candidate images may be determined. Furthermore, the presence or absence of an AI processing relationship between the search image and the candidate images may be determined. Furthermore, the similarity between the search image and the candidate images may be calculated. Of course, other processes may also be performed.

[0070] For example, the image search server 111 (also referred to as the first information processing device) may include a similarity information derivation unit 212 that derives similarity information 301 regarding the data similarity between a search image and a candidate image that is a candidate for a related image that has a processing relationship with the search image, and a processing determination unit 205 that determines the processing relationship between the search image and the candidate image based on the similarity information 301. Furthermore, an information processing method of the image search server 111 (the first information processing device) may derive similarity information regarding the data similarity between a search image and a candidate image that is a candidate for a related image that has a processing relationship with the search image, and determine the processing relationship between the search image and the candidate image based on the similarity information.

[0071] By determining the editing relationship in this way, it is possible to grasp the editing relationship during image retrieval, thereby enabling more accurate retrieval of related images that can be used as comparison targets when determining the authenticity of a searched image.

[0072] As described above, the similarity information 301 may include a feature point matching result 312 between the search image and the candidate image, and may also include a difference in perceptual hash values ​​between the search image and the candidate image (a long perceptual hash value difference 311).

[0073] The similarity information derivation unit 212 may derive such similarity information 301. For example, the similarity information derivation unit 212 may derive a feature point matching result 312 between the search image and the candidate image, and include the result in the similarity information 301. Furthermore, the similarity information derivation unit 212 may derive a difference value 311 of the long perceptual hash values ​​between the search image and the candidate image, and include the difference value 311 in the similarity information 301.

[0074] <Method 1-1> When Method 1 is applied, for example, as shown in the second row from the top of the table in FIG. 5, the learning results may be used to search for related images (candidate images) (Method 1-1).

[0075] For example, in the image search server 111 (first information processing device), the processing determination unit 205 (learning unit 232) may perform learning using similarity information 301 between an unprocessed image and an edited image as training data, and derive a learning model that inputs the similarity information 301 and outputs processing-related information related to the processing-related determination result. Note that the similarity information 301 used as training data may include a determination result (AI processing determination result 313) of the presence or absence of AI processing, which is a predetermined type of processing to be detected, between the unprocessed image and the edited image. The similarity information 301 may also include a feature point matching result (feature point matching result 312) between the unprocessed image and the edited image. The similarity information 301 may also include a difference in perceptual hash values ​​between the unprocessed image and the edited image (difference value 311 between long perceptual hash values).

[0076] The similarity information derivation unit 212 may derive such similarity information 301. For example, the similarity information derivation unit 212 may derive a difference value 311 of long perceptual hash values ​​between the pre-processed image and the post-processed image, and include the difference value 311 in the similarity information 301. The similarity information derivation unit 212 may also derive a feature point matching result 312 between the pre-processed image and the post-processed image, and include the result in the similarity information 301. For example, the similarity information derivation unit 212 may derive this similarity information 301 using the pre-processed image and the post-processed image acquired as authenticity determination result information.

[0077] Furthermore, in the image search server 111 (first information processing device), the processing determination unit 205 (processing relationship information derivation unit 233) may determine the processing relationship between the search image and the candidate images using a learning model that inputs similarity information and outputs processing relationship information regarding the processing relationship determination result. Note that this processing relationship information may include a determination result of the presence or absence of AI processing, which is a predetermined processing to be detected, between the search image and the candidate images. Furthermore, this processing relationship information may include the similarity between the search image and the candidate images.

[0078] By using the learning results to determine whether an image has been altered in this way, it is possible to grasp the relationship between alterations more accurately in image retrieval, thereby enabling more accurate retrieval of images related to a search image.

[0079] <Method 1-2> When Method 1 is applied, for example, as shown in the third row from the top of the table in FIG. 5, the candidate images may be narrowed down using related image filtering information 302 (Method 1-2).

[0080] For example, the image search server 111 (first information processing device) may further include an image filtering unit 223 that narrows down candidate images using related image filtering information 302 for narrowing down candidate images. The processing determination unit 205 may then determine the processing relationship between the narrowed-down candidate images and the search image. The related image filtering information 302 may include a perceptual hash value (short perceptual hash value 322) of the image. The related image filtering information 302 may also include image metadata 321. The related image filtering information 302 may also include a category tag (object category tag 323) of the image.

[0081] The image search server 111 (first information processing device) may further include a related image filtering information derivation unit 211 that derives such related image filtering information 302. For example, the related image filtering information derivation unit 211 may derive short perceptual hash values ​​322 of the unedited image and the edited image, and include the short perceptual hash values ​​322 in the related image filtering information 302. The related image filtering information derivation unit 211 may also include metadata 321 of the unedited image and the edited image in the related image filtering information 302. The related image filtering information derivation unit 211 may also derive object category tags 323 of the unedited image and the edited image, and include the object category tags 323 in the related image filtering information 302. The image search server 111 (first information processing device) may further include an image filtering information DB 222 that manages such related image filtering information 302. Furthermore, the image search server 111 (first information processing device) may further include a candidate image storage unit 221 that stores the unprocessed image and processed image used to derive the related image filtering information 302 as candidate images.

[0082] Furthermore, the related image filtering information derivation unit 211 may derive a short perceptual hash value 322 of the search image and include it in the related image filtering information 302. Furthermore, the related image filtering information derivation unit 211 may include metadata 321 of the search image in the related image filtering information 302. Furthermore, the related image filtering information derivation unit 211 may derive an object category tag 323 of the search image and include it in the related image filtering information 302. Then, the image filtering unit 223 may use the related image filtering information 302 derived by the related image filtering information derivation unit 211 (and the related image filtering information 302 managed by the image filtering information DB 222) to narrow down the candidate images corresponding to the search image from among the candidate images stored in the candidate image storage unit 221.

[0083] In this way, by narrowing down the candidate images based on the related image filtering information 302, it is possible to more easily search for candidate images related to a search image.

[0084] <Configuration Related to Learning Processing> Fig. 6 is a diagram showing an example of the main configuration related to the learning (machine learning) processing of the image search server 111. That is, when Method 1-1 and Method 1-2 are applied, in the learning (machine learning) processing, for example, as shown in Fig. 6, the authenticity determination result acquisition unit 202 acquires authenticity determination result information supplied from the authenticity determination server 112 via the communication unit 201, and supplies the information to the information generation unit 203.

[0085] The related image filtering information derivation unit 211 of the information generation unit 203 uses the unedited image and edited image included in the authenticity determination result information to derive related image filtering information 302 for those images, and supplies the derived related image filtering information together with the images to the candidate image filtering unit 204. The candidate image storage unit 221 of the candidate image filtering unit 204 stores the supplied unedited image and edited image as candidate images. The image filtering information DB 222 of the candidate image filtering unit 204 stores and manages the supplied related image filtering information.

[0086] The similarity information derivation unit 212 of the information generation unit 203 derives similarity information 301 between the unedited image and the edited image included in the authenticity determination result information, and supplies the derived similarity information 301 to the editing determination unit 205. The learning data management DB 231 of the editing determination unit 205 stores and manages the similarity information 301 as learning data (teacher data). The learning unit 232 performs learning (machine learning) using the learning data managed by the learning data management DB 231, and derives a learning model that inputs the similarity information 301 (difference value 311 of long perceptual hash values ​​and feature point matching result 312) and outputs editing-related information (AI editing determination result 313) regarding the editing-related determination result.

[0087] <Learning Process Flow> An example of the flow of such a learning process will be described with reference to the flowchart in Fig. 7. When the learning process starts, the authenticity determination result acquisition unit 202 acquires authenticity determination result information supplied from the authenticity determination server 112 via the communication unit 201 in step S111.

[0088] In step S112 , the authenticity determination result acquisition unit 202 supplies the acquired authenticity determination result information to the information generation unit 203 .

[0089] In step S121, the information generating unit 203 acquires the authenticity determination result information.

[0090] In step S122, the related image filtering information derivation unit 211 of the information generation unit 203 derives the related image filtering information 302 for the unedited image and the edited image included in the acquired authenticity determination result information.

[0091] In step S123 , the information generating unit 203 supplies the derived related image filtering information 302 and the image (the unprocessed image or the processed image) corresponding to the related image filtering information 302 to the candidate image filtering unit 204 .

[0092] In step S131, the candidate image filtering unit 204 acquires the related image filtering information 302 and the image (the unprocessed image and the processed image).

[0093] In step S132, the candidate image storage unit 221 of the candidate image filtering unit 204 stores the acquired images (unprocessed images and processed images) as candidate images. The image filtering information DB 222 of the candidate image filtering unit 204 also stores and manages the acquired related image filtering information 302.

[0094] In step S124, the similarity information derivation unit 212 of the information generation unit 203 derives similarity information 301 between the pre-processed image and the post-processed image using the pre-processed image and the post-processed image contained in the acquired authenticity determination result information.

[0095] In step S125 , the information generating unit 203 supplies the derived similarity information 301 to the manipulation determining unit 205 .

[0096] In step S141, the manipulation determination unit 205 acquires the similarity information 301. Then, the learning data management DB 231 of the manipulation determination unit 205 stores and manages the acquired similarity information 301 as learning data.

[0097] In step S142, the learning unit 232 performs learning using the managed learning data (i.e., the similarity information 301) and constructs a learning model that inputs the similarity information 301 (the difference value 311 of the long perceptual hash value and the feature point matching result 312) and outputs processing relationship information regarding the processing relationship determination result.

[0098] When the process of step S142 is completed, the learning process ends.

[0099] <Configuration Related to Editing Determination Processing> Fig. 8 is a diagram showing an example of the main configuration related to the editing determination processing of the image search server 111. That is, when Method 1-1 and Method 1-2 are applied, in the editing determination processing, for example, as shown in Fig. 8, the search image acquisition unit 241 of the image search unit 206 acquires a search image supplied from the terminal device 113 or the like via the communication unit 201. That is, a request for searching for related images (candidate images) corresponding to the search image is accepted. The candidate image request unit 242 of the image search unit 206 supplies the search image to the information generation unit 203, thereby requesting a search for related images (candidate images) corresponding to the search image.

[0100] The related image filtering information derivation unit 211 of the information generation unit 203 derives related image filtering information 302 for the search image and supplies the derived related image filtering information 302 to the candidate image filtering unit 204. The image filtering unit 223 of the candidate image filtering unit 204 searches for related image filtering information 302 managed by the image filtering information DB 222 that corresponds to the supplied related image filtering information 302. The image filtering unit 223 also selects a candidate image that corresponds to the searched related image filtering information 302 from the candidate images stored in the candidate image storage unit 221. The image filtering unit 223 then supplies the selected candidate image (narrowed down candidate image) to the image search unit 206.

[0101] Image search unit 206 acquires the supplied candidate image in response to the request by candidate image request unit 242. Editing judgment request unit 243 of image search unit 206 requests editing judgment between the search image and the candidate image by supplying the search image and the acquired candidate image to information generation unit 203.

[0102] The similarity information derivation unit 212 of the information generation unit 203 uses the supplied search image and candidate image to derive similarity information 301 between them (a difference value 311 between long perceptual hash values ​​and a feature point matching result 312) and supplies the derived similarity information 301 to the editing determination unit 205. The editing determination unit 205 acquires the similarity information 301 and performs editing determination between the search image and the candidate image using the acquired similarity information 301. For example, the editing relationship information derivation unit 233 of the editing determination unit 205 inputs the acquired similarity information 301 (a difference value 311 between long perceptual hash values ​​and a feature point matching result 312) into a learning model and outputs editing relationship information (an AI editing determination result 313). The editing determination unit 205 supplies the derived editing relationship information to the image search unit 206.

[0103] The image search unit 206 acquires the supplied processing-related information in response to the request from the processing judgment request unit 243. The search result providing unit 244 of the image search unit 206 supplies the acquired processing-related information, the searched candidate images, etc. as search results to the terminal device 113 via the communication unit 201.

[0104] <Flow of Editing Determination Process> An example of the flow of such editing determination process will be described with reference to the flowcharts in Fig. 9 and Fig. 10. When the editing determination process is started, the search image acquisition unit 241 of the image search unit 206 acquires a search image supplied from the terminal device 113 or the like via the communication unit 201 in step S211 of Fig. 9. That is, the search image acquisition unit 241 accepts a request to search for related images (candidate images) corresponding to the search image.

[0105] In step S212, the candidate image request unit 242 of the image search unit 206 supplies the acquired search image to the information generation unit 203, thereby requesting a search for related images (candidate images) corresponding to the search image.

[0106] In step S221, the information generating unit 203 acquires the search image.

[0107] In step S222, the related image filtering information derivation unit 211 of the information generation unit 203 derives the related image filtering information 302 for the acquired search image.

[0108] In step S223, the information generating unit 203 supplies the derived related image filtering information 302 to the candidate image filtering unit 204.

[0109] In step S231, the candidate image filtering unit 204 acquires the related image filtering information 302.

[0110] In step S232, the image filtering unit 223 of the candidate image filtering unit 204 searches for related image filtering information 302 managed by the image filtering information DB 222 that corresponds to the acquired related image filtering information 302. The image filtering unit 223 also selects a candidate image that corresponds to the searched related image filtering information 302 from the candidate images stored in the candidate image storage unit 221.

[0111] In step S233, the candidate image filtering unit 204 supplies the selected candidate images (narrowed candidate images) to the image search unit 206.

[0112] In step S213, the image search unit 206 acquires the supplied candidate images in response to the request by the candidate image request unit 242. When the process of step S213 ends, the process proceeds to FIG.

[0113] In step S251 of FIG. 10, the manipulation judgment requesting unit 243 of the image searching unit 206 supplies the search image and the acquired candidate image to the information generating unit 203, thereby requesting a manipulation judgment between the search image and the candidate image.

[0114] In step S261, the similarity information derivation unit 212 of the information generation unit 203 acquires the supplied search image and candidate images.

[0115] In step S262, the similarity information derivation unit 212 of the information generation unit 203 uses the acquired search image and candidate image to derive similarity information 301 between them (the difference value 311 of the long perceptual hash value and the feature point matching result 312).

[0116] In step S263, the information generating unit 203 supplies the derived similarity information 301 to the manipulation determining unit 205.

[0117] In step S281, the modification determination unit 205 acquires the similarity information 301.

[0118] In step S282, the processing-related information derivation unit 233 of the processing determination unit 205 performs processing determination between the search image and the candidate image using the acquired similarity information 301. For example, the processing-related information derivation unit 233 inputs the acquired similarity information 301 (the difference value 311 of the long perceptual hash value and the feature point matching result 312) into a learning model, and outputs processing-related information (AI processing determination result 313).

[0119] In step S283, the processing determination unit 205 supplies the derived processing-related information to the image search unit 206.

[0120] In step S252, the image search unit 206 acquires the supplied processing-related information in response to the request from the processing judgment request unit 243.

[0121] In step S253, the search result providing unit 244 of the image search unit 206 supplies the acquired processing-related information, the searched candidate images, etc. as search results to the terminal device 113 via the communication unit 201. When the processing of step S253 ends, the processing determination process ends.

[0122] By performing each process in this way and determining the editing relationship, it is possible to grasp the editing relationship during image retrieval, thereby enabling more accurate retrieval of related images that can be used as comparison targets when determining the authenticity of the searched image.

[0123] <Method 1-3> Furthermore, when Method 1 is applied, the search results may be displayed, for example, as shown in the bottom row of the table in Fig. 5. That is, a display image displaying the search results may be generated (Method 1-3).

[0124] For example, the terminal device 113 (second information processing device) may include a display image generation unit 406 that generates a display image indicating the processing relationship between a search image and a candidate image that is a candidate for a related image that has a processing relationship with the search image, and a display unit 407 that displays the display image. Also, in an information processing method of the terminal device 113 (second information processing device), a display image indicating the processing relationship between a search image and a candidate image that is a candidate for a related image that has a processing relationship with the search image may be generated, and the display image may be displayed.

[0125] In this way, by generating and displaying a display image showing the editing relationship between the search image and candidate images that are candidates for related images that have an editing relationship with the search image, the user can understand the editing relationship during image search. Therefore, related images that can be used as comparison targets when determining the authenticity of the search image can be searched for with higher accuracy.

[0126] <Flow of Related Image Search Process> An example of the flow of related image search process executed by the terminal device 113 will be described with reference to the flowchart of FIG.

[0127] When the related image search process is started, the search image selection unit 402 selects a search image in step S401 based on an external instruction supplied from the input unit 401, for example.

[0128] In step S402, the search image supply unit 403 supplies the selected search image to the image search server 111 via the communication unit 404. In response, the search image supply unit 403 requests a search for (candidate images for) related images of the search image.

[0129] In step S403, the search result acquisition unit 405 acquires the search results provided by the image search server 111 as a response to the request via the communication unit 404.

[0130] In step S404, the display image generating unit 406 generates a display image for displaying the search results, that is, the display image includes the contents of the search results.

[0131] In step S405, the display unit 407 displays the generated display image using a display device. That is, the search results are displayed. When the process of step S405 ends, the related image search process ends.

[0132] By performing each process as described above, the terminal device 113 can display not only related images (searched candidate images) of a search image but also the processing relationship between the search image and the candidate images. This allows the user to understand the processing relationship during image search. Therefore, related images that can be used as comparison targets when determining the authenticity of the search image can be searched for with higher accuracy.

[0133] <Display Image Example 1> Fig. 12 is a diagram showing an example of this display image. Display image 500 shown in Fig. 12 is a display image generated by display image generation unit 406, and is an image showing the processing relationship between the search image and candidate images that are candidates for related images in a processing relationship.

[0134] 12, the display image 500 has a search image display area 501 and a candidate image display area 502. The search image display area 501 is an area for displaying information related to the search image. The candidate image display area 502 is an area for displaying information related to the candidate image. The candidate image display area 502 is provided with an AI unrelated related image display area 511, an AI related related image display area 512, and an unrelated similar image display area 513.

[0135] A search image 521 is displayed in the search image display area 501. In other words, the display image 500 includes the search image 521. The search image display area 501 may also display metadata (meta information) of the search image 521. In the example of FIG. 12 , the metadata (meta information) displayed includes the creator (AAA), creation date and time (YYYY-MM-DD), and shooting location (SSS) of the search image 521. Of course, the content of the displayed metadata may be any content and is not limited to the example of FIG. 12 . The search image display area 501 also displays an upload button (UPLOAD) 522. The upload button 522 is a GUI that allows a user to input an upload instruction. That is, for example, when a user selects the search image 521 and presses the upload button 522, the search image 521 is uploaded to the image search server 111, and a search for related images is requested.

[0136] In the candidate image display area 502, search results (candidate images, etc.) of images related to this search image 521 are displayed.

[0137] The AI ​​unrelated related image display area 511 is an area for displaying information about AI unrelated related images. The AI ​​related related image display area 512 is an area for displaying information about AI related related images. The unrelated similar image display area 513 is an area for displaying information about unrelated similar images.

[0138] The AI-unprocessed related images indicate candidate images that have been determined by an image search performed by the image search server 111 to be related images that have no AI processing relationship with the search image 521. In other words, these AI-unprocessed related images have a processing relationship with the search image 521, but do not have an AI processing relationship. In other words, the AI-unprocessed related images are images to which processing other than AI processing has been applied to the search image 521.

[0139] For example, the candidate images determined as the AI ​​unprocessed related images may be displayed in the AI ​​unprocessed related image display area 511. That is, in the example of Fig. 12, the candidate images 531 and 532 displayed in the AI ​​unprocessed related image display area 511 are candidate images determined as the AI ​​unprocessed related images.

[0140] The AI-processed related images indicate candidate images that have been determined as related images having an AI processing relationship with the search image 521 through an image search by the image search server 111. In other words, the AI-processed related images have a processing relationship with the search image 521 and also have an AI processing relationship. In other words, the AI-processed related images are images in which AI processing has been applied to the search image 521.

[0141] For example, the candidate images determined to be AI-processed related images may be displayed in the AI-processed related image display area 512. That is, in the example of Fig. 12, the candidate images 533 and 534 displayed in the AI-processed related image display area 512 are candidate images determined to be AI-unprocessed related images.

[0142] The unrelated similar images indicate candidate images that are not processed relative to the search image 521 and that are subjectively similar (have a relatively high degree of similarity) to the search image 521, as a result of an image search by the image search server 111. In other words, the unrelated similar images are not processed relative to the search image 521. In other words, the unrelated similar images are not images that have been processed relative to the search image 521.

[0143] For example, the candidate image determined to be an unrelated similar image may be displayed in the unrelated similar image display area 513. That is, in the example of Fig. 12, the candidate image 535 displayed in the unrelated similar image display area 513 is a candidate image determined to be an unrelated similar image.

[0144] In this way, the candidate images may be classified into AI-processed related images, AI-processed related images, and unrelated similar images and displayed in the display image 500. In other words, the display image 500 may include candidate images classified into first related images that have a processing relationship with the search image but have no AI processing relationship with the search image, which is the predetermined processing to be detected, second related images that have an AI processing relationship with the search image, and similar images that have no processing relationship with the search image.

[0145] By displaying the image in this way, users can more easily understand the relationship between the search image and the candidate images, and whether or not they have been edited by AI. This allows for more accurate searches of related images that can be used as comparison targets when determining the authenticity of the search image.

[0146] Furthermore, the AI ​​unedited related image display area 511 may display the degree of match between the candidate image (AI unedited related image) and the search image 521. In other words, the display image 500 may include the degree of match between the first related image (AI unedited related image) and the search image 521. In the example of FIG. 12 , the AI ​​unedited related image display area 511 displays the degree of match between the candidate image 531 and the search image 521 (90%) and the degree of match between the candidate image 532 and the search image 521 (70%).

[0147] This display allows users to more easily understand the degree of match between the searched image and the related images that have not been processed by AI, thereby enabling more accurate searches for related images that can be used as comparison targets when determining the authenticity of the searched image.

[0148] Furthermore, the AI-edited related image display area 512 may display the level of editing for the candidate image (AI-edited related image). This editing level indicates the degree of editing applied to the search image 521 to generate the AI-edited related image. In other words, the display image 500 may include the level of editing for the second related image (AI-edited related image). In the example of FIG. 12 , the AI-edited related image display area 512 displays the editing level (20%) for candidate image 533 and the editing level (16%) for candidate image 534.

[0149] By displaying the image in this way, users can more easily understand the level of editing of related images that have been edited by AI, which allows for more accurate searches of related images that can be used as comparison targets when determining the authenticity of a searched image.

[0150] <Display Image Example 2> Fig. 13 is a diagram showing another example of this display image. The display image 600 shown in Fig. 13 is a display image generated by the display image generation unit 406, and is an image showing the processing relationship between the search image and candidate images that are candidates for related images that have a processing relationship with each other. As shown in Fig. 13, the display image 600 has a search image display area 601 and a candidate image display area 602. The search image display area 601 is an area for displaying information related to the search image. The candidate image display area 602 is an area for displaying information related to the candidate images.

[0151] A search image 611 is displayed in the search image display area 601. In other words, the display image 600 includes the search image 611.

[0152] The candidate image display area 602 displays search results (candidate images, etc.) of images related to the search image 611. In the example of Fig. 13, the candidate image display area 602 displays the searched candidate images 621 to 625.

[0153] Furthermore, in the candidate image display area 602, icons may be used to indicate whether or not each candidate image has an AI-processing relationship, which is a predetermined processing that should be detected, with the search image 611. In other words, the displayed image may include a candidate image for which a processing relationship with the search image has been determined, and a first icon indicating whether or not the candidate image has an AI-processing relationship, which is a predetermined processing that should be detected, with the search image. In the example of FIG. 13 , an icon is displayed in the upper left corner of each candidate image from candidate image 621 to candidate image 625, and this icon indicates whether or not the candidate image has an AI-processing relationship with the search image.

[0154] For example, a diagonal line icon is displayed in the upper left corner of each of candidate images 621, 622, and 623. This diagonal line icon indicates that there is an AI processing relationship between that candidate image and search image 611. In other words, this diagonal line icon indicates that candidate image 621, candidate image 622, and candidate image 623 are related images with an AI processing relationship.

[0155] Additionally, a white icon is displayed in the upper left corner of candidate image 624. This white icon indicates that the candidate image has a processing relationship with search image 611, but does not have an AI processing relationship. In other words, this white icon indicates that candidate image 624 is an AI-unprocessed related image.

[0156] Furthermore, no icon is displayed in the upper left corner of candidate image 625. The absence of this icon indicates that there is no processing relationship between the candidate image and search image 611. In other words, the absence of this icon indicates that candidate image 625 is an unrelated similar image.

[0157] By displaying the image in this way, users can more easily understand the relationship between the search image and the candidate images, and whether or not they have been edited by AI. This allows for more accurate searches of related images that can be used as comparison targets when determining the authenticity of the search image.

[0158] Furthermore, in the candidate image display area 602, whether or not each candidate image has undergone AI processing may be indicated by an icon. In other words, the displayed image may further include a second icon indicating whether or not each candidate image has undergone AI processing. In the example of FIG. 13 , an icon is displayed in the upper right corner of each candidate image from candidate image 621 to candidate image 625, and this icon indicates whether or not the candidate image has undergone AI processing. In other words, this icon indicates whether or not the candidate image has undergone AI processing.

[0159] For example, a "R" icon is displayed in the upper right corner of each of candidate images 621, 622, 623, and 625. This icon indicates that the candidate image is not an AI-processed image. In other words, the display image 600 indicates that candidate images 621, 622, 623, and 625 have not been subjected to AI processing. In contrast, an "AI" icon is displayed in the upper right corner of candidate image 624. This icon indicates that the candidate image is an AI-processed image. In other words, the display image 600 indicates that candidate image 624 has been subjected to AI processing.

[0160] By displaying the image in this way, users can more easily understand whether each candidate image has been processed using AI, which allows for more accurate searches of related images that can be used as comparison targets when determining the authenticity of a searched image.

[0161] <Display Image Example 3> Fig. 14 is a diagram showing another example of this display image. Display image 700 shown in Fig. 14 is an image for displaying more detailed information about a candidate image. For example, this display image 700 may be displayed when a user or the like selects a candidate image, and may include detailed information about the selected candidate image.

[0162] 14, a display image 700 has a search image display area 701 and a candidate image display area 702. The search image display area 701 is an area for displaying information about a search image. The candidate image display area 702 is an area for displaying information about a candidate image.

[0163] A search image 711 is displayed in the search image display area 701. In other words, the display image 700 includes the search image 711. The search image display area 701 may also display metadata (meta information) of the search image 711. In the example of FIG. 14 , the metadata (meta information) displayed includes the creator (AAA), creation date and time (YYYY-MM-DD), and shooting location (SSS) of the search image 711. Of course, the content of the displayed metadata may be any content and is not limited to the example of FIG. 14 . The search image display area 701 also displays an upload button (UPLOAD) 712. The upload button 712 is a GUI that allows a user to input an upload instruction. That is, for example, when a user selects the search image 711 and presses the upload button 712, the search image 711 is uploaded to the image search server 111, and a search for related images is requested.

[0164] Candidate images and detailed information about the candidate images are displayed in the candidate image display area 702 as search results for images related to the search image 711. In the example of Fig. 14, a candidate image 721 is displayed in the candidate image display area 702. This candidate image 721 may be, for example, a candidate image selected by a user or the like from the display image 500, the display image 600, or the like. In other words, the display image 700 may include the selected candidate image and detailed information about the candidate image.

[0165] This detailed information may be any information relating to the candidate image. For example, this detailed information may include metadata about the candidate image. That is, the metadata of the candidate image 721 may be displayed in the candidate image display area 702. In the example of FIG. 14 , the metadata (meta information) displayed includes the creator (BBB), creation date and time (yyyy-mm-dd), and shooting location (QQQ) of the candidate image 721. Of course, the content of this metadata may be anything and is not limited to the example of FIG. 14 .

[0166] The detailed information may also include information about the processing performed between the candidate image 721 and the search image 711. That is, the candidate image display area 702 may include information about the processing performed between the candidate image 721 and the search image 711. This information may be derived by a device other than the image search server 111, such as the authenticity determination server 112. For example, the terminal device 113 may acquire this detailed information by supplying the authenticity determination server 112 with a candidate image 721 selected from the candidate images included in the search results. In the example of FIG. 14 , the processing information displayed includes information about the processing performed on the candidate image 721 to obtain the candidate image 721 from the search image 711, such as the degree of similarity (70%) with the search image 711, cropping (10%), rotation (10%), brightness (10%), color (30%), and distortion (0%). Of course, the content of this "information about processing" may be anything and is not limited to the example of FIG. 14 .

[0167] Also displayed in the candidate image display area 702 is a back button (BACK) 722. This back button 722 is a GUI that, when operated by a user or the like, inputs an instruction to return the candidate image 721 to the selected screen (e.g., display image 500, display image 600, etc.). In other words, for example, when the back button 722 is pressed, the display returns to display image 500, display image 600, etc.

[0168] This display allows the user to grasp detailed information about the candidate image, thereby enabling more accurate searches for related images that can be used as comparison targets when determining the authenticity of the searched image.

[0169] <Display Image Example 4> Fig. 15 is a diagram showing another example of this display image. Similar to display image 700, display image 800 shown in Fig. 15 is an image for displaying more detailed information about a candidate image. For example, display image 800 may be displayed when a user or the like selects a candidate image, and may include detailed information about the selected candidate image.

[0170] 15, a display image 800 has a search image display area 801 and a candidate image display area 802. The search image display area 801 is an area for displaying information about a search image. The candidate image display area 802 is an area for displaying information about a candidate image.

[0171] A search image 811 is displayed in the search image display area 801. In other words, the display image 800 includes the search image 811. The search image display area 801 may also display metadata (meta information) of the search image 811. In the example of FIG. 15 , the metadata (meta information) displayed includes the creator (AAA) of the search image 811, the creation date and time (YYYY-MM-DD), the shooting location (SSS), and the like. Of course, the content of the displayed metadata may be any content and is not limited to the example of FIG. 15 . The search image display area 801 also displays an upload button (UPLOAD) 812. The upload button 812 is a GUI that allows a user or the like to input an upload instruction. That is, for example, when a user selects the search image 811 and presses the upload button 812, the search image 811 is uploaded to the image search server 111, and a search for related images is requested.

[0172] Candidate images and detailed information about the candidate images are displayed in the candidate image display area 802 as search results for images related to the search image 711. In the example of Fig. 15, a candidate image 821 is displayed in the candidate image display area 802. This candidate image 821 may be, for example, a candidate image selected by a user or the like from the display image 500, the display image 600, or the like. In other words, the display image 800 may include the selected candidate image and detailed information about the candidate image.

[0173] This detailed information may be any information relating to the candidate image. For example, this detailed information may include metadata about the candidate image. That is, the metadata of the candidate image 821 may be displayed in the candidate image display area 802. In the example of FIG. 15 , the metadata (meta information) displayed includes the photographer, shooting date and time, shooting location, etc. of the candidate image 821. Of course, the content of this metadata may be anything and is not limited to the example of FIG. 15 .

[0174] This detailed information may also include information about AI processing, which is a predetermined processing to be detected between the candidate image 821 and the search image 811. In other words, the candidate image display area 802 may include information about AI processing between the candidate image 821 and the search image 811. This information may be derived by a device other than the image search server 111, such as the authenticity determination server 112. For example, the terminal device 113 may obtain this detailed information by supplying the authenticity determination server 112 with a candidate image 721 selected from the candidate images included in the search results. In the example of FIG. 15 , information about the AI ​​processing applied to the candidate image 821 to obtain it from the search image 811, such as the AI ​​processing level (16%), object change (16%), and feature point coordinate change (0%), is displayed as information about AI processing. Of course, the content of this "information about AI processing" may be anything and is not limited to the example of FIG. 15 .

[0175] By displaying information about AI processing in this way, users can grasp the information about the AI ​​processing applied to the candidate image, which allows for more accurate searches of related images that can be used as comparison targets when determining the authenticity of the searched image.

[0176] Furthermore, the information about this AI processing may include information indicating the areas where AI processing has been applied. In the example of Fig. 15, the areas where AI processing has been applied are indicated by a thick frame 821A on the candidate image 821 in the candidate image display area 802.

[0177] By showing the areas where AI processing has been applied, users can easily understand where AI processing has been applied, which allows for more accurate searches of related images that can be used for comparison when determining the authenticity of a searched image.

[0178] Also displayed in the candidate image display area 802 is a back button (BACK) 822. This back button 822 is a GUI that, when operated by a user or the like, inputs an instruction to return the candidate image 821 to the selected screen (for example, display image 500, display image 600, etc.). In other words, for example, when this back button 822 is pressed, the display returns to display image 500, display image 600, etc.

[0179] <Generation of Display Images by Image Search Server 111> The image search server 111 may generate the display images described above and supply the generated display images to the terminal device 113 as search results.

[0180] In this case, the image search server 111 (the image search unit 206 (FIG. 2)) has a display image generation unit 406 (FIG. 4). The display image generation unit 406 of the image search unit 206 generates a display image as described above (i.e., in the same way as in the case of the terminal device 113) based on the candidate images and processing-related information supplied to the image search unit 206. The search result provision unit 244 supplies the generated display image as a search result to the terminal device 113 via the communication unit 201.

[0181] In this case, the display image generation unit 406 may be omitted from the terminal device 113 ( FIG. 4 ). The search result acquisition unit 405 acquires the display image supplied from the image search server 111 as the search result via the communication unit 404. The search result acquisition unit 405 supplies the display image to the display unit 407. The display unit 407 displays the display image supplied from the search result acquisition unit 405.

[0182] In this case, the same effect as that obtained when the terminal device 113 generates the display image can be obtained.

[0183] <Scope of Application of Explanation> In this specification, an explanation given for a higher-level method also applies to lower-level methods belonging to that method, unless a contradiction occurs.

[0184] <Combination> Furthermore, each of the above-described methods may be applied in combination with any other method as long as no contradiction occurs. Three or more methods may be applied in combination. Furthermore, techniques that can be combined may include not only those shown in the table of FIG. 5 as "methods," but also all elements described in this specification. Furthermore, each of the above-described methods may be applied in combination with methods other than those described above.

[0185] <4. Supplementary Notes> <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that make up the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, for example, that can execute various functions by installing various programs.

[0186] FIG. 16 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0187] In a computer 900 shown in FIG. 16, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0188] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0189] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and an input terminal. The output unit 912 includes, for example, a display, a speaker, and an output terminal. The storage unit 913 includes, for example, a hard disk, a RAM disk, and a non-volatile memory. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0190] In a computer configured as described above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0191] The program executed by the computer can be applied by recording it on, for example, a removable medium 921 such as a package medium. In this case, the program can be installed in the storage unit 913 via the input / output interface 910 by inserting the removable medium 921 into the drive 915.

[0192] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0193] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0194] <Application of the Present Technology> The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0195] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a set in which other functions are added to a unit (e.g., a video set).

[0196] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0197] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0198] <Fields and uses to which this technology can be applied> Systems, devices, processing units, etc. to which this technology is applied can be used in any field, for example, transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, nature monitoring, etc. In addition, the uses thereof are also arbitrary.

[0199] <Others> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. That is, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be included not only in a bitstream, but also in a bitstream that includes differential information of the identification information relative to certain reference information. Therefore, in this specification, "flag" and "identification information" encompass not only the information itself, but also differential information relative to the reference information.

[0200] Furthermore, various types of information (e.g., metadata) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, the associated pieces of data may be combined into one piece of data or may be separate pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Furthermore, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer not to the entire data, but to only part of the data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0201] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0202] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0203] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0204] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0205] Also, for example, each step of a single flowchart may be executed by a single device, or may be shared and executed by multiple devices. Furthermore, when a single step includes multiple processes, the multiple processes may be executed by a single device, or may be shared and executed by multiple devices. In other words, multiple processes included in a single step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0206] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0207] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0208] The present technology may also be configured as follows. (1) An information processing device including: a similarity information derivation unit that derives similarity information regarding data similarity between a search image and a candidate image that is a candidate for a related image in a processing relationship with the search image; and a processing determination unit that determines the processing relationship between the search image and the candidate image based on the similarity information. (2) The information processing device described in (1), in which the similarity information includes a matching result of feature points between the search image and the candidate image. (3) The information processing device described in (1) or (2), in which the similarity information includes a difference in perceptual hash values ​​between the search image and the candidate image. (4) The information processing device described in any of (1) to (3), in which the processing determination unit determines the processing relationship between the search image and the candidate image using a learning model that inputs the similarity information and outputs processing relationship information regarding the determination result of the processing relationship. (5) The information processing device described in (4), in which the processing relationship information includes a determination result of the presence or absence of AI processing, which is a predetermined processing to be detected, between the search image and the candidate image. (6) The information processing device according to (4) or (5), wherein the processing-related information includes a similarity between the search image and the candidate image. (7) The information processing device according to any of (4) to (6), wherein the processing determination unit performs learning using the similarity information between the before-processing image and the after-processing image as training data, and derives the learning model. (8) The information processing device according to (7), wherein the similarity information includes a determination result of whether or not AI processing, which is a predetermined processing to be detected, has occurred between the before-processing image and the after-processing image. (9) The information processing device according to (7) or (8), wherein the similarity information includes a matching result of feature points between the before-processing image and the after-processing image. (10) The information processing device according to any of (7) to (9), wherein the similarity information includes a difference in perceptual hash values ​​between the before-processing image and the after-processing image.(11) The information processing device according to any one of (1) to (10), further comprising an image filtering unit that narrows down the candidate images using related image filtering information for narrowing down the candidate images, wherein the processing determination unit is configured to determine the processing relationship between the narrowed down candidate images and the search image. (12) The information processing device according to (11), wherein the related image filtering information includes a perceptual hash value of an image. (13) The information processing device according to (11) or (12), wherein the related image filtering information includes image metadata. (14) The information processing device according to any one of (11) to (13), wherein the related image filtering information includes a category tag of an image. (15) The information processing device according to any one of (1) to (14), further comprising a display image generation unit that generates a display image indicating the determined processing relationship between the search image and the candidate images. (16) The information processing device according to (15), wherein the display image includes the candidate images classified into: a first related image that has the processing relationship with the search image but does not have a relationship of AI processing, which is a predetermined processing to be detected, with the search image; a second related image that has the AI ​​processing relationship with the search image; and a similar image that does not have the processing relationship with the search image. (17) The information processing device according to (16), wherein the display image further includes a degree of match between the first related image and the search image. (18) The information processing device according to (16) or (17), wherein the display image further includes a processing level for the second related image. (19) The information processing device according to any of (16) to (18), wherein the display image further includes the search image. (20) The information processing device according to any of (15) to (19), wherein the display image includes the candidate image for which the processing relationship with the search image has been determined, and a first icon indicating whether or not there is a relationship of AI processing, which is a predetermined processing to be detected, between the candidate image and the search image. (21) The information processing device according to (20), wherein the display image further includes a second icon indicating whether or not the AI ​​processing has been performed for each of the candidate images.(22) The information processing device according to (20) or (21), wherein the display image further includes the search image. (23) The information processing device according to any one of (15) to (22), wherein the display image includes the selected candidate image and detailed information about the candidate image. (24) The information processing device according to (23), wherein the detailed information includes metadata about the candidate image. (25) The information processing device according to (23) or (24), wherein the detailed information includes information about AI processing, which is a predetermined processing to be detected between the candidate image and the search image. (26) The information processing device according to (25), wherein the information about AI processing includes information indicating a portion where the AI ​​processing has been applied. (27) An information processing method, comprising: deriving similarity information about data similarity between a search image and a candidate image that is a candidate for a related image in a processing relationship with the search image; and determining the processing relationship between the search image and the candidate image based on the similarity information.

[0209] (31) An information processing device comprising: a display image generation unit that generates a display image indicating the processing relationship between a search image and candidate images that are candidates for related images that have a processing relationship with the search image; and a display unit that displays the display image. (32) The information processing device described in (31), wherein the display image includes the candidate images classified into: a first related image that has the processing relationship with the search image and does not have an AI processing relationship with the search image, which is a predetermined processing to be detected; a second related image that has the AI ​​processing relationship with the search image; and a similar image that does not have the processing relationship with the search image. (33) The information processing device described in (32), wherein the display image further includes a degree of match between the first related image and the search image. (34) The information processing device described in (32) or (33), wherein the display image further includes a processing level for the second related image. (35) The information processing device described in any of (32) to (34), wherein the display image further includes the search image. (36) The information processing device according to any one of (31) to (35), wherein the display image includes the candidate image for which the processing relationship with the search image has been determined, and a first icon indicating the presence or absence of a relationship of AI processing, which is a predetermined processing to be detected, between the candidate image and the search image. (37) The information processing device according to (36), wherein the display image further includes a second icon indicating the presence or absence of the AI ​​processing for each of the candidate images. (38) The information processing device according to (36) or (37), wherein the display image further includes the search image. (39) The information processing device according to any one of (31) to (38), wherein the display image includes the selected candidate image and detailed information regarding the candidate image. (40) The information processing device according to (39), wherein the detailed information includes metadata of the candidate image. (41) The information processing device according to (39) or (40), wherein the detailed information includes information regarding AI processing, which is a predetermined processing to be detected between the candidate image and the search image. (42) The information processing device according to (41), wherein the information relating to the AI ​​processing includes information indicating a location where the AI ​​processing has been performed.(43) An information processing method, comprising: generating a display image indicating a processing relationship between a search image and a candidate image that is a candidate for a related image in a processing relationship; and displaying the display image.

[0210] REFERENCE SIGNS LIST 100 Image retrieval system, 110 Network, 111 Image retrieval server, 112 Authenticity determination server, 113 Terminal device, 201 Communication unit, 202 Authenticity determination result acquisition unit, 203 Information generation unit, 204 Candidate image filtering unit, 205 Editing determination unit, 206 Image retrieval unit, 211 Related image filtering information derivation unit, 212 Similarity information derivation unit, 221 Candidate image storage unit, 222 Image filtering information DB, 223 Image filtering unit, 231 Learning data management DB, 232 Learning unit, 233 Editing-related information derivation unit, 241 Retrieved image acquisition unit, 242 Candidate image request unit, 243 Editing determination request unit, 244 Search result provision unit, 401 Input unit, 402 Retrieved image selection unit, 403 Search image supply unit, 404 communication unit, 405 search result acquisition unit, 406 display image generation unit, 407 display unit, 900 computer

Claims

a similarity information deriving unit that derives similarity information regarding data similarity between a search image and a candidate image that is a candidate for a related image that has a processing relationship with the search image; a modification determination unit that determines the modification relationship between the search image and the candidate image based on the similarity information; An information processing device comprising: 　 The similarity information includes a matching result of feature points between the search image and the candidate image. The information processing device according to claim 1 . 　 The similarity information includes a difference in perceptual hash values ​​between the search image and the candidate image. The information processing device according to claim 1 . 　 The processing determination unit determines the processing relationship between the search image and the candidate image using a learning model that receives the similarity information and outputs processing relationship information related to the determination result of the processing relationship. The information processing device according to claim 1 . 　 The processing-related information includes a determination result of whether or not AI processing, which is a predetermined processing to be detected, exists between the search image and the candidate image. The information processing device according to claim 4 . 　 The processing-related information includes a similarity between the search image and the candidate image. The information processing device according to claim 4 . 　 The processing determination unit performs learning using the similarity information between the unprocessed image and the processed image as training data, and derives the learning model. The information processing device according to claim 4 . 　 The similarity information includes a determination result of whether or not AI processing, which is a predetermined processing to be detected, exists between the unprocessed image and the processed image. The information processing device according to claim 7 . 　 The similarity information includes a matching result of feature points between the unprocessed image and the processed image. The information processing device according to claim 7 . 　 The similarity information includes a difference in perceptual hash value between the unprocessed image and the processed image. The information processing device according to claim 7 . 　 an image filtering unit that narrows down the candidate images using related image filtering information for narrowing down the candidate images; The modification determination unit is configured to determine the modification relationship between the narrowed-down candidate images and the search image. The information processing device according to claim 1 . 　 The associated image filtering information includes a perceptual hash value of the image. The information processing device according to claim 11. 　 The associated image filtering information includes image metadata. The information processing device according to claim 11. 　 The related image filtering information includes a category tag of the image. The information processing device according to claim 11. 　 Deriving similarity information regarding the data similarity between the search image and a candidate image that is a candidate for a related image that has a processing relationship with the search image; The processing relationship between the search image and the candidate image is determined based on the similarity information. Information processing methods. 　 a display image generating unit that generates a display image indicating the processing relationship between the search image and a candidate image that is a candidate for a related image having a processing relationship; a display unit that displays the display image; An information processing device comprising: 　 The display image is a first related image that has the processing relationship with the search image and does not have an AI processing relationship with the search image, which is a predetermined processing to be detected; A second related image that has the AI ​​processing relationship with the search image; A similar image that has no processing relationship with the searched image The candidate images are classified as The information processing device according to claim 16. 　 The display image includes the candidate image whose processing relationship with the search image has been determined, and a first icon indicating whether or not there is a relationship of AI processing, which is a predetermined processing to be detected, between the candidate image and the search image. The information processing device according to claim 16. 　 The display image includes the selected candidate image and detailed information about the candidate image. The information processing device according to claim 16. 　 generating a display image showing the processing relationship between the search image and candidate images that are candidates for related images in a processing relationship; Display the display image Information processing methods.

Images