Image processing apparatus and method

Abstract

The present disclosure relates to an image processing device and method capable of more accurately determining the authenticity of an image. A partial region is generated by dividing an entire region of the image, and the estimated 3D information and the sensor 3D information are compared using the partial region. The estimated 3D information is 3D information estimated based on the image, and the sensor 3D information is 3D information acquired by the sensor and associated with the image. The present disclosure can be applied to, for example, an image processing device, an information processing device, an electronic apparatus, an image processing method, an information processing method, a program, an image processing system, etc.

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus and method, and more particularly, to an image processing apparatus and method capable of more accurately determining the authenticity of an image. Background Art

[0002] Conventionally, a method has been proposed in which a captured image or the like is converted into a hash value in a digital camera or the like, and an electronic signature using the hash value is added to the captured image for falsification detection of the captured image. However, in this method, a fake image generated by so-called trick shooting or the like cannot be detected.

[0003] Therefore, a method of detecting a false image based on the consistency between information indicating the focal length at the time of imaging and the focal length obtained from the captured image is considered (see, for example, Patent Document 1). In addition, a method of detecting a false image by determining whether the subject of the captured image is a plane based on multi-point distance measurement data of a camera is considered (see, for example, Patent Document 2).

[0004] Reference List

[0005] Patent Literature

[0006] Patent Document 1: International Publication No. 2020 / 246166

[0007] Patent Document 2: Japanese Patent Application Publication No. 2002-198958 Summary of the invention

[0008] Problems to be solved by the present invention

[0009] However, in the method described in Patent Document 1, only the consistency of the focal length is determined, and it is not possible to confirm the consistency of the unevenness of the subject, etc. In addition, in the method described in Patent Document 2, it is only determined whether the subject is a plane, and it is not possible to confirm the fine consistency between the distance measurement data and the unevenness of the subject, etc.

[0010] The present disclosure has been made in view of such circumstances, and an object thereof is to enable the authenticity of an image to be determined more accurately. Solution to the problem

[0011] An image processing device according to one aspect of the present technology is an image processing device comprising: a region division unit configured to generate a partial region by dividing an entire region of an image; and a comparison unit configured to compare estimated 3D information with sensor 3D information using the partial region, wherein the estimated 3D information is 3D information estimated based on the image, and the sensor 3D information is 3D information acquired by a sensor and associated with the image.

[0012] An image processing method according to one aspect of the present technology is an image processing method comprising: generating a partial area by dividing an entire area of ​​an image; and comparing estimated 3D information with sensor 3D information using the partial area, wherein the estimated 3D information is 3D information estimated based on the image, and the sensor 3D information is 3D information acquired by a sensor and associated with the image.

[0013] In an image processing device and method according to one aspect of the present technology, a partial area is generated by dividing the entire area of ​​an image, and the estimated 3D information is compared with the sensor 3D information using the partial area. The estimated 3D information is 3D information estimated based on the image. In addition, the sensor 3D information is 3D information acquired by the sensor and associated with the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a diagram illustrating a main configuration example of an authenticity determination apparatus.

[0015] Figure 2 is a diagram illustrating a main configuration example of an image file.

[0016] Figure 3 is a diagram illustrating an example of a main image.

[0017] Figure 4 is a diagram illustrating an example of estimating 3D information.

[0018] Figure 5 is a diagram illustrating an example of a region division state.

[0019] Figure 6 is a diagram illustrating an example of a region division state.

[0020] Figure 7 is a diagram illustrating an example of sensor 3D information.

[0021] Figure 8 is a diagram illustrating an example of presentation information.

[0022] Fig. 9 is a diagram illustrating an example of presentation information.

[0023] Fig.10 is a flowchart illustrating an example of the flow of authenticity determination processing.

[0024] Fig.11 is a flowchart illustrating an example of the flow of authenticity determination processing.

[0025] Fig.12 is a flowchart illustrating an example of the flow of authenticity determination processing.

[0026] Fig.13 is a flowchart illustrating an example of the flow of authenticity determination processing.

[0027] Fig.14 is a flowchart illustrating an example of the flow of authenticity determination processing.

[0028] Fig.15 is a flowchart illustrating an example of the flow of authenticity determination processing.

[0029] Fig.16 is a flowchart illustrating an example of the flow of authenticity determination processing.

[0030] Fig.17 is a diagram for explaining a case where the authenticity determination target is a processed image.

[0031] Fig.18 is a flowchart illustrating an example of the flow of authenticity determination processing.

[0032] Fig.19 is a diagram illustrating an example of presentation information.

[0033] Fig. 20 is a diagram illustrating an example of a reduced image.

[0034] Fig.21 is a flowchart illustrating an example of the flow of authenticity determination processing.

[0035] Fig. 22 is a block diagram illustrating a main configuration example of an image processing system.

[0036] Fig.23 is a block diagram illustrating a main configuration example of a computer. DETAILED DESCRIPTION

[0037] Next, a mode for practicing the present disclosure (hereinafter, referred to as an embodiment) will be described. Note that the description is given in the following order.

[0038] 1. Image authenticity determination

[0039] 2. Use partial area image authenticity determination

[0040] 3. Determination of authenticity of processed images

[0041] 4. Determination of the authenticity of moving images

[0042] 5. Embodiment (Image processing system)

[0043] 6. Additional notes

[0044] <1. Determination of the authenticity of the image>

[0045] Conventionally, the following method has been proposed: a captured image or the like is converted into a hash value in a digital camera or the like, and an electronic signature using the hash value is added to the captured image for forgery detection of the captured image. However, in order to ensure the authenticity of the image (whether the image is a correct image), it is necessary to check not only whether the image is forged but also whether there is fraud at the time of creation. In the above method, fraud at the time of creation (for example, a pseudo image generated by so-called trick shooting or the like) cannot be detected. A pseudo image is a captured image in which a non-existent situation appears to exist, that is, a captured image of a non-existent situation that looks like a captured image obtained by capturing a real situation. Trick shooting refers to a shooting technique that generates a pseudo image by using a tool, designed shooting, etc.

[0046] Therefore, for example, as described in Patent Document 1, a method of detecting a false image based on the consistency between information indicating the focal length at the time of imaging and the focal length obtained from the captured image is considered. In addition, for example, as described in Patent Document 2, a method of detecting a false image by determining whether the subject of the captured image is a plane based on multi-point distance measurement data of a camera is considered.

[0047] However, in these methods, there is a possibility that the detection of a false image becomes inaccurate. For example, in the method described in Patent Document 1, only the consistency of the focal length is determined, and the consistency of the unevenness of the subject, etc. cannot be confirmed. Therefore, for example, even in the case of a false image of a subject that is generated by capturing a facial photo or the like and pretends to be a photo captured, if the focal length matches the metadata, it is difficult to detect the false image by the method described in Patent Document 1.

[0048] Furthermore, Patent Document 2 does not specifically describe how to detect the contradiction between the image and the distance measurement data, and only describes determining whether the subject is a plane. Therefore, in this method, it is not possible to confirm the fine consistency between the distance measurement data and the unevenness of the subject, etc. For example, by capturing an image of a person as a main subject in front of a flat plate on which a background is drawn, a pseudo image can be generated as if the person exists at the place drawn on the flat plate. Then, in the case of such a trick shot, distance measurement data that is not a plane (distance measurement data in which at least a part of the person is not a plane) can be generated. In the method described in Patent Document 2, it is difficult to detect such a trick shot.

[0049] As mentioned above, there is room for improvement in the accuracy of image authenticity determination.

[0050] <2. Image authenticity determination using partial areas>

[0051] Therefore, when determining the authenticity of the image, the interior of the image is divided into a plurality of partial areas, and the 3D information (depth) of the partial areas is compared. Note that, below, the 3D information indicates information indicating the distance (depth) from the imaging position to the position of the subject (object in the image frame). The 3D information includes distance information (information indicating the distance from the imaging position to the body position) for each pixel in the frame or for each small area including a plurality of pixels. Note that the 3D information can be any information as long as it includes information that allows the distance to be derived. That is, the 3D information can directly indicate the distance, such as length, etc. In addition, the 3D information can indirectly indicate the distance, such as phase, coordinates, etc.

[0052] For example, the image processing device may include a region division unit and a comparison unit, the region division unit being configured to generate a partial region by dividing the entire region of the image, and the comparison unit being configured to compare the estimated 3D information with the sensor 3D information using the partial region. In addition, in the image processing method, the partial region may be generated by dividing the entire region of the image, and the estimated 3D information may be compared with the sensor 3D information using the partial region. Note that the estimated 3D information is 3D information estimated based on the image. In addition, the sensor 3D information is 3D information acquired by the sensor and associated with the image. Note that the generation of the partial region may be to extract an image of the partial region from the entire region of the image, or may be to specify the address of a pixel in a frame at the boundary of the partial region (i.e., to enable extraction). In addition, below, "partial region" not only indicates (the range of) the partial region, but also indicates the image in the partial region in some cases.

[0053] Since the use of the partial area in this manner enables a more detailed comparison of the image with the 3D information, the authenticity of the image can be more accurately determined based on the comparison result.

[0054] <Authenticity Determination Device>

[0055] Figure 1 : is a block diagram illustrating a configuration example of an authenticity determination device as an aspect of an image processing apparatus to which the present technology is applied. Figure 1 The authenticity determination device 100 illustrated in FIG. 1 is a device for determining the authenticity of an image (captured image). Figure 1 In the figure, the main parts of the processing unit, data flow, etc. are illustrated, and Figure 1 That is, the authenticity determination apparatus 100 may include those not shown in the figure. Figure 1 In addition, there may be a device or processing unit that is not shown in the figure. Figure 1 The arrows in the figure represent the data or processing flow.

[0056] like Figure 1 As illustrated in FIG. 1 , the authenticity determination apparatus 100 includes a control unit 101 , an image analysis engine 111 , and an input / output interface unit 112 .

[0057] The control unit 101 controls the image analysis engine 111 and the input / output interface unit 112 .

[0058] The image analysis engine 111 includes an image file acquisition unit 131, a 3D information estimation unit 132, an area division unit 133, a sensor 3D information acquisition unit 134, a comparison unit 135, and a rendering processing unit 136. The input / output interface unit 112 includes an input unit 151, a rendering unit 152, a storage unit 153, a communication unit 154, and a driver 155.

[0059] The image file acquisition unit 131 performs processing related to the acquisition of an image file. An image file is a file container that stores image data. Hereinafter, image data is also simply referred to as an image.

[0060] For example, the image file acquisition unit 131 may acquire an image file including an image as an authenticity determination target via the input / output interface unit 112. For example, the image file acquisition unit 131 may read and acquire an image file stored in the storage unit 153. For example, the image file acquisition unit 131 may request the storage unit 153 for an image file storing an image whose authenticity is to be determined, to acquire the image file. In addition, the image file acquisition unit 131 may acquire an image file supplied from another device via the communication unit 154. For example, the image file acquisition unit 131 may request another device for an image file storing an image whose authenticity is to be determined, to acquire the image file, via the communication unit 154. In addition, the image file acquisition unit 131 may read and acquire an image file recorded on a removable recording medium (not shown) mounted on the drive 155. For example, the image file acquisition unit 131 may request the drive 155 for an image file storing an image whose authenticity is to be determined, and cause the drive 155 to read the image file from the removable recording medium, to acquire the image file.

[0061] The image file acquisition unit 131 may include a storage medium, and cause the storage medium to store (hold) the acquired image file. Note that the storage medium may be of any type. For example, the storage medium may be a magnetic recording medium such as a hard disk or a semiconductor memory such as a random access memory (RAM).

[0062] Figure 2 is a diagram illustrating a main configuration example of an image file. Figure 2 The image file 200 illustrated in FIG. 2 stores a main image 211 and a reduced image 212. The main image 211 may be a compression-encoded image such as, for example, a Joint Photographic Experts Group (JPEG) image, may be a non-compression-encoded image such as, for example, a RAW image or a YUV image, or may be both of them. The reduced image 212 may be an image (e.g., a JPEG image) obtained by compressing and encoding a YUV image reduced by a resizing process, may be a reduced YUV image, or may be both of them.

[0063] Furthermore, in the image file 200, standard metadata 213 and additional metadata 214 are stored as metadata. The standard metadata 213 includes, for example, items defined by a standard, etc. The additional metadata 214 includes items not included in the standard metadata 213 such as items set by a manufacturer.

[0064] For example, the additional metadata 214 may include sensor 3D information 221, image capture information 222, unique device ID 223, and signature 224. The sensor 3D information 221 includes 3D information about an object (subject, etc.) in the image frame of the main image 211, and is 3D information corresponding to the main image 211. The sensor 3D information 221 is detected by a sensor and associated with the main image 211. The sensor may be any sensor such as, for example, a distance measurement sensor as long as 3D information can be detected (generated), and its detection method may be freely determined. For example, the sensor may acquire the sensor 3D information 221 from an optical image on the same optical axis as the optical axis of the main image 211 acquired by the image sensor. Note that the optical axis refers to a principal ray passing through the center of a light beam in an optical system that passes through the entire system. “Acquiring the sensor 3D information 221 from an optical image on the same optical axis as the optical axis of the main image 211” means that the optical axis on which the optical image of the main image 211 is obtained and the optical axis on which the sensor 3D information 221 is obtained are identical to each other. For example, in an imaging device, the main image 211 and the sensor 3D information 221 may be acquired from one optical image. Furthermore, in an imaging device, one optical image may be divided into two optical images (optical images identical to each other) by a beam splitter (half mirror) or the like using a prism or the like, the main image 211 may be acquired from one optical image, and the sensor 3D information 221 may be acquired from the other optical image. In this manner, the optical axis of the optical image used to acquire the main image 211 and the optical axis of the optical image used to acquire the sensor 3D information 221 are identical to each other. Therefore, with respect to a subject within the range (angle of view) of the scene included in the main image 211, the sensor 3D information 221 of the subject may be acquired from the same angle as in the case of the main image 211. That is, “the sensor 3D information 221 is acquired from the optical image on the same optical axis as that of the main image 211” can be said to indicate that, with respect to a subject within the angle of view of the main image 211, the sensor 3D information 221 is acquired for the subject from the same angle as in the case of the main image 211. Note that the range of the sensor 3D information 221 acquired from the optical image can be freely determined. For example, the range of the sensor 3D information 221 may be the same as the angle of view of the main image 211, or may be a range that includes part or all of the angle of view. For example, for a plurality of positions within the angle of view of the main image 211, the sensor 3D information 221 may be obtained from the same angle as in the case of the main image 211. The sensor 3D information 221 may be associated with the main image 211 by, for example, a device that generates the image file 200. For example, an imaging device that images a subject and generates the main image 211 may generate the sensor 3D information 221 and associate the sensor 3D information 221 with the main image 211.

[0065] The image capture information 222 includes information about image capture of the subject. The unique device ID 223 includes a unique ID of a device (such as an imaging device) that has generated the image file 200 (or the main image 211). The signature 224 is an electronic signature corresponding to the main image 211 and the sensor 3D information 221. The signature 224 includes information that encrypts at least a hash value of the main image 211 and the sensor 3D information 221 using predetermined key information. The key information may be a secret key corresponding to the device that generates the signature 224, or may be a public key shared with a device that confirms the signature 224 (a method in which the signing party and the confirming party use the same key). The signature 224 may be generated by, for example, a device that generates the image file 200. For example, the imaging device that images the subject and generates the main image 211 may generate the signature 224.

[0066] Note that, for example, in the case where both a RAW image and a JPEG image are stored as a main image 211 in the image file 200, as the signature 224, the signature of the RAW image may be stored in the image file 200, the signature of the JPEG image may be stored in the image file 200, or the signatures of the RAW image and the JPEG image may be stored in the image file 200.

[0067] Back to Figure 1 , the 3D information estimation unit 132 performs processing related to the estimation of 3D information about the main image. For example, the 3D information estimation unit 132 may acquire the main image stored in the image file held by the image file acquisition unit 131. In addition, the 3D information estimation unit 132 may generate estimated 3D information corresponding to the main image. The estimated 3D information is 3D information about an object (subject, etc.) in the main image (image frame) estimated based on the main image. That is, the 3D information estimation unit 132 may estimate the 3D information (depth) of an object (subject, etc.) in the main image (image frame) based on the main image.

[0068] For example, suppose Figure 2 The main image 211 included in the image file 200 is as follows Figure 3 The same image as in the example shown in . Figure 3 In the example of , objects such as a person 241, a person 242, and a person 243 are included (appear) in the main image 211. The person 241 is a main object as a subject. The person 242 and the person 243 are objects constituting the background.

[0069] Figure 4 The estimated 3D information 251 illustrated in FIG. 1 indicates the estimated 3D information generated by the 3D information estimation unit 132 and corresponding to Figure 3251. Each polygon in the estimated 3D information 251 indicates estimated 3D information (distance from the imaging position) of the position of the polygon in the estimated main image 211. Each polygon indicates that as the number of corners is smaller, the object in its area is farther away from the imaging position. In other words, each polygon indicates that as the number of corners is larger, the object in the area is closer to the imaging position. The resolution of the estimated 3D information 251 can be freely determined, but unless the data amount, processing load, etc. are taken into consideration, a higher resolution (for example, the same resolution as that of the main image 211) is desired.

[0070] As in Figure 4 As illustrated in the example of , in the estimated 3D information 251, the person 241 is estimated to be located at a location (mainly a decagon) closest to the imaging location. In addition, the person 242 is estimated to be located at a location (mainly a heptagon) farther from the imaging location than the person 241, and the person 243 is estimated to be located at a location (mainly a quadrangle) farther from the imaging location.

[0071] exist Figure 4 , the content (picture) of the main image 211 is indicated by a dotted line for the sake of explanation, but this information about the main image 211 may not be included in the actual estimated 3D information 251. That is, the estimated 3D information 251 only needs to include estimated 3D information of each position indicated by the polygon.

[0072] Note that the method by which the 3D information estimating unit 132 estimates the 3D information can be freely determined. For example, the 3D information estimating unit 132 may generate estimated 3D information corresponding to the main image by inputting the main image to a neural network which is input with the image and outputs estimated 3D information corresponding to the image. The 3D information estimating unit 132 may supply the generated estimated 3D information to the region dividing unit 133. In addition, the 3D information estimating unit 132 may supply the generated estimated 3D information to the comparing unit 135.

[0073] The region division unit 133 performs processing related to region division of the main image. For example, the region division unit 133 may generate a partial region by dividing the entire region of the main image. At this time, the region division unit 133 may acquire the main image stored in the image file held by the image file acquisition unit 131, and perform region division based on the acquired main image.

[0074] In this case, the region division method can be freely determined. For example, the region division unit 133 can detect a main object as a subject in the main image, and divide the entire region of the main image based on the detection result. In addition, in this case, the region division unit 133 can generate an object region and a non-object region, the object region is a partial region including the detected object, and the non-object region is a partial region not including the object.

[0075] For example, the area division unit 133 may be as in Figure 5 The same division as in the example Figure 3 In the example of FIG. Figure 5 In the figure, the thick lines indicate the boundaries of the partial regions. Figure 5 In the case of the example of , a partial region 261 including the person 241, a partial region 262 including the person 242, a partial region 263 including the person 243, and a partial region 264 other than these are generated. Note that in Figure 5 , the content (picture) of the main image 211 is indicated by a dotted line.

[0076] In the case of this example, the region dividing unit 133 detects the person 241 , the person 242 , the person 243 as objects, and generates partial regions (partial region 261 , partial region 262 , and partial region 263 ) and another partial region 264 including the respective persons.

[0077] At this time, the region division unit 133 can identify the partial region including the object as the object region, and identify the partial region not including the object as the non-object region. The allocation method of the object region and the non-object region can be freely determined. The number of partial regions set as the object region can be freely determined. Similarly, the number of partial regions set as the non-object region can also be freely determined.

[0078] For example, in Figure 5 In the example of , the region division unit 133 may detect the person 241 as the main object as the subject, set the partial region 261 as the object region, and set the other partial regions 262 to 264 as the non-object region. In addition, the region division unit 133 may detect the person 241 and the person 242 as the main objects as the subjects, set the partial regions 261 and the partial regions 262 as the object regions, and set the partial regions 263 and the partial regions 264 as the non-object regions. In addition, the region division unit 133 may detect the person 241 to 243 as the main objects as the subjects, set the partial regions 261 to 263 as the object regions, and set the partial region 264 as the non-object region. Of course, there may be other modes.

[0079] Note that the way to set a partial area is not limited to Figure 5 .

[0080] For example, the shape of the partial area can be freely determined. Figure 5 In the example of , the partial areas 261 to 263 are set to rectangles, but the boundaries of the partial areas can be set along the outer shape (contour) of the detected object, for example. For example, the shape of the partial area 261 can be the same as the shape of the person 241. In order to compare 3D information using partial areas, it is preferable to more accurately segment the object area and the non-object area, and it is preferable to set the boundaries of the partial areas along the outer shape (contour) of the detected object. In addition, the number of partial areas generated can be freely determined. For example, the entire area of ​​the main image 211 can be divided into both the partial area 261 and the other partial areas. In addition, multiple objects can be included in one partial area. For example, a partial area including all people 241 to 243 can be generated. In addition, part or all of a certain partial area can overlap with another partial area.

[0081] In addition, the region division unit 133 may detect the main object as the subject by any method. For example, the region division unit 133 may detect the object by performing pattern analysis in the main image. In addition, the region division unit 133 may detect the object using artificial intelligence (AI). For example, the region division unit 133 may detect the object by inputting the main image to a neural network, which is input with the image and outputs a detection result of the main object as the subject in the image.

[0082] Furthermore, the region division unit 133 may acquire the estimated 3D information supplied from the 3D information estimation unit 132 and perform region division based on the acquired estimated 3D information.

[0083] In this case, the region division method may be freely determined. For example, the region division unit 133 may divide the entire region of the main image by clustering the estimated 3D information.

[0084] exist Figure 6 In the example shown in FIG. 2 , the region dividing unit 133 estimates the 3D information 251 ( Figure 4 ) is clustered, and the entire area of ​​the main image is divided based on the clustering result thereof. As a result, partial areas 271 to 277 are generated.

[0085] The method for the clustering can be freely determined. For example, the region division unit 133 can perform the clustering using artificial intelligence (AI). For example, by inputting the estimated 3D information to a neural network that is input with the estimated 3D information and outputs a clustering result, the region division unit 133 can derive a clustering result, and divide the entire region of the main image by using the clustering result.

[0086] Note that, in addition, in the case of dividing the region of the main image based on the estimated 3D information as described above, an object region or a non-object region may be set. For example, by inputting the estimated 3D information into the neural network, the region dividing unit 133 may generate an object region that is a partial region including a main object as a subject in the main image, and a non-object region that is a partial region that does not include an object.

[0087] The region dividing unit 133 may supply information indicating the partial region generated as described above (information indicating the division result of the entire region of the main image) to the comparing unit 135 .

[0088] The sensor 3D information acquisition unit 134 performs processing related to acquisition of sensor 3D information. For example, the sensor 3D information acquisition unit 134 may acquire sensor 3D information stored in an image file held by the image file acquisition unit 131. In addition, the sensor 3D information acquisition unit 134 may supply the acquired sensor 3D information to the comparison unit 135.

[0089] Figure 7 The sensor 3D information 221 shown in FIG. 2 is the same as the main image 211 ( Figure 3 ) is an example of sensor 3D information corresponding to the estimated 3D information 251. Similar to the case of the estimated 3D information 251, each polygon in the sensor 3D information 221 indicates 3D information about the position of the polygon in the main image 211 (the distance from the imaging position). As described above, this 3D information is detected by the sensor. Each polygon indicates that as the number of corners decreases, the object in its area is farther away from the imaging position. In other words, each polygon indicates that as the number of corners increases, the object in the area is closer to the imaging position. The resolution of the sensor 3D information 221 can be freely determined, but unless the amount of data, processing load, etc. are taken into consideration, a higher resolution (for example, the same resolution as that of the main image 211) is desired.

[0090] Note that in Figure 7 , the content (picture) of the main image 211 is indicated by a dotted line for the sake of explanation, but this information about the main image 211 may not be included in the actual sensor 3D information 221. That is, the sensor 3D information 221 only needs to include estimated 3D information of each position indicated by the polygon.

[0091] The comparison unit 135 performs processing related to comparison between the estimated 3D information and the sensor 3D information. For example, the comparison unit 135 may acquire the estimated 3D information supplied from the 3D information estimation unit 132. In addition, the comparison unit 135 may acquire information supplied from the area division unit 133 and indicating the partial area (information indicating the division result of the entire area of ​​the main image). In addition, the comparison unit 135 may acquire the sensor 3D information supplied from the sensor 3D information acquisition unit 134.

[0092] The comparison unit 135 may compare the estimated 3D information with the sensor 3D information. For example, the comparison unit 135 may compare the estimated 3D information with the sensor 3D information using the partial region generated by the region division unit 133. In addition, the comparison unit 135 may supply information indicating the comparison result to the rendering processing unit 136.

[0093] The method of comparing the estimated 3D information with the sensor 3D information can be freely determined as long as the partial region is used. For example, the comparison unit 135 can compare the sensor 3D information with the estimated 3D information for each partial region. That is, the comparison unit 135 can compare the sensor 3D information with the estimated 3D information for the partial region generated by the region dividing unit 133.

[0094] For example, when targeting Figure 5 Comparison of each part area Figure 7 Sensor 3D information 221 and Figure 4 When estimating 3D information 251, it can be seen that Figure 7 The 3D information indicated by the grayscale in the image is different from each other. That is, it can be seen that the 3D information of the partial area 262 is different.

[0095] In the case of detecting 3D information by using a sensor, it is difficult to always obtain completely correct 3D information. Similarly, in the case of estimating 3D information from an image, it is also difficult to always obtain completely correct 3D information. That is, even if a main image is generated by a legitimate method, both the estimated 3D information and the sensor 3D information are likely to include errors and are unlikely to be completely consistent. Therefore, it is realistic to determine the authenticity of an image while allowing a certain degree of difference between the estimated 3D information and the sensor 3D information.

[0096] In the conventional method described in Patent Document 2, the image is compared with the distance measurement data (the distance estimated from the image) for the entire image, thereby determining the authenticity of the image while allowing differences in the entire image to some extent. Figure 7 As in the example of , even in the case where there is a significant difference in 3D information, if the area is small, the difference is within the allowable range as the entire image, and there is a possibility that it is difficult to detect a false image.

[0097] On the other hand, the comparison unit 135 compares the estimated 3D information with the sensor 3D information using the partial area as described above. Therefore, the comparison unit 135 can more accurately detect partial inconsistencies between the image (estimated 3D information) and the sensor 3D information, such as Figure 7 Therefore, the authenticity determination apparatus 100 can determine the authenticity of the image more accurately.

[0098] Furthermore, authenticity determination of an image (detection of a false image) may not always be correctly performed. For example, suppose that in a captured image of a person as a subject captured with a flat panel as a background, inconsistency between estimated 3D information and sensor 3D information is detected in the flat panel. However, it is difficult for the comparison unit 135 to determine whether a flat panel is installed to generate a false image (for trick shooting) or whether a signboard installed in a town appears. Therefore, a legitimately generated image may be detected as a false image.

[0099] In the conventional method as described in Patent Document 2, since the distance measurement data is compared with the image (the distance estimated from the image) for the entire image, it is not possible to specify where and how the inconsistency occurs. Therefore, in the case of determining a false image as described above, it is difficult to determine whether the determination is wrong.

[0100] On the other hand, the comparison unit 135 compares the estimated 3D information with the sensor 3D information using the partial area as described above. Therefore, the comparison unit 135 can more easily specify the partial area where the inconsistency between the image (estimated 3D information) and the sensor 3D information occurs, such as in Figure 7 As in the example of . Therefore, for example, the user can determine whether the determination is wrong based on the image of the partial area. Therefore, the authenticity determination device 100 can determine the authenticity of the image more accurately.

[0101] As described above, in the case of comparing the sensor 3D information of a partial area with the estimated 3D information, the method of comparison can be freely determined.

[0102] For example, the comparison unit 135 may derive a difference between the sensor 3D information and the estimated 3D information for each position included in the partial region as a processing target, and may use the difference as a comparison result. In addition, the comparison unit 135 may derive a statistical value (e.g., a sum, a maximum value, a minimum value, a representative value, an average value, a median value, an error rate, etc.) about the difference for the partial region as a processing target, and use the statistical value as a comparison result.

[0103] In addition, the comparison unit 135 can export statistical values ​​(e.g., sum, maximum value, minimum value, representative value, average value, median value, error rate, etc.) of each of the sensor 3D information and the estimated 3D information for each position included in the partial area as the processing target, export the difference between the statistical values, and use the difference as the comparison result.

[0104] In addition, the comparison unit 135 can determine the authenticity of the partial area as the processing target based on the statistical value about such a difference or the difference between the statistical values, and can use its determination result as the comparison result. For example, the comparison unit 135 can determine whether the partial area as the processing target is a legitimate image as the determination of authenticity. In addition, the comparison unit 135 can derive the probability (or confidence) that the partial area as the processing target is a legitimate image as the determination of authenticity. In other words, the comparison unit 135 can derive the probability (or distrust) that the partial area as the processing target is an illegitimate image as the determination of authenticity.

[0105] Note that, in the case of comparing the sensor 3D information and the estimated 3D information of the partial area in this manner, the comparison unit 135 may compare the sensor 3D information and the estimated 3D information for all the partial areas generated by the area division unit 133. Furthermore, the comparison may be performed only for some partial areas. In this case, the comparison unit 135 may perform the comparison by using the partial information of the main image. For example, the comparison unit 135 may compare the sensor 3D information and the estimated 3D information for an object area that is a partial area including a main object as a subject in the main image. Furthermore, for example, the comparison unit 135 may compare the sensor 3D information and the estimated 3D information for a non-object area (excluding the main object) as a partial area other than the object area.

[0106] In addition, for example, the comparison unit 135 may derive a first relative positional relationship between the partial regions by comparing the sensor 3D information between the partial regions. In addition, the comparison unit 135 may derive a second relative positional relationship between the partial regions by comparing the estimated 3D information between the partial regions. In addition, the comparison unit 135 may compare the derived first relative positional relationship with the derived second relative positional relationship. That is, the comparison unit 135 may compare the relative positional relationship between the partial regions in the depth direction between the sensor 3D information and the estimated 3D information.

[0107] The method of deriving the relative positional relationship between the partial regions can be freely determined. For example, the comparison unit 135 can derive a statistical value (e.g., a sum, a maximum value, a minimum value, a representative value, an average value, a median value, an error rate, etc.) of the 3D information included in each partial region, and derive the relative positional relationship by using the statistical value for each partial region.

[0108] As described above, also in the case of detecting the inconsistency of the relative positional relationship between the partial regions, the occurrence of the inconsistency of the 3D information can be detected in units of the partial regions. That is, the occurrence of the local inconsistency can be detected more accurately. Therefore, the authenticity determination device 100 can determine the authenticity of the image more accurately. In addition, it is easier to specify the partial region where the inconsistency between the image (estimated 3D information) and the sensor 3D information occurs. Therefore, for example, the user can determine whether the determination is wrong based on the image of the partial region. Therefore, the authenticity determination device 100 can determine the authenticity of the image more accurately.

[0109] Note that, in this case, the comparison unit 135 can easily perform comparison with a smaller amount of processing than in the case where 3D information is compared for each of all positions in the main image. Figure 7 In the example of , due to the influence of reflectivity, etc., the sensor 3D information of partial area 262 is farther than that of partial area 263. By comparing the relative positional relationship between such partial areas, the occurrence of inconsistency in 3D information can be easily detected without comparing a large amount of 3D information for each position.

[0110] In addition, the comparison unit 135 can determine the authenticity of the main image based on the comparison result of such relative positional relationship, and can use its determination result as the comparison result. For example, the comparison unit 135 can determine whether the main image is a legitimate image as the determination of authenticity. In addition, the comparison unit 135 can derive the probability (or reliability) that the main image is a legitimate image as the determination of authenticity. In other words, the comparison unit 135 can derive the probability (or distrust) that the main image is an illegitimate image as the determination of authenticity.

[0111] In addition, for example, the comparison unit 135 can compare the difference between the 3D information of the partial area and the 3D information of the area around the partial area between the estimated 3D information and the sensor 3D information. In this way, by comparing the difference between the 3D information between the partial area and its surroundings between the estimated 3D information and the sensor 3D information, it can be determined whether the relative position of the partial area with respect to the surroundings is consistent between the estimated 3D information and the sensor 3D information. That is, the occurrence of local inconsistency can be detected more accurately. Therefore, the authenticity determination device 100 can determine the authenticity of the image more accurately. In addition, it is easier to specify the partial area where the inconsistency between the image (estimated 3D information) and the sensor 3D information occurs. Therefore, for example, the user can determine whether the determination is wrong based on the image of the partial area. Therefore, the authenticity determination device 100 can determine the authenticity of the image more accurately.

[0112] Note that, in this case, the comparison unit 135 can easily perform the comparison with less processing amount compared to the case where the 3D information is compared for each of all positions in the main image. In addition, the comparison unit 135 can perform the comparison by using partial information of the main image. For example, the comparison unit 135 can compare the difference between the 3D information of the object area in the main image and the 3D information of the area around the object area. In addition, the comparison unit 135 can compare the difference between the 3D information of the non-object area and the 3D information of the area around the non-object area. By doing so, the authenticity of the main image can be determined with less processing amount.

[0113] In addition, the comparison unit 135 may compare the estimated 3D information with the sensor 3D information by a plurality of techniques. For example, the comparison unit 135 may perform the comparison by using two or more techniques among the method of comparing the sensor 3D information of the partial region with the estimated 3D information, the method of comparing the relative position relationship between the partial region between the sensor 3D information and the estimated 3D information, and the method of comparing the 3D information difference between the partial region and its surroundings between the estimated 3D information and the sensor 3D information. In addition, the comparison unit 135 may perform the comparison by combining any one or more of the above techniques with other techniques.

[0114] In this case, the comparison unit 135 may select any one of the obtained multiple comparison results. In addition, the comparison unit 135 may combine the obtained multiple comparison results.

[0115] As described above, by applying various techniques, the authenticity determination apparatus 100 can more accurately determine the authenticity of an image.

[0116] The presentation processing unit 136 performs processing related to presentation of the authenticity determination result. For example, the presentation processing unit 136 may acquire information indicating the comparison result between the estimated 3D information obtained using the partial area and the sensor 3D information supplied from the comparison unit 135. Furthermore, based on the information, the presentation processing unit 136 may generate presentation information indicating the authenticity determination result of the partial area of ​​the main image or the authenticity determination result of the entire main image. Furthermore, the presentation processing unit 136 may present the generated presentation information.

[0117] Note that, in this specification, "presentation" indicates providing presentation information to a user, another processing unit, an application, etc. Note that this "presentation" may be performed in the authenticity determination device 100, or may be performed in another device. In other words, "presentation" may include not only the authenticity determination device 100 providing presentation information, but also the authenticity determination device 100 causing another device to provide presentation information. That is, "presentation" may include supplying presentation information to another device so that the other device provides presentation information. The presentation information may be supplied via a communication medium or a recording medium.

[0118] For example, the presentation processing unit 136 may supply the presentation information to the presentation unit 152 of the input / output interface unit 112. In this case, the presentation unit 152 provides the presentation information to a user, another processing unit, an application, etc. In addition, the presentation processing unit 136 may supply the presentation information to the communication unit 154 of the input / output interface unit 112. In this case, the communication unit 154 communicates with another device and supplies the presentation information to the other device through the communication. In addition, the presentation processing unit 136 may supply the presentation information to the drive 155 of the input / output interface unit 112. In this case, the drive 155 writes the presentation information to a removable recording medium (not shown) installed in itself. When the removable recording medium is installed in the drive of another device and the presentation information is read, the presentation information is supplied to the other device.

[0119] Note that the presentation information may be presented in any form. For example, the presentation information may be presented to a user, etc., as visual information such as an image or character information. Furthermore, the presentation information may be presented to a user, etc., as auditory information such as a sound. Furthermore, the presentation information may be presented to a user, etc., as visual information and auditory information. Furthermore, the presentation information may be presented to another processing unit, an application, etc., as data.

[0120] In other words, the presentation information may include any type of information. For example, the presentation information may include at least one of visual information or auditory information. In addition, the presentation information may include other information.

[0121] In addition, the content of the presentation information (the content to be presented) can be of any type as long as the content indicates the authenticity determination result of a partial area of ​​the main image or the authenticity determination result of the entire main image. For example, the presentation information may include a warning indicating that the authenticity is suspicious. In other words, in the case where the authenticity of the main image is suspicious based on the comparison result, the presentation processing unit 136 may generate presentation information including a warning indicating that the authenticity is suspicious. The warning may be based on the authenticity determination result for a partial area of ​​the main image, that is, the authenticity determination result for a partial area of ​​the main image may be indicated. In addition, the warning may be based on the authenticity determination result for the entire main image, that is, the authenticity determination result for the entire main image may be indicated. In addition, in the presentation information, the partial area of ​​suspicious authenticity may be clearly indicated. In other words, in the case where the authenticity of the main image is suspicious, the presentation processing unit 136 may generate presentation information that clearly indicates the area of ​​suspicious authenticity based on the comparison result. In addition, in the presentation information, the object area may be clearly indicated. In other words, the presentation processing unit 136 may generate presentation information that clearly indicates the object area.

[0122] Figure 8 The presentation screen 290 illustrated in illustrates an example of presentation information as visual information (image). On the presentation screen 290, the entire main image 211 is displayed on the main image display portion 291. In addition, a warning mark and a warning message "It is possible that an image is captured in front of the panel" are displayed above the main image display portion 291. The warning mark and the warning message are displayed as a warning indicating that the authenticity is questionable. That is, it can be said that the authenticity determination result for the entire main image 211 is indicated. That is, it can be said that the presentation screen 310 indicates the authenticity determination result for the entire image. In addition, in the main image display portion 291, the partial area 262 is clearly indicated as an "area of ​​questionable authenticity" by "shading". Such presentation is made possible by the comparison unit 135 using the partial area to compare the estimated 3D information with the sensor 3D information.

[0123] With these displays, the user who views the presentation screen 290 can easily grasp that, for example, the partial area 262 is a flat plate (that is, the image is an image showing (printing) a person 242 on a flat plate). Therefore, the user can check whether the determination result is an erroneous determination by checking whether the presence of the flat plate in the partial area 262 is legitimate. That is, the user can easily check whether the determination result is an erroneous determination, compared with the case where only the determination result of the entire main image is indicated. Therefore, the user can more accurately determine the authenticity of the main image 211 (whether the main image 211 is a fake image).

[0124] Note that in Figure 8In the case of the example of , in the main image display part 291, the thick line frame 292 clearly indicates that the partial area 261 is an object area. With such a display, the user can more easily grasp the object area. In other words, the user can easily grasp whether the "region of doubtful authenticity" is an object area or a non-object area. As a result, for example, the user can easily implement control that changes the corresponding processing according to whether the "region of doubtful authenticity" is an object area or a non-object area.

[0125] Note that in Figure 8 In the case of the example of , the reason for doubtful authenticity is presented as a warning message. In this way, for example, the presentation processing unit 136 can analyze the reason for doubtful authenticity based on the comparison result and clearly indicate such as Figure 8 In addition, facts of questionable authenticity can be simply presented as warning messages.

[0126] Fig. 9 Another example of a presentation screen 290 is shown. Fig. 9 In the case of the example of , the entire area of ​​the main image 211 is divided into an object area 293 including a person 241 and a non-object area 294 excluding the object area 293. In addition, the object area 293 is formed into a shape along the outer shape of the person 241. Then, in the main image display portion 291, it is clearly indicated by "shading" that the non-object area 294 is an "area of ​​doubtful authenticity". As the number of partial areas increases, the load of the processing related to the comparison may increase. In a case where whether the "area of ​​doubtful authenticity" is an object area is the most important, the increase in the processing load can be suppressed by such area division. In addition, with such a display, the user can easily grasp that there is a high possibility that the non-object area 294 (excluding the object area 293) is a flat plate.

[0127] Note that in Fig. 9 In the case of the example, a warning mark is further displayed in the non-object region 294, and it is more clearly indicated that the non-object region 294 is a "region of questionable authenticity".

[0128] Note that, in the case where the comparison unit 135 compares the relative positional relationship in the depth direction between the partial areas between the sensor 3D information and the estimated 3D information, the presentation processing unit 136 can generate presentation information that clearly indicates information about the relative positional relationship. In addition, in the case where the comparison unit 135 compares the difference between the 3D information of the partial area and the 3D information of the area around the partial area between the estimated 3D information and the sensor 3D information, the presentation processing unit 136 can generate presentation information that clearly indicates information about the difference. In addition, the presentation processing unit 136 can confirm the validity of the electronic signature corresponding to the image and indicate the confirmation result in the presentation information. For example, the presentation processing unit 136 acquires the signature 224 from the image file 200 through the image file acquisition unit 131. The signature 224 corresponds to the main image 211 and the sensor 3D information 221. In addition, the presentation processing unit 136 confirms the validity of the signature 224 by using key information (device public key or public key) corresponding to another device (e.g., imaging device) that is a supply source of the image file 200. Then, in the case where the validity of the signature 224 cannot be confirmed, the presentation processing unit 136 determines that the authenticity of the main image 211 does not exist. Then, the presentation processing unit 136 indicates the confirmation result of such an electronic signature in the presentation information. For example, in the case where the validity of the electronic signature cannot be confirmed, the presentation processing unit 136 can cause the presentation screen to display a message indicating this fact, etc. In addition, the presentation processing unit 136 can cause the presentation screen to display a message indicating the presence or absence of the validity of the electronic signature, etc. By doing so, the authenticity determination device 100 can detect the falsification of the image or 3D information by using the signature. Therefore, the authenticity determination device 100 can determine the authenticity of the image more accurately.

[0129] Return to Figure 1 , the input unit 151 of the input / output interface unit 112 includes an input device and performs processing related to information input. The input device may be of any type. For example, the input unit 151 may include a keyboard, a mouse, a microphone, a physical switch, a physical button, a touch panel, an input terminal, etc. For example, the input unit 151 may receive information input from the outside such as a user or another device via the input device, and supply the information to the control unit 101 or the image analysis engine 111.

[0130] The presentation unit 152 includes an output device and performs processing related to the presentation of the presentation information. The output device can be of any type. For example, the presentation unit 152 can include a display, a speaker, etc. The display presents the presentation information as visual information. In addition, the speaker presents the presentation information as auditory information. For example, the presentation unit 152 can obtain the presentation information from the presentation processing unit 136 and present the presentation information to a user, etc. via the output device.

[0131] The storage unit 153 includes a storage medium and performs processing related to the storage of information. The storage medium may be of any type. For example, the storage unit 153 may include a hard disk, a RAM disk, a non-volatile memory, etc. For example, the storage unit 153 may store data, programs, etc. In addition, the storage unit 153 may supply them to the control unit 101 and the image analysis engine 111 when necessary. For example, the storage unit 153 may store (keep) an image file or presentation information whose authenticity of a main image is determined.

[0132] The communication unit 154 includes a communication interface and performs processing related to communication with another device. For example, the communication unit 154 can communicate with another device via the communication interface in a manner that complies with any communication standard. The communication can be wired communication, wireless communication, or both.

[0133] For example, the communication unit 154 may acquire a program or data supplied from another device through communication. In addition, the communication unit 154 may supply the program or data to the storage unit 153 for storage, or may supply the program or data to the drive 155 to be recorded in a removable recording medium (not shown). In addition, the communication unit 154 may supply the program or data to the control unit 101 or the image analysis engine 111. For example, the communication unit 154 may acquire an image file supplied from another device, and supply the image file to the image file acquisition unit 131.

[0134] In addition, the communication unit 154 can supply a program or data to another device through communication. For example, the communication unit 154 can supply a program or data read from the storage unit 153 to another device. In addition, the communication unit 154 can supply a program or data read from a removable recording medium (not shown) via the drive 155 to another device. In addition, the communication unit 154 can supply presentation information supplied from the presentation processing unit 136 to another device.

[0135] The drive 155 drives a removable recording medium (not shown) attached to itself, and performs processing related to reading information from and writing information to the removable recording medium. The removable recording medium may be any standard recording medium. For example, the removable recording medium may be a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory. For example, the drive 155 may read data, programs, etc. recorded in the removable recording medium, and supply the data, programs, etc. to the control unit 101 and the image analysis engine 111. In addition, the drive 155 may record data, programs, etc. on the removable recording medium. For example, the drive 155 may record an image file whose authenticity of the main image is determined and presentation information provided from the presentation processing unit 136 on the removable recording medium.

[0136] By having the above configuration, the authenticity determination apparatus 100 can more accurately detect partial inconsistency between an image (estimated 3D information) and sensor 3D information, and can more accurately determine the authenticity of the image.

[0137] <Example 1 of Flow of Authenticity Determination Processing>

[0138] The authenticity determination apparatus 100 performs an authenticity determination process in order to determine the authenticity of an image. Fig.10 An example of the flow of authenticity determination processing is described in the flowchart of FIG. Fig.10 In FIG. 1 , a description is given of a flow of a case where the region division unit 133 performs region division based on a main image and the comparison unit 135 compares the sensor 3D information with the estimated 3D information for a partial region.

[0139] In this case, when the authenticity determination process starts, in step S101 , the image file acquisition unit 131 of the authenticity determination apparatus 100 acquires an image file for determining the authenticity of a master image.

[0140] In step S102 , the 3D information estimating unit 132 acquires a main image from an image file, and generates estimated 3D information corresponding to the main image based on the main image.

[0141] In step S103, the region division unit 133 acquires a main image from an image file, detects a main object as a subject in the main image, and divides regions based on the detection result of the object. For example, the region division unit 133 generates an object region and a non-object region.

[0142] In step S104 , the sensor 3D information acquisition unit 134 acquires sensor 3D information from the image file.

[0143] In step S105 , the comparison unit 135 compares the sensor 3D information and the estimated 3D information for each partial region generated in step S103 .

[0144] In step S106 , the presentation processing unit 136 generates and presents presentation information indicating the result of the comparison performed in step S105 .

[0145] When the process of step S106 ends, the authenticity determination process ends.

[0146] By performing the authenticity determination process in this manner, the authenticity determination apparatus 100 can more accurately detect partial inconsistency between the image (estimated 3D information) and the sensor 3D information, and can more accurately determine the authenticity of the image.

[0147] <Example 2 of the flow of authenticity determination processing>

[0148] Next, we will refer to Fig.11 Another example of the flow of authenticity determination processing is described in the flowchart of FIG. Fig.11 In FIG. 1 , a description is given of a flow of a case where the region division unit 133 performs region division based on the estimated 3D information and the comparison unit 135 compares the sensor 3D information with the estimated 3D information for a partial region.

[0149] In this case, when the authenticity determination process starts, in step S121 , the image file acquisition unit 131 of the authenticity determination apparatus 100 acquires an image file for determining the authenticity of a master image.

[0150] In step S122 , the 3D information estimating unit 132 acquires the main image from the image file, and generates estimated 3D information corresponding to the main image based on the main image.

[0151] In step S123 , the region division unit 133 clusters the estimated 3D information generated in step S122 , and divides regions based on the clustering result thereof.

[0152] In step S124 , the sensor 3D information acquisition unit 134 acquires sensor 3D information from the image file.

[0153] In step S125 , the comparison unit 135 compares the sensor 3D information and the estimated 3D information for each partial region generated in step S123 .

[0154] In step S126 , the presentation processing unit 136 generates and presents presentation information indicating the result of the comparison performed in step S125 .

[0155] When the processing of step S126 ends, the authenticity determination processing ends.

[0156] By performing the authenticity determination process in this manner, the authenticity determination apparatus 100 can more accurately detect partial inconsistency between the image (estimated 3D information) and the sensor 3D information, and can more accurately determine the authenticity of the image.

[0157] <Example 3 of the flow of authenticity determination processing>

[0158] Next, we will refer to Fig.12 Another example of the flow of authenticity determination processing is described in the flowchart of FIG. Fig.12 In the following, a description is given of a flow in a case where the region division unit 133 performs region division based on a main image and the comparison unit 135 compares the relative positional relationship between partial regions in the depth direction between the sensor 3D information and the estimated 3D information.

[0159] In this case, when the authenticity determination process starts, in step S141 , the image file acquisition unit 131 of the authenticity determination apparatus 100 acquires an image file for determining the authenticity of a master image.

[0160] In step S142 , the 3D information estimating unit 132 acquires a main image from the image file, and generates estimated 3D information corresponding to the main image based on the main image.

[0161] In step S143 , the region division unit 133 acquires a main image from the image file, detects a main object as a subject in the main image, and divides the region based on the detection result of the object.

[0162] In step S144 , the sensor 3D information acquisition unit 134 acquires sensor 3D information from the image file.

[0163] In step S145 , the comparison unit 135 obtains the relative positional relationship in the depth direction between the partial areas generated in step S143 in each of the sensor 3D information and the estimated 3D information, and compares the relative positional relationship in the sensor 3D information with the relative positional relationship in the estimated 3D information.

[0164] In step S146 , the presentation processing unit 136 generates and presents presentation information indicating the result of the comparison performed in step S145 .

[0165] When the processing of step S146 ends, the authenticity determination processing ends.

[0166] By performing the authenticity determination process in this manner, the authenticity determination apparatus 100 can more accurately detect partial inconsistency between the image (estimated 3D information) and the sensor 3D information, and can more accurately determine the authenticity of the image.

[0167] <Example 4 of the flow of authenticity determination processing>

[0168] Next, we will refer to Fig.13 Another example of the flow of authenticity determination processing is described in the flowchart of FIG. Fig.13 In the following, a description is given of a flow in a case where the region division unit 133 performs region division based on estimated 3D information and the comparison unit 135 compares the relative positional relationship between partial regions in the depth direction between the sensor 3D information and the estimated 3D information.

[0169] In this case, when the authenticity determination process starts, in step S161 , the image file acquisition unit 131 of the authenticity determination apparatus 100 acquires an image file for determining the authenticity of a master image.

[0170] In step S162 , the 3D information estimating unit 132 acquires the main image from the image file, and generates estimated 3D information corresponding to the main image based on the main image.

[0171] In step S163 , the region dividing unit 133 clusters the estimated 3D information generated in step S162 , and divides regions based on the clustering result thereof.

[0172] In step S164 , the sensor 3D information acquisition unit 134 acquires sensor 3D information from the image file.

[0173] In step S165 , the comparison unit 135 obtains the relative positional relationship in the depth direction between the partial areas generated in step S163 in each of the sensor 3D information and the estimated 3D information, and compares the relative positional relationship in the sensor 3D information with the relative positional relationship in the estimated 3D information.

[0174] In step S166 , the presentation processing unit 136 generates and presents presentation information indicating the result of the comparison performed in step S165 .

[0175] When the processing of step S166 ends, the authenticity determination processing ends.

[0176] By performing the authenticity determination process in this manner, the authenticity determination apparatus 100 can more accurately detect partial inconsistency between the image (estimated 3D information) and the sensor 3D information, and can more accurately determine the authenticity of the image.

[0177] <Example 5 of the flow of authenticity determination processing>

[0178] Next, we will refer to Fig.14 Another example of the flow of authenticity determination processing is described in the flowchart of FIG. Fig.14 In the following, a description is given of a flow in a case where the area division unit 133 performs area division based on a main image and the comparison unit 135 compares a 3D information difference between a partial area and its surrounding area between estimated 3D information and sensor 3D information.

[0179] In this case, when the authenticity determination process starts, in step S181 , the image file acquisition unit 131 of the authenticity determination apparatus 100 acquires an image file for determining the authenticity of a master image.

[0180] In step S182 , the 3D information estimating unit 132 acquires the main image from the image file, and generates estimated 3D information corresponding to the main image based on the main image.

[0181] In step S183 , the region division unit 133 acquires a main image from the image file, detects a main object as a subject in the main image, and divides the region based on the detection result of the object.

[0182] In step S184 , the sensor 3D information acquisition unit 134 acquires sensor 3D information from the image file.

[0183] In step S185 , the comparison unit 135 derives a difference in 3D information between the partial region generated in step S183 and its surrounding region in each of the sensor 3D information and the estimated 3D information, and compares the difference in the sensor 3D information with the difference in the estimated 3D information.

[0184] In step S186 , the presentation processing unit 136 generates and presents presentation information indicating the result of the comparison performed in step S185 .

[0185] When the processing of step S186 ends, the authenticity determination processing ends.

[0186] By performing the authenticity determination process in this manner, the authenticity determination apparatus 100 can more accurately detect partial inconsistency between the image (estimated 3D information) and the sensor 3D information, and can more accurately determine the authenticity of the image.

[0187] <Example 6 of the flow of authenticity determination processing>

[0188] Next, we will refer to Fig.15 Another example of the flow of authenticity determination processing is described in the flowchart of FIG. Fig.15In the following, a description is given of a flow in a case where the region division unit 133 performs region division based on estimated 3D information and the comparison unit 135 compares a 3D information difference between a partial region and its surrounding region between the estimated 3D information and the sensor 3D information.

[0189] In this case, when the authenticity determination process starts, in step S201 , the image file acquisition unit 131 of the authenticity determination apparatus 100 acquires an image file for determining the authenticity of a master image.

[0190] In step S202 , the 3D information estimating unit 132 acquires a main image from the image file, and generates estimated 3D information corresponding to the main image based on the main image.

[0191] In step S203 , the region division unit 133 clusters the estimated 3D information generated in step S202 , and divides regions based on the clustering result thereof.

[0192] In step S204 , the sensor 3D information acquisition unit 134 acquires sensor 3D information from the image file.

[0193] In step S205 , the comparison unit 135 derives a 3D information difference between the partial region generated in step S183 and its surrounding region in each of the sensor 3D information and the estimated 3D information, and compares the difference in the sensor 3D information with the difference in the estimated 3D information.

[0194] In step S206 , the presentation processing unit 136 generates and presents presentation information indicating the result of the comparison performed in step S205 .

[0195] When the processing of step S206 ends, the authenticity determination processing ends.

[0196] By performing the authenticity determination process in this manner, the authenticity determination apparatus 100 can more accurately detect partial inconsistency between the image (estimated 3D information) and the sensor 3D information, and can more accurately determine the authenticity of the image.

[0197] <Example 7 of Flow of Authenticity Determination Processing>

[0198] Note that, Figures 10 to 15 The single examples in may be applied in combination. That is, the region division unit 133 may perform region division based on the main image or the estimated 3D information or based on both the main image and the estimated 3D information. In addition, the comparison unit 135 may compare the 3D information by a plurality of methods and derive a final comparison result based on a single comparison result. Fig.16 An example of the flow of the authenticity determination processing in this case will be described with reference to the flowchart of FIG.

[0199] In this case, when the authenticity determination process starts, in step S221 , the image file acquisition unit 131 of the authenticity determination apparatus 100 acquires an image file for determining the authenticity of the main image.

[0200] In step S222 , the 3D information estimating unit 132 acquires the main image from the image file, and generates estimated 3D information corresponding to the main image based on the main image.

[0201] In step S223, the region dividing unit 133 divides the region using at least any one of the main image stored in the image file or the estimated 3D information generated in step S222. That is, the region dividing unit 133 may divide the region based on the main image similarly to the case of the above-mentioned steps S103, S143, S183, or may divide the region based on the estimated 3D information similarly to the case of the above-mentioned steps S123, S163, S203. Furthermore, both of them may be combined.

[0202] In step S224 , the sensor 3D information acquisition unit 134 acquires sensor 3D information from the image file.

[0203] In step S225, the comparison unit 135 uses the partial area generated in step S183 to compare the sensor 3D information with the estimated 3D information by a variety of methods. For example, the comparison unit 135 compares the sensor 3D information with the estimated 3D information for each partial area generated in step S223 similarly to the case of the above-mentioned steps S105 and S125. Then, the comparison unit 135 obtains the relative positional relationship in the depth direction between the partial area generated in step S223 in each of the sensor 3D information and the estimated 3D information similarly to the case of the above-mentioned steps S145 and S165, and compares the relative positional relationship in the sensor 3D information with the relative positional relationship in the estimated 3D information. In addition, the comparison unit 135 derives the difference in 3D information between the partial area generated in step S223 and its surrounding area in each of the sensor 3D information and the estimated 3D information similarly to the case of the above-mentioned steps S185 and S205, and compares the difference in the sensor 3D information with the difference in the estimated 3D information.

[0204] In step S226, the comparison unit 135 derives a final comparison result based on the comparison results of the individual methods derived in step S225. For example, the comparison unit 135 may select a result with a high evaluation obtained based on a predetermined method among the comparison results of the individual methods as the final comparison result. In addition, the comparison unit 135 may combine the comparison results of the individual methods (among which a plurality of comparison results) to generate a final comparison result.

[0205] In step S227 , the presentation processing unit 136 generates and presents presentation information indicating the final comparison result derived in step S226 .

[0206] When the processing of step S227 ends, the authenticity determination processing ends.

[0207] By performing the authenticity determination process in this manner, the authenticity determination apparatus 100 can more accurately detect partial inconsistency between the image (estimated 3D information) and the sensor 3D information, and can more accurately determine the authenticity of the image.

[0208] <3. Determination of authenticity of processed images>

[0209] The authenticity determination target may be different from the captured image. For example, the authenticity determination apparatus 100 may set a processed image obtained by processing the captured image as the authenticity determination target. In this case, the authenticity determination apparatus 100 may determine the authenticity of the processed image by using an original image associated with the processed image instead of the processed image.

[0210] For example, it is assumed that the main image 211 as the original image is different from the main image 211 ( Figure 5 ) of a portion of the partial region 262 and an image 301 ( Fig.17 ), and the authenticity determination apparatus 100 is able to acquire the original image based on the image 301. When the authenticity determination apparatus 100 determines the authenticity of such an image 301 (cropped image), the authenticity determination can be performed by setting the main image 211 as the original image as a processing target and setting the image 301 as a partial area (cropped area) in the main image 211.

[0211] For example, the image file acquisition unit 131 may acquire an image file storing an original image (main image 211) of the image 301. Furthermore, the 3D information estimation unit 132 may acquire the main image 211 from the image file, estimate 3D information based on the main image 211, and generate estimated 3D information corresponding to the main image 211. By doing so, since the 3D information estimation unit 132 can estimate 3D information by also using a portion other than the partial area 262, 3D information can be estimated more accurately than in the case of estimating 3D information only from the image 301.

[0212] Furthermore, the region dividing unit 133 may acquire the main image 211 from the image file, divide the entire region of the main image 211 (original image), and generate the partial region 262. That is, the region dividing unit 133 may set the region corresponding to the image 301 (processed image) in the main image 211 as the partial region.

[0213] The comparison unit 135 can compare the estimated 3D information with the sensor 3D information using the partial area 262 (the partial area corresponding to the image 301) set in this way. As a result, the comparison unit 135 can process the image 301 as a partial area. Therefore, for example, similar to the reference Fig.10 and Fig.11 In the case described in the flowchart of FIG. 1 , the comparison unit 135 can compare the estimated 3D information of the partial area 262 with the sensor 3D information. Fig.12 and Fig.13 In the case described in the flowchart of FIG. 1 , the comparison unit 135 may also compare the relative positional relationship between the partial regions in the depth direction between the sensor 3D information and the estimated 3D information. Fig.14 and Fig.15 In the case described in the flowchart of FIG. 1 , the comparison unit 135 can also compare the 3D information difference between the partial area and its surrounding area between the estimated 3D information and the sensor 3D information. In addition, the comparison unit 135 can compare the reference 3D information with the reference 3D information. Fig.16 The case described by the flowchart of is similarly compared by a variety of methods.

[0214] Therefore, the authenticity determination apparatus 100 can more accurately detect partial inconsistency between the image (estimated 3D information) and the sensor 3D information, and can more accurately determine the authenticity of the image.

[0215] Note that the presentation processing unit 136 may generate presentation information indicating a determination result of the authenticity of the image 301 based on such a comparison result. Fig.17 The presentation screen 310 illustrated in illustrates an example of presentation information as visual information (image) in this case. In the presentation screen 310, the image 301 is displayed on the image display portion 311. In addition, a warning mark and a warning message "It is possible that the image is captured in front of the panel" are displayed above the image display portion 311. These are displayed as warnings indicating that the authenticity is questionable. As described above, the comparison unit 135 processes the image 301 as a cropped image as a partial area of ​​the main image 211. That is, it can be said that a partial area of ​​the main image 211 is displayed on the image display portion 311 of the presentation screen 310. In addition, it can be said that the warning message on the presentation screen 310 indicates the result of the determination of the authenticity of the partial area. That is, it can be said that the presentation screen 310 indicates the result of the authenticity determination for the partial area of ​​the image.

[0216] <Example 8 of Flow of Authenticity Determination Processing>

[0217] Reference Fig.18 An example of the flow of authenticity determination processing in this case is described in the flowchart of FIG. Fig.18 In this paper, the above method is applied to Fig.10 Description of the processing flow of an example situation.

[0218] In this case, when the authenticity determination process starts, in step S301 , the image file acquisition unit 131 of the authenticity determination apparatus 100 acquires an image file storing a processed image used for authenticity determination.

[0219] In step S302 , the image file acquisition unit 131 acquires an original image corresponding to the main image (processed image) stored in the image file.

[0220] In step S303 , the 3D information estimating unit 132 estimates 3D information of the original image based on the original image, and generates estimated 3D information corresponding to the original image.

[0221] In step S304 , the region dividing unit 133 divides the entire region of the original image, and generates a partial region corresponding to the processed image.

[0222] In step S305 , the sensor 3D information acquisition unit 134 acquires sensor 3D information corresponding to the original image from the image file of the original image.

[0223] In step S306 , the comparison unit 135 compares the sensor 3D information with the estimated 3D information using the partial region corresponding to the processed image generated in step S304 .

[0224] In step S307 , the presentation processing unit 136 generates and presents presentation information indicating the result of the comparison performed in step S306 .

[0225] When the processing of step S307 ends, the authenticity determination processing ends.

[0226] By performing the authenticity determination process in this manner, the authenticity determination apparatus 100 can more accurately detect partial inconsistency between a processed image (which estimates 3D information) and sensor 3D information, and can more accurately determine the authenticity of the processed image.

[0227] Note that the method of determining the authenticity of a processed image by using the original image of the processed image described above can be similarly applied to Figures 11 to 16 A single example of .

[0228] <4. Determination of the authenticity of moving images>

[0229] The authenticity determination target of the authenticity determination apparatus 100 may be a still image or a moving image. In the case of determining the authenticity of a moving image, the authenticity determination apparatus 100 may determine the authenticity for each frame of the moving image similarly to the case of a still image.

[0230] For example, the 3D information estimation unit 132 may estimate the 3D information of each frame and generate estimated 3D information corresponding to each frame. In addition, the region division unit 133 may divide the entire region for each frame to generate a partial region. In addition, the comparison unit 135 may compare the estimated 3D information with the sensor 3D information using the partial region for each frame. In addition, the rendering processing unit 136 may generate rendering information based on the comparison result for each frame. By doing so, the authenticity of the moving image may be determined for each frame.

[0231] For example, the presentation processing unit 136 may set the presentation information as visual information and display the authenticity determination result as a moving image. Fig.19 The presentation screen 320 illustrated in illustrates an example of presentation information as visual information (image). On the presentation screen 320, a moving image as a main image is displayed on the main image display portion 321. In addition, in the main image display portion 321, it is clearly indicated by "shadow" that the partial area 262 is an "area of ​​questionable authenticity". This display is displayed for each frame of the moving image. That is, in the case where the position and size of the partial area 262 in the moving image change in the time direction, the range of the "shadow" changes similarly. The comparison unit 135 can perform such presentation by comparing the estimated 3D information with the sensor 3D information using the partial area for each frame.

[0232] Note that in Fig.19 In the case of the example of , in the main image display portion 321, the thick-line frame 292 clearly indicates that the partial area 261 is the object area. With such a display, the user can more easily grasp the object area. Note that this display is displayed for each frame of the moving image. That is, in the case where the position and size of the partial area 261 in the moving image change in the time direction, the thick-line frame 292 changes similarly.

[0233] In addition, a reproduction control button 322 is provided as a graphical user interface (GUI) below the main image display portion 321. For example, when the user operates the reproduction control button 322, the reproduction, stop, etc. of the moving image and the comparison result displayed on the main image display portion 321 are controlled. That is, for example, when the user operates the reproduction control button 322, the image of a desired frame and the comparison result can be displayed on the main image display portion 321.

[0234] Note that the presentation processing unit 136 may generate presentation information indicating the range in the time direction of the area where the authenticity is questionable in the moving image. For example, in the presentation information, the range of the frame in which the area where the authenticity is questionable exists may be clearly indicated. In addition, in the presentation information, the time during which the area where the authenticity is questionable exists may be clearly indicated.

[0235] For example, Fig.19 As illustrated in FIG. 3 , a time bar 323 may be provided below the main image display portion 321 as a GUI, and the range of the frame in which the area of ​​doubtful authenticity exists may be indicated in the time bar 323. Fig.19 In the case of the example, the range 324 indicated by the diagonal pattern indicates the range of the frame in which the "region of doubtful authenticity" exists. By clearly indicating such information in the presentation information, the user can more easily grasp the frame of the moving image in which the "region of doubtful authenticity" exists.

[0236] In addition, the rendering processing unit 136 may generate a reduced image of the frame in which there are areas of questionable authenticity. Fig. 20 As illustrated in FIG. 1 , the presentation processing unit 136 can generate a reduced image 340 of a frame in which an area of ​​questionable authenticity exists, and associate the reduced image with the main image. In the reduced image 340, an object area 341 and a non-object area 342 are formed as partial areas, and the non-object area 342 is clearly indicated as a "region of questionable authenticity" by "shading". By referring to such a reduced image 340, the user can easily grasp that there is a high possibility that the non-object area 342 (except the object area 341) is a flat plate. Note that in Fig. 20 In the case of the example, a warning mark is further displayed in the non-object region 342, and it is more clearly indicated that the non-object region 342 is a “region of doubtful authenticity”.

[0237] <Example 9 of Flow of Authenticity Determination Processing>

[0238] Reference Fig.21 An example of the flow of authenticity determination processing in this case is described in the flowchart of FIG. Fig.21 In this paper, the above method is applied to Fig.10 Description of the processing flow of the example case.

[0239] In this case, when the authenticity determination process starts, in step S331 , the image file acquisition unit 131 of the authenticity determination apparatus 100 acquires an image file for determining the authenticity of a main image (moving image).

[0240] In step S332, the image file acquisition unit 131 selects an unprocessed frame as a processing target frame from among a single frame of the motion image as the main image. For example, the image file acquisition unit 131 selects each frame as the processing target frame in time order. For example, in the case of the first time, the image file acquisition unit 131 selects the head frame of the motion image. In addition, in the case of the second time and subsequent times, the image file acquisition unit 131 updates the processing target frame to the next frame in time order.

[0241] In step S333 , the 3D information estimating unit 132 acquires the processing target frame of the main image (moving image) from the image file, and generates estimated 3D information corresponding to the processing target frame based on the processing target frame.

[0242] In step S334, the region division unit 133 acquires the processing target frame of the main image (moving image) from the image file, detects the main object as the subject in the processing target frame, and divides the entire region of the processing target frame based on the detection result of the object. For example, the region division unit 133 generates an object region and a non-object region in the processing target frame.

[0243] In step S335 , the sensor 3D information acquisition unit 134 acquires sensor 3D information of the processing target frame from the image file.

[0244] In step S336, for the processing target frame, the comparison unit 135 compares the sensor 3D information with the estimated 3D information for each partial region generated in step S334. By doing so, a comparison result of the processing target frame is obtained.

[0245] In step S337, the image file acquisition unit 131 determines whether all frames have been processed. In the case where it is determined that there are unprocessed frames, the process returns to step S332, and the subsequent process is performed. That is, for each frame of the main image (moving image), each process of steps S332 to S337 is performed, and a comparison result is obtained. In the case where it is determined in step S337 that all frames of the main image (moving image) have been processed, the process proceeds to step S338.

[0246] In step S338 , the presentation processing unit 136 generates and presents presentation information indicating the comparison result of each frame obtained as described above.

[0247] When the processing of step S338 ends, the authenticity determination processing ends.

[0248] By performing the authenticity determination process in this manner, the authenticity determination apparatus 100 can determine the authenticity of each frame of the moving image, and can determine the authenticity of the moving image more accurately.

[0249] <5. Embodiment (Image processing system)>

[0250] The present technology is not limited to the authenticity determination device 100 described above, and can be applied to any configuration. For example, the present technology can also be applied to an image processing system. Fig. 22 is a diagram illustrating a configuration example of an image processing system to which the present technology is applied. Fig. 22 The image processing system 400 shown in FIG. 1 is a system for managing images. Fig. 22 As illustrated in , the image processing system 400 includes an authenticity determination server 411 , a storage server 412 , an imaging device 413 , and a terminal device 414 , which are communicably connected to each other via a network 401 .

[0251] The network 401 is a communication network used as a communication medium between single devices. The network 401 may be a communication network for wired communication, a communication network for wireless communication, or a communication network for both wireless communication and wired communication. For example, the network 401 may be a wired local area network (LAN), a wireless LAN, a public telephone line network, a wide area communication network for wireless mobile bodies such as so-called 4G lines or 5G lines, the Internet, etc., or a combination thereof. In addition, the network 401 may be a single communication network or a plurality of communication networks. In addition, for example, part or all of the network 401 may be configured by a communication cable of a predetermined standard such as a universal serial bus (USB) (registered trademark) cable, a high-definition multimedia interface (HDMI) (registered trademark) cable, etc.

[0252] The imaging device 413 generates a captured image by, for example, capturing an image of a subject, and generates an image file storing the captured image. The imaging device 413 uploads the generated image file to the storage server 412, and causes the image file to be managed. The storage server 412 stores and manages the uploaded image file. The terminal device 414 downloads a desired image file from the storage server 412 and reproduces it.

[0253] The authenticity determination server 411 determines the authenticity of the image. For example, the authenticity determination server 411 acquires an image file uploaded to the storage server 412 by the imaging device 413, and determines the authenticity of the main image stored in the image file. The authenticity determination server 411 supplies presentation information indicating the result of its determination to the imaging device 413, and causes the presentation information to be presented. In addition, the authenticity determination server 411 acquires an image file requested to be downloaded to the storage server 412 by the terminal device 414, and determines the authenticity of the main image stored in the image file. The authenticity determination server 411 supplies presentation information indicating the result of its determination to the terminal device 414, and causes the presentation information to be presented.

[0254] By doing so, for example, the authenticity of the master image can be confirmed by the user of the imaging device 413 or the terminal device 414. In addition, for example, only image files whose authenticity has been confirmed can be managed by the storage server 412.

[0255] In such an image processing system 400, Figure 1 The authenticity determination apparatus 100 in the embodiment can be applied as the authenticity determination server 411. That is, the authenticity determination server 411 can have a configuration similar to that of the authenticity determination apparatus 100, and perform processing similar to that of the authenticity determination apparatus 100. By doing so, the present technology described in <2. Image authenticity determination using a partial area>, <3. Authenticity determination of a processed image>, or <4. Authenticity determination of a moving image> can be applied to the authenticity determination server 411. That is, the authenticity determination server 411 can obtain an effect similar to that of the above-described authenticity determination apparatus 100. That is, the authenticity determination server 411 (image processing system 400) can determine the authenticity of an image more accurately.

[0256] Note that the configuration of the image processing system 400 can be freely determined. For example, a plurality of authenticity determination servers 411 or a plurality of terminal devices 414 may be provided. Furthermore, devices other than the authenticity determination server 411 or the terminal device 414 may be connected to the network 401.

[0257] <6. Additional Notes>

[0258] <Computer>

[0259] The above series of processing can be performed by hardware or software. In the case where a series of processing is performed by software, the program included in the software is installed on a computer. Here, the computer includes, for example, a computer installed in special hardware, a general-purpose personal computer capable of performing various functions by installing various programs, etc.

[0260] Fig.23 : is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processes by a program.

[0261] exist Fig.23 In a computer 900 illustrated in FIG. 9 , a central processing unit (CPU) 901 , a read only memory (ROM) 902 , and a random access memory (RAM) 903 are connected to one another via a bus 904 .

[0262] Furthermore, an input / output interface 910 is also connected to the bus 904. An input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected to the input / output interface 910.

[0263] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a nonvolatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives a removable recording medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0264] In the computer configured as described above, for example, the CPU 901 loads the program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executes the program. As a result, the above-mentioned series of processing is performed. The RAM 903 can appropriately store data and the like necessary for the CPU 901 to perform various processing.

[0265] For example, the program executed by the computer may be recorded in the removable recording medium 921 as a package medium or the like and applied. In this case, the program can be read from the removable recording medium 921 attached to the drive 915 and mounted in the storage unit 913 via the input / output interface 910 .

[0266] Furthermore, the program may be provided via any wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program may be received by the communication unit 914 via the input / output interface 910 and installed in the storage unit 913 .

[0267] Furthermore, the program may be pre-installed in the ROM 902 , the storage unit 913 , or both.

[0268] <Targets of application of this technology>

[0269] The present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0270] In addition, for example, the present technology can also be implemented as a partial configuration of a device such as a processor as a system large-scale integration (LSI) (e.g., a video processor), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a collection obtained by further adding other functions to a unit (e.g., a video collection).

[0271] In addition, for example, the present technology can also be applied to a network system including a plurality of devices. For example, the present technology can be implemented as cloud computing that is collaboratively shared and processed by a plurality of devices through a network. For example, the present technology can be implemented in a cloud service that provides services related to images (moving images) to any terminal such as a computer, an audio-visual (AV) device, a portable information processing terminal, or an Internet of Things (IoT) device.

[0272] Note that in this specification, a system means a collection of multiple components (devices, modules (components), etc.), and it does not matter whether all components are in the same housing. Therefore, multiple devices stored in different housings and connected via a network and one device in which multiple modules are stored in one housing are both systems.

[0273] <Others>

[0274] Note that in this specification, the term "associated" means, for example, that other data is allowed to be used (linked) when processing one data. That is, data associated with each other can be collected as one data, or it can be made to be a single data. For example, information associated with the encoded data (image) can be transmitted on a transmission path different from the transmission path of the encoded data (image). In addition, for example, information associated with the encoded data (image) can be recorded in a recording medium different from the recording medium of the encoded data (image) (or another recording area of ​​the same recording medium). Note that this "association" may not be all data, but part of the data. For example, a moving image and information corresponding to the moving image can be associated with each other in any unit such as multiple frames, one frame, or a part within a frame.

[0275] Note that in this specification, terms such as "combine", "multiplex", "add", "merge", "include", "store", "put into", "introduce" and "insert" mean combining multiple objects into one, such as combining encoded data and metadata into one data, and mean a method of "association" mentioned above.

[0276] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the scope of the present technology.

[0277] For example, a configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configuration described above as a plurality of devices (or processing units) may also be collectively configured as one device (or processing unit). Furthermore, it goes without saying that a configuration other than the above configuration may be added to the configuration of each device (or each processing unit). Furthermore, if the configuration and operation of the entire system are substantially the same, part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or another processing unit).

[0278] In addition, for example, the above-mentioned program can be executed in any device. In this case, it is only necessary for the device to have necessary functions (functional blocks, etc.) and obtain necessary information.

[0279] In addition, for example, each step in a flowchart may be performed by one device, or may be performed by being shared by a plurality of devices. In addition, in the case where a plurality of processes are included in one step, the plurality of processes may be performed by one device, or may be shared and performed by a plurality of devices. In other words, the plurality of processes included in one step may be performed as a plurality of steps. Conversely, the processes described as a plurality of steps may also be performed together as one step.

[0280] In addition, for example, in a program executed by a computer, the processing of the steps describing the program may be performed in the order described in this specification in a time series order, or may be performed in parallel or independently at a required timing (such as when a call is made). That is, as long as there is no contradiction, the processing of each step may also be performed in an order different from the above order. In addition, the processing in the steps describing the program may be performed in parallel with the processing of another program, or may be performed in combination with the processing of another program.

[0281] In addition, for example, as long as there is no contradiction, multiple technologies related to the present technology can be independently implemented as a single entity. It goes without saying that any multiple present technologies can be implemented in combination. For example, part or all of the present technology described in any one of the embodiments can be implemented in combination with part or all of the present technology described in other embodiments. In addition, part or all of any of the above-mentioned present technologies can be implemented together with another technology not described above.

[0282] Note that the present technology may also have the following configurations.

[0283] (1) An image processing device comprising:

[0284] a region dividing unit configured to generate a partial region by dividing the entire region of the image; and

[0285] a comparing unit configured to compare the estimated 3D information with the sensor 3D information using the partial area, wherein

[0286] The estimated 3D information is 3D information estimated based on the image, and

[0287] The sensor 3D information is 3D information acquired by a sensor and associated with the image.

[0288] (2) The image processing apparatus according to (1), wherein the region division unit detects an object that is primary and is a subject in the image, and divides the entire region based on a detection result.

[0289] (3) The image processing apparatus according to (2), wherein the region division unit generates an object region which is a partial region including the detected object and a non-object region which is a partial region which does not include the object.

[0290] (4) The image processing apparatus according to (2) or (3), wherein the area division unit detects the object by performing pattern analysis in the image.

[0291] (5) The image processing apparatus according to (2) or (3), wherein the region division unit detects the object using a neural network that is input with the image and outputs a detection result of the object.

[0292] (6) The image processing apparatus according to (1), wherein the region division unit divides the entire region by clustering the estimated 3D information.

[0293] (7) The image processing apparatus according to (6), wherein the region division unit divides the entire region using a neural network that is input with the estimated 3D information and outputs a result of the clustering.

[0294] (8) The image processing device according to (7), wherein, by inputting the estimated 3D information into the neural network, the region division unit generates an object region and a non-object region, the object region being a partial region including an object that is main and a subject in the image, and the non-object region being a partial region not including the object.

[0295] (9) The image processing apparatus according to any one of (1) to (8), wherein the comparison unit compares the sensor 3D information with the estimated 3D information for each of the partial regions.

[0296] (10) The image processing apparatus according to (9), wherein the comparison unit compares the sensor 3D information and the estimated 3D information for all of the partial regions generated by the region division unit.

[0297] (11) The image processing apparatus according to (9), wherein the comparison unit compares the sensor 3D information with the estimated 3D information for an object region that is a partial region that is main and serves as a subject in the image.

[0298] (12) The image processing apparatus according to (9), wherein the comparison unit compares the sensor 3D information with the estimated 3D information for a non-object region that is a partial region not including a main object in the image.

[0299] (13) The image processing apparatus according to any one of (1) to (12), wherein the comparison unit

[0300] deriving a first relative position relationship between the partial areas by comparing the sensor 3D information between the partial areas,

[0301] deriving a second relative positional relationship between the partial regions by comparing the estimated 3D information between the partial regions, and

[0302] The first relative position relationship is compared with the second relative position relationship.

[0303] (14) The image processing apparatus according to any one of (1) to (13), wherein the comparison unit compares a difference between the 3D information of the partial area and 3D information of an area surrounding the partial area between the estimated 3D information and the sensor 3D information.

[0304] (15) The image processing apparatus according to (14), wherein:

[0305] The comparison unit compares a difference between 3D information of an object region, which is a partial region including an object that is main and a subject in the image, and 3D information of a region surrounding the object region.

[0306] (16) The image processing apparatus according to any one of (1) to (15), wherein the comparison unit compares the estimated 3D information with the sensor 3D information by using a plurality of techniques.

[0307] (17) The image processing apparatus according to (16), wherein the comparison unit selects any one of the obtained plurality of comparison results.

[0308] (18) The image processing apparatus according to (16), wherein the comparison unit combines the obtained plurality of comparison results.

[0309] (19) The image processing apparatus according to any one of (1) to (18), further comprising:

[0310] A rendering processing unit configured to generate rendering information indicating a determination result of the authenticity of the partial area of ​​the image or a determination result of the authenticity of the entire image based on a comparison result between the estimated 3D information obtained using the partial area and the sensor 3D information.

[0311] (20) The image processing apparatus according to (19), wherein the presentation information includes at least either visual information or auditory information.

[0312] (21) The image processing apparatus according to (19) or (20), wherein, in a case where the authenticity is doubtful based on the comparison result, the presentation processing unit generates presentation information including a warning indicating that the authenticity is doubtful.

[0313] (22) The image processing apparatus according to any one of (19) to (21), wherein, in a case where the authenticity is doubtful based on the comparison result, the presentation processing unit generates presentation information that clearly indicates the area where the authenticity is doubtful.

[0314] (23) The image processing device according to any one of (19) to (22), wherein the rendering processing unit generates rendering information that clearly indicates an object region that is a partial region including an object that is main and a subject in the image.

[0315] (24) The image processing apparatus according to any one of (19) to (23), wherein, when the image is a moving image,

[0316] The region division unit divides the entire region for each frame and generates the partial region,

[0317] The comparing unit compares the estimated 3D information with the sensor 3D information using the partial area for each frame, and

[0318] The presentation processing unit generates the presentation information based on the comparison result for each frame.

[0319] (25) The image processing device according to (24), wherein the presentation processing unit generates presentation information indicating a range in the time direction of a region where authenticity is questionable in the moving image.

[0320] (26) The image processing device according to (24) or (25), wherein the rendering processing unit generates a reduced image of a frame in which a region whose authenticity is questionable exists.

[0321] (27) The image processing apparatus according to any one of (19) to (26), further comprising:

[0322] A presentation unit is configured to present the presentation information.

[0323] (28) The image processing apparatus according to (19) to (27), wherein the presentation processing unit confirms validity of an electronic signature corresponding to the image and indicates a confirmation result in the presentation information.

[0324] (29) The image processing apparatus according to any one of (1) to (28), further comprising:

[0325] an estimating unit configured to estimate the 3D information corresponding to the image based on the image and generate the estimated 3D information, wherein

[0326] The comparison unit compares the estimated 3D information generated by the estimation unit with the sensor 3D information.

[0327] (30) The image processing apparatus according to any one of (1) to (29), further comprising:

[0328] A sensor 3D information acquisition unit, wherein the sensor 3D information acquisition unit is configured to acquire the sensor 3D information corresponding to the image, wherein:

[0329] The comparison unit compares the estimated 3D information with the sensor 3D information acquired by the sensor 3D information acquisition unit.

[0330] (31) The image processing apparatus according to any one of (1) to (30), further including:

[0331] An image file acquisition unit is configured to acquire an image file storing the image and the sensor 3D information corresponding to the image, wherein:

[0332] The area division unit divides an entire area of ​​an image stored in the image file, and the comparison unit compares the estimated 3D information with the sensor 3D information stored in the image file.

[0333] (32) The image processing apparatus according to (31), wherein:

[0334] In the case where the image is a processed image obtained by processing an original image,

[0335] The image file acquisition unit acquires the image file storing the original image,

[0336] the area division unit generates the partial area corresponding to the processed image by dividing the entire area of ​​the original image stored in the image file, and

[0337] The comparison unit compares the sensor 3D information of the partial area with estimated 3D information, where the estimated 3D information is estimated based on the original image.

[0338] (33) The image processing apparatus according to (32), wherein the processed image is a cropped image obtained by cropping a portion of the original image.

[0339] (34) An image processing method, comprising:

[0340] generating a partial region by dividing the entire region of the image; and

[0341] The estimated 3D information is compared with the sensor 3D information using the partial area, wherein:

[0342] The estimated 3D information is 3D information estimated based on the image, and

[0343] The sensor 3D information is 3D information acquired by a sensor and associated with the image.

[0344] Reference Symbols List

[0345] 100 Authenticity determination device

[0346] 101 Control Unit

[0347] 111 Image Analysis Engine

[0348] 112 Input / output interface unit

[0349] 131 Image file acquisition unit

[0350] 132 3D information estimation unit

[0351] 133 Regional Division Unit

[0352] 134 Sensor 3D information acquisition unit

[0353] 135 Comparison Unit

[0354] 136 Rendering Processing Unit

[0355] 151 Input unit

[0356] 152 Presentation Unit

[0357] 153 Storage Units

[0358] 154 Communication Unit

[0359] 155 Driver

[0360] 400 Image Processing System

[0361] 401 Network

[0362] 411 Authenticity confirmed server

[0363] 412 Storage Server

[0364] 413 Imaging Device

[0365] 414 Terminal Device

[0366] 900 Computer

Claims

1. An image processing device, comprising: a region dividing unit configured to generate a partial region by dividing the entire region of the image; and A comparison unit configured to compare the estimated 3D information with the sensor 3D information using the partial area, wherein: The estimated 3D information is 3D information estimated based on the image, and The sensor 3D information is 3D information acquired by a sensor and associated with the image.

2. The image processing apparatus according to claim 1, wherein: The region division unit detects an object that is main and serves as a subject in the image, and divides the entire region based on the detection result.

3. The image processing apparatus according to claim 1, wherein: The region dividing unit divides the entire region by clustering the estimated 3D information.

4. The image processing apparatus according to claim 1, wherein: The comparison unit compares the sensor 3D information with the estimated 3D information for each of the partial regions.

5. The image processing apparatus according to claim 1, wherein: The comparison unit deriving a first relative position relationship between the partial areas by comparing the sensor 3D information between the partial areas, deriving a second relative positional relationship between the partial regions by comparing the estimated 3D information between the partial regions, and The first relative position relationship is compared with the second relative position relationship.

6. The image processing apparatus according to claim 1, wherein: The comparison unit compares a difference between the 3D information of the partial area and the 3D information of an area around the partial area between the estimated 3D information and the sensor 3D information.

7. The image processing apparatus according to claim 1, wherein: The comparison unit compares the estimated 3D information with the sensor 3D information by using a plurality of techniques.

8. The image processing device according to claim 1, further comprising: A rendering processing unit configured to generate rendering information indicating a determination result of the authenticity of the partial area of ​​the image or a determination result of the authenticity of the entire image based on a comparison result between the estimated 3D information obtained using the partial area and the sensor 3D information.

9. The image processing apparatus according to claim 8, wherein: The presentation information includes at least any one of visual information or auditory information.

10. The image processing apparatus according to claim 8, wherein: In a case where the authenticity is doubtful based on the comparison result, the presentation processing unit generates the presentation information including a warning indicating that the authenticity is doubtful.

11. The image processing apparatus according to claim 8, wherein: In a case where the authenticity is doubtful based on the comparison result, the presentation processing unit generates the presentation information clearly indicating the area where the authenticity is doubtful.

12. The image processing apparatus according to claim 8, wherein: In the case where the image is a moving image, The region division unit divides the entire region for each frame and generates the partial region, The comparison unit compares the estimated 3D information with the sensor 3D information using the partial area for each frame, and The presentation processing unit generates the presentation information based on the comparison result for each frame.

13. The image processing apparatus according to claim 12, wherein: The presentation processing unit generates presentation information indicating a range in the time direction of a region where authenticity is questionable in the moving image.

14. The image processing apparatus according to claim 12, wherein: The rendering processing unit generates a reduced image of a frame in which an area of ​​questionable authenticity exists.

15. The image processing device according to claim 8, further comprising: A presentation unit is configured to present the presentation information.

16. The image processing apparatus according to claim 8, wherein: The presentation processing unit confirms the validity of the electronic signature corresponding to the image and indicates the confirmation result in the presentation information.

17. The image processing device according to claim 1, further comprising: an estimating unit configured to estimate the 3D information corresponding to the image based on the image and generate the estimated 3D information, wherein The comparison unit compares the estimated 3D information generated by the estimation unit with the sensor 3D information.

18. The image processing device according to claim 1, further comprising: A sensor 3D information acquisition unit, wherein the sensor 3D information acquisition unit is configured to acquire the sensor 3D information corresponding to the image, wherein: The comparison unit compares the estimated 3D information with the sensor 3D information acquired by the sensor 3D information acquisition unit.

19. The image processing device according to claim 1, further comprising: An image file acquisition unit is configured to acquire an image file storing an original image and the sensor 3D information corresponding to the original image, wherein: The area division unit generates the partial area corresponding to the processed image obtained by processing the original image by dividing the entire area of ​​the original image stored in the image file, and The comparison unit compares the sensor 3D information of the partial area with estimated 3D information, where the estimated 3D information is estimated based on the original image.

20. An image processing method, comprising: Generate partial regions by dividing the entire region of the image; and The estimated 3D information is compared with the sensor 3D information using the partial area, wherein: The estimated 3D information is 3D information estimated based on the image, and The sensor 3D information is 3D information acquired by a sensor and associated with the image.

Images