Image processing device

Abstract

An image processing device (10) according to the present disclosure comprises: a determination unit (11) that, on the basis of the image feature amount of images in an image group comprising a plurality of reference images and a query image, the camera attitude at the time of capturing each of the images, and the number of three-dimensional points captured in common between each of the images and the other images in the image group, calculates the similarity degree, for each of the images in the image group, between each of the images and the other images in the image group, and determines, as a deletion candidate image, an image having the highest similarity degree to the other images from among the images in the image group; and an update unit (12) that, when the deletion candidate image is a reference image, deletes the deletion candidate image from a reference DB and stores the query image as a reference image in the reference DB.

Description

Image Processing Device

[0001] The present disclosure relates to an image processing device.

[0002] Non-Patent Document 1 describes a technique for estimating the camera pose (position and orientation of the camera at the time of capturing) of an input image (query image) captured by a camera. In the technique described in Non-Patent Document 1, a reference database (hereinafter referred to as a "reference DB") is prepared in advance, which stores reference images, which are images whose camera pose at the time of capturing is known. Then, in the technique described in Non-Patent Document 1, a reference image that has an appearance similar to the query image is selected from the reference images stored in the reference DB, and the camera pose of the query image is estimated from the camera pose of the selected reference image, etc.

[0003] In the technology described in Non-Patent Document 1, in order to ensure the accuracy of camera pose estimation, it is necessary to store many reference images with different camera poses in a reference DB. Furthermore, it is desirable that the reference images are recent images that are close to the situation in which the query image is captured.

[0004] As a reference image with a known camera orientation, an image acquired by a mobile mapping system (MMS) or surveying can be used. However, an MMS is an expensive system that uses a vehicle equipped with dedicated equipment. Furthermore, surveying requires personnel costs for on-site work. Therefore, it is difficult to frequently acquire reference images by an MMS or surveying, and a method for timely updating a reference DB has not been established.

[0005] P. -E. Sarlin, C. Cadena, R. Siegwart and M. Dymczyk, "From Coarse to Fine: Robust Hierarchical Localization at Large Scale," 2019 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 12708-12717, doi: 10.1109 / CVPR.2019.01300.

[0006] In outdoor environments, the landscape changes due to the construction, demolition, and reconstruction of buildings, the expansion of roads and sidewalks, the installation, removal, relocation, and redesign of signs, convex mirrors, streetlights, utility poles, and store signs, the replanting of street trees and vegetation, and seasonal changes. When estimating the camera pose of a query image captured in an outdoor environment, there is a problem that the estimation accuracy drops significantly if the reference image and the actual landscape differ. In particular, when camera pose estimation is used for applications such as infrastructure inspection, long-term operation over 10 or 20 years is desired, but the inability to respond to changes in the landscape during that time poses a major problem, and the decrease in camera pose estimation accuracy becomes a major issue.

[0007] Therefore, it is desirable to change the reference images stored in the reference DB to current images as appropriate in accordance with changes in the scenery, but as mentioned above, it is not realistic from a cost perspective to frequently obtain reference images using MMS or surveying, etc.

[0008] Furthermore, increasing the number of reference images stored in the reference DB too much increases the cost of maintaining the data and the calculation cost of estimating the camera pose. Therefore, it is desirable to update the reference DB appropriately (replace the reference images) so as to ensure the accuracy of estimating the camera pose within the allowable number of reference images. However, determining whether or not to replace the reference images and determining the reference images to be replaced is not easy and is a costly task.

[0009] In consideration of the above-mentioned problems, the purpose of the present disclosure is to provide an image processing device that can suppress the increase in reference images stored in a database that stores reference images, while updating the database, thereby suppressing the decrease in the estimation accuracy of the camera posture.

[0010] An image processing device according to one embodiment is an image processing device that updates a database storing a plurality of reference images captured by a camera, the reference images being of known camera orientation at the time of capture, with a query image captured by the camera, and is equipped with: a determination unit that calculates, for each image in the image group consisting of the plurality of reference images and the query image, the similarity between that image and the other images in the image group based on the image features of each image in the image group, the camera orientation at the time each image was captured, and the number of three-dimensional points that appear in each image and the other images in the image group, and determines, from among the images in the image group, the image that has the highest similarity to the other images as a candidate image for deletion; and an update unit that, if the candidate image for deletion is the reference image, deletes the candidate image for deletion from the database and stores the query image in the database as a reference image.

[0011] According to the present disclosure, it is possible to suppress an increase in the number of reference images stored in the database, while updating the database, thereby suppressing a decrease in the estimation accuracy of the camera posture.

[0012] Fig. 2 is a diagram illustrating an example of the configuration of an image processing device according to an embodiment of the present disclosure. Fig. 3 is a flowchart illustrating an example of the operation of the image processing device shown in Fig. 1. Fig. 4 is a diagram illustrating the operation of the image processing device shown in Fig. 1. Fig. 5 is a diagram illustrating calculation of similarity by a similarity calculation unit shown in Fig. 1. Fig. 6 is a diagram illustrating calculation of similarity by a similarity calculation unit shown in Fig. 1. Fig. 7 is a diagram illustrating an example of the configuration of a computer that functions as an image processing device according to the present disclosure.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0014] 1 is a diagram illustrating an example of the configuration of an image processing device 10 according to an embodiment of the present disclosure. The image processing device 10 according to the present disclosure updates a reference DB, which is a database that stores multiple reference images whose camera postures (camera positions and orientations) at the time of image capture are known, with a query image captured by a camera.

[0015] As shown in FIG. 1, an image processing apparatus 10 according to this embodiment includes a determination unit 11 and an update unit 12.

[0016] A query image captured by a camera is input to the determination unit 11. The determination unit 11 calculates, for each image, a similarity between the image and other images in the image group based on image feature values ​​of each image in the image group consisting of a plurality of reference images and the query image, the camera orientation at the time of capturing each image, and the number of three-dimensional points commonly captured in each image and the other images in the image group. The determination unit 11 then determines, from among the images in the image group, an image that has the highest similarity to the other images as a deletion candidate image. As shown in FIG. 1 , the determination unit 11 includes a search unit 111, an orientation estimation unit 112, and a similarity calculation unit 113.

[0017] The search unit 111 searches for reference images (hereinafter referred to as "first similar images") similar to the query image based on image features of the plurality of reference images and the query image. Specifically, the search unit 111 extracts image features of the plurality of reference images and the query image using a deep neural network (DNN) or the like. Then, the search unit 111 searches for a plurality of reference images (M (M>1)) similar to the query image (first similar images) from among the plurality of reference images stored in the reference DB using a method such as vector of locally aggregated descriptors (VLAD) that evaluates the similarity between descriptors that describe image features.

[0018] The pose estimation unit 112 estimates the camera pose when capturing the query image. The pose estimation unit 112 estimates the camera pose of the query image using, for example, a feature matching method or a scene coordinate regression method. Then, the pose estimation unit 112 selects, from the plurality of first similar images, a plurality of (N (M≧N>1)) reference images (second similar images) whose camera poses are similar to that of the query image. For example, the pose estimation unit 112 selects, from the first similar images, N reference images that have a small Euclidean distance from the query image as the second similar images.

[0019] For each image in the image group consisting of the second similar image and the query image, the similarity calculation unit 113 calculates the similarity between the image and the other images in the image group based on image feature amounts of the image and the other images in the image group, camera attitudes of the image and the other images in the image group, and the number of three-dimensional points commonly captured in the image and the other images in the image group.The similarity calculation unit 113 then determines, from among the second similar image and the query image, the image that has the highest similarity to the other images in the image group as a candidate image for deletion.

[0020] In this manner, in this embodiment, a first similar image having image features similar to those of the query image and a second similar image having a camera pose similar to that of the query image are selected, and the similarity between each image and other images in the image group consisting of the selected images and the query image is calculated. This reduces calculation costs compared to calculating the similarity between each image and other images for all of the reference images and query images stored in the reference DB.

[0021] The update unit 12 determines whether to update the reference DB with the deletion candidate image determined by the determination unit 11. Updating the reference DB means deleting the deletion candidate image from the reference DB and storing the query image in the reference DB as a reference image. The update unit 12 updates the reference DB when the deletion candidate image is a reference image. That is, when the deletion candidate image is a reference image, the update unit 12 deletes the deletion candidate image from the reference DB and stores the query image in the reference DB as a reference image. When the update unit 12 adds the query image to the reference DB as a reference image, the update unit 12 may estimate the camera attitude (and the terrain map) of the image using a method such as Structure from Motion and correct the camera attitude in conjunction with other reference images stored in the reference DB.

[0022] Next, the operation of the image processing device 10 according to this embodiment will be described. Fig. 2 is a flowchart showing an example of the operation of the image processing device 10 according to this embodiment, and is a diagram for explaining an image processing method executed by the image processing device 10 according to this embodiment.

[0023] The determination unit 11 calculates, for each image, the degree of similarity between the image and other images in the image group based on the image feature values ​​of each image in the image group, which is made up of a plurality of reference images and a query image stored in the reference DB, and the camera posture at the time of capturing each image in the image group. Then, the determination unit 11 determines, from among the images in the image group, the image that has the highest degree of similarity to the other images as a candidate image for deletion (step S11).

[0024] Specifically, the determination unit 11 (search unit 111) searches for a plurality of first similar images, which are reference images similar to the query image, based on image feature amounts of the plurality of reference images and the query image. Next, the determination unit 11 (posture estimation unit 112) estimates the camera posture at the time of capturing the query image. Then, the posture estimation unit 112 selects a plurality of second similar images, from the plurality of first similar images, whose camera postures are similar to that of the query image.

[0025] The determination unit 11 (similarity calculation unit 113) calculates, for each image in the image group consisting of the second similar image and the query image, the similarity between the image and the other images in the image group based on the image feature amounts of the image and the other images in the image group, the camera orientations of the image and the other images in the image group, and the number of three-dimensional points commonly captured in the image and the other images in the image group.The similarity calculation unit 113 then determines, from the second similar image and the query image, the image that has the highest similarity to the other images in the image group as a candidate image for deletion.

[0026] The similarity calculation unit 113 calculates the similarity S between one image i in the image group consisting of the second similar image and the query image and another image j in the image group. ij is calculated based on the following formula (1).

[0027]

[0028] In formula (1), R i , R j is the camera rotation matrix in the world coordinate system when images i and j were taken, and d ｉｊ is the L2 norm of the camera position at the time of capturing images between images i and j, and d ｍａｘ , d ｍｉｎare the maximum and minimum values ​​of the L2 norm between the camera positions when the second similar image and the query image were captured, respectively, and l ij is the distance (e.g., L2 norm) between the image features of image i and image j in the feature space, and l ｍａｘ , l ｍｉｎ is the distance l between the image features of two images in the feature space, the second similar image and the query image. ij The maximum and minimum values ​​of k are 1 or more. ij is the number of 3D points commonly seen in image i and image j, and N ｍａｘ , N ｍｉｎ are the maximum and minimum values ​​of the number of 3D points that appear in two images, the second similar image and the query image. Note that by determining the correspondence between the reference image and the 3D points when preparing the reference DB, the number of 3D points that appear in common between the query image and the reference image can be determined from the correspondence between the image feature points in the query image and the reference image.

[0029] The similarity calculation unit 113 calculates the above-mentioned similarity for each of all images in the image group consisting of the second similar image and the query image, and for each of the other images in the image group. The similarity calculation unit 113 compares the calculated similarities and determines the image with the highest similarity to the other images as a deletion candidate image. The similarity calculation unit 113 determines the deletion candidate image by, for example, comparing the maximum similarity or average similarity with the other images calculated for each image in the image group consisting of the second similar image and the query image.

[0030] The update unit 12 determines whether the deletion candidate image is a reference image that has already been stored in the reference DB (step S12).

[0031] If it is determined that the deletion candidate image is not a reference image (i.e., the deletion candidate image is a query image) (step S12: No), the update unit 12 determines that updating of the reference DB is not necessary and terminates the process.

[0032] If it is determined that the deletion candidate image is a reference image (step S12: Yes), the update unit 12 updates the reference DB (step S13). Specifically, the update unit 12 deletes the deletion candidate image from the reference DB and stores the query image in the reference DB as a reference image.

[0033] 3 is a diagram schematically illustrating the operation of the image processing device 10 according to this embodiment. In FIG. 3, an example will be described in which three reference images R1 to R3 are stored in the reference DB. As shown in FIG. 3, the three reference images R1 to R3 partially overlap each other.

[0034] A query image q is input, and as shown in FIG. 3 , the query image q overlaps with each of the reference images R1 to R3. The similarity calculation unit 113 calculates the similarity between each of the reference images R1 to R3 and the query image q and the other images. The similarity calculation unit 113 then determines, as a deletion candidate image, the image that has the highest similarity to the other images among the reference images R1 to R3 and the query image q. The image that has the highest similarity to the other images is the image that overlaps the most with the other images. In the example shown in FIG. 3 , the overlap between the reference image R2 and the images other than the reference image R2 (reference images R1, R3, and query image q) is the highest. Therefore, the similarity calculation unit 113 determines the reference image R2 as a deletion candidate image.

[0035] The update unit 12 determines whether the deletion candidate image is a reference image. In the example shown in FIG. 3 , the deletion candidate image is reference image R2, so the reference DB is updated. Specifically, as shown in FIG. 3 , the update unit 12 deletes reference image R2, which has been determined as the deletion candidate image, from the reference DB and stores the query image q in the reference DB as a new reference image R4. Since the deletion candidate image is deleted and then the query image is added to the reference DB as a new reference image, an increase in the number of reference images stored in the reference DB can be suppressed.

[0036] It is preferable to store reference images with little overlap in the reference DB. When there is little overlap between the query image and the reference image, there is a high possibility that an area not captured in the reference image is captured in the query image. By adding such a query image as a reference image to the reference DB, it is possible to increase the variety of reference images stored in the reference DB.

[0037] As shown in Figure 3, one possible method for identifying images that overlap significantly with other images is to use a similarity calculated based only on the image features of each image in the image group (e.g., the distance in the feature space between the image features of each image). However, this method may result in images that have different image features but are captured at close locations (e.g., images captured at locations where the scenery changes significantly, such as an intersection) being swapped with each other. In this case, the capture locations of the reference images may be concentrated, resulting in a decrease in the accuracy of estimating the camera pose.

[0038] Another method for identifying images that overlap with other images is to use a similarity calculated based only on the camera pose of each image in the image group. However, this method uses images that have similar appearances (e.g., images captured at locations with similar scenery, such as buildings or forests) as reference images, even though they were taken at different angles and locations, resulting in a decrease in the accuracy of the camera pose estimation.

[0039] Another possible method for identifying images that overlap significantly with other images is to use a similarity calculated based on the image feature values ​​and camera poses of each image in the image group. By taking the image feature values ​​and camera poses into consideration, this method makes it possible to replace the reference image (update the reference DB) without reducing the accuracy of the camera pose estimation, even when the query image is captured in a location where similar scenery continues or at a location such as an intersection where the scenery changes significantly.

[0040] Here, as shown in FIG. 4A, assume that multiple query images q1 and q2 (two in the example shown in FIG. 4A) are input. Furthermore, assume that the query images q1 and q2 have approximately the same inter-camera distance and angle (similar camera postures) with respect to one reference image R. In this case, ambiguity remains because the appearance of each image is evaluated based only on image features, and it may not be possible to properly update the reference DB.

[0041] On the other hand, in this embodiment, the similarity calculation unit 113 uses similarity calculated based on the image feature values, camera pose, and the number of 3D points commonly captured with other images for each image in the image group. Here, as shown in FIG. 4B , for example, assume that the number of 3D points on object o1 commonly captured in query image q1 and reference image R is two, and the number of 3D points on object o1 commonly captured in query image q2 and reference image R is one. In this case, the overlap between query image q2 and reference image R is smaller than the overlap between query image q1 and reference image R, and there is a possibility that an object o2 other than object o1 is also captured in query image q2. As described above, it is desirable for the reference image to have less overlap with other reference images. Therefore, in the example shown in FIG. 4B , it is desirable to store query image q2 as the reference image in the reference DB rather than query image q1. In this embodiment, in addition to the image features and camera pose of each image in the image group, the number of three-dimensional points that are common to other images is also taken into consideration, so even if multiple query images with similar camera poses are input for one reference image R, the reference DB can be updated with a more appropriate query image.

[0042] As described above, the image processing device 10 according to this embodiment includes a determination unit 11 and an update unit 12. The determination unit 11 calculates, for each image in the image group consisting of a plurality of reference images and a query image, the similarity between that image and the other images in the image group based on the image feature values ​​of each image, the camera orientation at the time of capturing each image, and the number of three-dimensional points commonly captured in each image and the other images in the image group. The determination unit 11 then determines, from among the images in the image group, an image that has the highest similarity to the other images as a deletion candidate image. If the deletion candidate image is a reference image, the update unit 12 deletes the deletion candidate image from the reference DB and stores the query image in the reference DB as a reference image.

[0043] After deleting the deletion candidate images, the query image is stored in the reference DB as a reference image, thereby suppressing an increase in the number of reference images stored in the reference DB. Furthermore, by determining the deletion candidate images based on the similarity with other images calculated based on the image features, camera pose, and the number of commonly captured 3D points, even when multiple query images with similar camera poses and image features are input, the reference DB can be updated with query images that have little overlap with the reference images, thereby suppressing a decrease in the accuracy of camera pose estimation. Therefore, according to the image processing device 10 of this embodiment, it is possible to suppress an increase in the number of reference images stored in the reference DB that stores the reference images, while updating the reference DB, thereby suppressing a decrease in the accuracy of camera pose estimation.

[0044] Furthermore, according to the image processing device 10 of this embodiment, even if an initial reference DB is constructed using only reference images acquired using an MMS, for example, images captured by a smartphone, drone, robot, or the like can be added to the reference DB as reference images as appropriate when updating the reference DB. In other words, according to the image processing device 10 of this embodiment, a reference DB capable of estimating camera pose with high accuracy can be constructed regardless of the imaging technique. Constructing a reference DB for each imaging technique would incur significant operational costs. As in this embodiment, a reference DB can be constructed using reference images acquired using an MMS, for example, and the reference DB can be used in a system for estimating camera pose, while updating the reference DB with query images obtained using various imaging techniques.

[0045] In addition, the reference DB that stores reference images for which the camera orientation at the time of shooting is known may be used not only for estimating the camera orientation, but also for, for example, creating and updating three-dimensional city models or maps, and for position recognition in AR (Augmented Reality).

[0046] The image processing device 10 described above can be realized by a computer 20 shown in FIG. 5. A program for causing the computer 20 to function as the image processing device 10 may be provided. The program may be stored in a storage medium or provided via a network. FIG. 5 is a block diagram showing a schematic configuration of the computer 20 functioning as the image processing device 10. The computer 20 may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, or the like. The program instructions may be program code, code segments, or the like for executing necessary tasks.

[0047] 5, the computer 20 includes a processor 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a storage 24, an input unit 25, a display unit 26, and a communication interface (I / F) 27. Each component is communicably connected to one another via a bus 29. The processor 21 is specifically a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a SoC (System on a Chip), or the like, and may be configured with multiple processors of the same or different types.

[0048] The processor 21 is a control unit that controls each component and performs various arithmetic processing. That is, the processor 21 reads a program from the ROM 22 or the storage 24 and executes the program using the RAM 23 as a work area. The processor 21 controls each component and performs various arithmetic processing in accordance with the program stored in the ROM 22 or the storage 24. In this embodiment, the ROM 22 or the storage 24 stores a program for operating the computer 20 as the image processing device 10 according to the present disclosure. The program is read and executed by the processor 21 to realize the determination unit 11 and the update unit 12 of the image processing device 10.

[0049] The program may be provided in a form stored on a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), a USB (Universal Serial Bus) memory, etc. The program may also be provided in a form downloaded from an external device via a network.

[0050] The ROM 22 stores various programs and various data. The RAM 23 temporarily stores programs or data as a working area. The storage 24 is configured with an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0051] The input unit 25 includes a pointing device such as a mouse and a keyboard, and is used to input various types of information.

[0052] The display unit 26 is, for example, a liquid crystal display, and displays various information. The display unit 26 may be a touch panel type and function as the input unit 25.

[0053] The communication interface 27 is an interface for communicating with other devices, for example, an interface for a LAN.

[0054] The following additional notes are provided regarding the above-described embodiments.

[0055] [Supplementary Item 1] An image processing device that updates a database that stores a plurality of reference images that were captured by a camera and whose camera orientation at the time of capturing the images is known, with a query image captured by the camera, comprising: a control unit, wherein the control unit calculates, for each image in the image group consisting of the plurality of reference images and the query image, a similarity between the image and other images in the image group based on image feature amounts of each image in the image group, the camera orientation at the time of capturing each of the images, and the number of three-dimensional points that appear in each of the images and the other images in the image group, and determines, from among the images in the image group, an image that has the highest similarity to the other images as a candidate image for deletion; and if the image that is candidate for deletion is the reference image, deletes the image that is candidate for deletion from the database, and stores the query image in the database as a reference image.

[0056] [Supplementary Item 2] In the image processing device according to Supplementary Item 1, the control unit searches for a plurality of first similar images from among the plurality of reference images, which are reference images similar to the query image, based on image feature amounts of each of the plurality of reference images and the query image; estimates a camera attitude at the time of capturing the query image, and selects a plurality of second similar images from among the plurality of first similar images, which have a camera attitude similar to that of the query image; calculates, for each image in an image group consisting of the plurality of second similar images and the query image, a similarity between the image and the other images in the image group based on image feature amounts of the image and the other images in the image group, the camera attitudes of the image and the other images in the image group, and the number of three-dimensional points commonly captured in the image and the other images in the image group; and determines, from among the second similar images and the query image, an image having the highest similarity to the other images in the image group as the deletion candidate image.

[0057] [Supplementary Item 3] An image processing method executed by an image processing device that updates a database storing a plurality of reference images captured by a camera, the reference images being of known camera orientation at the time of capturing the images, with a query image captured by the camera, the image processing method comprising: calculating, for each image in the image group consisting of the plurality of reference images and the query image, a similarity between that image and the other images in the image group based on image feature amounts of that image and the other images in the image group, the camera orientation at the time of capturing each of the images, and the number of three-dimensional points that appear in each of the images and the other images in the image group; determining, from among the images in the image group, an image that has the highest similarity to the other images as a candidate image for deletion; and if the candidate image for deletion is the reference image, deleting the candidate image for deletion from the database, and storing the query image in the database as a reference image.

[0058] [Supplementary Item 4] A non-transitory storage medium storing a program executable by a computer, the non-transitory storage medium storing the program causing the computer to operate as the image processing device according to Supplementary Item 1 or 2.

[0059] Although the above-described embodiments have been described as typical examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present disclosure. Therefore, the present invention should not be interpreted as being limited by the above-described embodiments, and various modifications and alterations are possible without departing from the scope of the claims. For example, multiple building blocks shown in the block diagrams of the embodiments can be combined into one, or one building block can be divided.

[0060] REFERENCE SIGNS LIST 10 Image processing device 11 Determination unit 12 Update unit 111 Search unit 112 Posture estimation unit 113 Similarity calculation unit 20 Computer 21 Processor 22 ROM 23 RAM 24 Storage 25 Input unit 26 Display unit 27 Communication I / F 29 Path

Claims

1. An image processing device that updates a database that stores multiple reference images captured by a camera, the reference images being known at the time of capture, with a query image captured by the camera, the image processing device comprising: a determination unit that calculates, for each image in the image group consisting of the multiple reference images and the query image, the similarity between that image and the other images in the image group based on the image feature values ​​of each image in the image group, the camera orientation at the time each image was captured, and the number of three-dimensional points that appear in each image and the other images in the image group, and determines, from among the images in the image group, the image that has the highest similarity to the other images as a candidate image for deletion; and an update unit that, if the candidate image for deletion is the reference image, deletes the candidate image for deletion from the database and stores the query image in the database as a reference image.

2. An image processing device according to claim 1, wherein the determination unit comprises: a search unit that searches for a plurality of first similar images, which are reference images similar to the query image, from among the plurality of reference images based on image feature amounts of each of the plurality of reference images and the query image; a posture estimation unit that estimates the camera posture at the time of capturing the query image and selects a plurality of second similar images from among the plurality of first similar images, whose camera postures are similar to that of the query image; and a similarity calculation unit that, for each image in an image group consisting of the plurality of second similar images and the query image, calculates the similarity between the image and the other images in the image group based on the image feature amounts of the image and the other images in the image group, the camera postures of the image and the other images in the image group, and the number of three-dimensional points commonly captured in the image and the other images in the image group, and determines, from among the second similar images and the query image, the image that has the highest similarity to the other images in the image group as the deletion candidate image.

Images