Image matching apparatus, image matching method, and non-transitory computer-readable medium

Abstract

The present disclosure provides an image matching apparatus that performs: acquiring a ground image; selecting one of two or more search methods based on at least one of selection criteria that includes a size of a field of view of the ground image, a degree of similarity of aerial scenes in a search area, or both; and searching, using the selected search method, for an aerial image that matches the ground image from among two or more of the aerial images in each of which an aerial scene of a part of the search area is captured.

Description

IMAGE MATCHING APPARATUS, IMAGE MATCHING METHOD, AND NON-TRANSITORY COMPUTER-READABLE MEDIUM

[0001] The present disclosure generally relates to an image matching apparatus, an image matching method, and a non-transitory computer-readable medium.

[0002] A computer system for performing data matching has been developed. For example, NPL1 discloses a system comprising a set of CNNs (Convolutional Neural Networks) to match a ground image against an aerial image. Specifically, one of the CNNs acquires a set of a ground image and orientation maps that indicate orientations (azimuth and altitude) for each location captured on the ground image, and extracts features therefrom. The other one acquires a set of an aerial image and orientation maps that indicate orientations (azimuth and range) for each location captured on the aerial image, and extracts features therefrom. Then, the system determines whether the ground image matches the aerial image based on the extracted features.

[0003] NPL1: Liu Liu and Hongdong Li, "Lending Orientation to Neural Networks for Cross-view Geo-localization", [online], March 29, 2019, [retrieved on 2024-7-30], retrieved from <arXiv, https: / / arxiv.org / pdf / 1903.12351.pdf>

[0004] In NPL1, it is assumed that there is a single fixed method for searching an aerial image that matches the given ground image. An objective of the present disclosure is to provide a novel technique to search for an aerial image that matches a given ground image.

[0005] The present disclosure provides an image matching apparatus comprising at least one memory that is configured to store instructions and at least one processor. 　　The at least one processor is configured to execute the instructions to: acquire a ground image; select one of two or more search methods based on at least one of selection criteria that includes a size of a field of view of the ground image, a degree of similarity of aerial scenes in a search area, or both; and search, using the selected search method, for an aerial image that matches the ground image from among two or more of the aerial images in each of which an aerial scene of a part of the search area is captured.

[0006] The present disclosure further provides an image matching method that is performed by a computer. 　　The image matching method comprise: acquiring a ground image; selecting one of two or more search methods based on at least one of selection criteria that includes a size of a field of view of the ground image, a degree of similarity of aerial scenes in a search area, or both; and searching, using the selected search method, for an aerial image that matches the ground image from among two or more of the aerial images in each of which an aerial scene of a part of the search area is captured.

[0007] The present disclosure further provides a non-transitory computer-readable medium storing a program that cause a computer to execute: acquiring a ground image; selecting one of two or more search methods based on at least one of selection criteria that includes a size of a field of view of the ground image, a degree of similarity of aerial scenes in a search area, or both; and searching, using the selected search method, for an aerial image that matches the ground image from among two or more of the aerial images in each of which an aerial scene of a part of the search area is captured.

[0008] According to the present disclosure, it is possible to provide a novel technique to search for an aerial image that matches a given ground image.

[0009] Fig. 1 illustrates an overview of an image matching apparatus.Fig. 2 illustrates an example of the search methods that differ in the strides used for the sliding window.Fig. 3 is a block diagram showing an example of the functional configuration of the image matching apparatus.Fig. 4 is a block diagram illustrating an example hardware configuration of a computer used to implement the image matching apparatus.Fig. 5 illustrates a geo-localization system that includes the image matching apparatus.Fig. 6 is a flowchart illustrating an example flow of processes performed by the image matching apparatus.Fig. 7 illustrates the case where the search methods differ in the window sizes of the sliding window.Fig. 8 is a flowchart illustrating an example flow of the search performed by the searching unit in the case where the search methods differ in the ways of acquiring the aerial images.Fig. 9 illustrates a case where the search methods differ in the rotation angle intervals of the aerial images.Fig. 10 is a flowchart illustrating an example flow of the search performed by the searching unit in the case where the search methods differ in the rotation angle intervals of the aerial image.Fig. 11 illustrates a case where the search methods differ in the size of the feature values.Fig. 12 is a flowchart illustrating an example flow of the search performed by the searching unit in the case where the search methods differ in the size of the feature values.

[0010] Example embodiments according to the present disclosure will be described hereinafter with reference to the drawings. The same numeral signs are assigned to the same elements throughout the drawings, and redundant explanations are omitted as necessary. In addition, predetermined information (e.g., a predetermined value or a predetermined threshold) is stored in advance in a storage device to which a computer using that information has access unless otherwise described.

[0011] FIRST EXAMPLE EMBODIMENT <Overview> 　　Fig. 1 illustrates an overview of an image matching apparatus 2000. It is noted that Fig. 1 does not limit operations of the image matching apparatus 2000, but merely show an example of possible operations of the image matching apparatus 2000.

[0012] The image matching apparatus 2000 acquires a ground image 10, and searches for an aerial scene that matches the scene in the ground image 10. The ground image 10 is a digital image (e.g., RGB image or grayscale image) that includes a ground view of a place. In other words, the ground image 10 is a digital image captured from a ground-level perspective. The ground image 10 is generated by a ground camera that may be held by a pedestrian or installed on a vehicle.

[0013] To search for an aerial scene that matches the scene in the ground image 10, the image matching apparatus 2000 also acquires two or more aerial images 20 to be compared with the ground image 10. The aerial image 20 is a digital image (e.g., RGB image or grayscale image) that includes an aerial view (i.e., top view) of a place. In other words, the aerial image 20 is a digital image captured from an elevated perspective. For example, the aerial image 20 is generated by an aerial camera that may be installed on a drone, an airplane, or a satellite.

[0014] To perform the search, the image matching apparatus 2000 selects one of two or more search methods that the image matching apparatus 2000 uses to search for the aerial image 20 that matches the ground image 10. The search methods differ in the level of detail of the search from each other. Hereinafter, each search method is denoted as the i-th search method, where a smaller value of the identifier i indicates a higher level of detail of the search: i.e., the level of detail of the search using the i-th search method is higher than the level of detail of the search using the (i+1)-th search.

[0015] In some cases, the search methods differ in the level of detail in the acquisition of the aerial images 20. Suppose that the aerial images 20 are extracted using a sliding window from a source aerial image, which is an aerial image larger than each aerial image 20. In this case, the search methods may differ in the strides used for the sliding window. More specifically, the smaller the identifier i of the search method is, the smaller the stride used by the i-th search method is.

[0016] Fig. 2 illustrates an example of the search methods that differ in the strides used for the sliding window. In Fig. 2, each aerial image 20 is extracted using a sliding window 40 from a source aerial image 30.

[0017] The first search method uses a sliding window 40 with stride S1, while the second search method uses a sliding window 40 with stride S2, where S1 is smaller than S2. Since S1 is smaller than S2, the sliding window 40 with the stride S1 extracts the aerial images 20 from the source aerial image 30 at finer intervals than the sliding window 40 with the stride S2 does. Thus, the first search method provides a more detailed search than the second search method does.

[0018] The image matching apparatus 2000 selects one of the search methods based on one or more of selection criteria. The selection criteria include the size of the field of view of the ground image 10, the similarity of the aerial scenes in a search area, or both. The search area is an area from which the image matching apparatus 2000 searches for an aerial scene that matches the scene in the ground image 10. Each aerial image 20 includes the aerial scene of a part of the search area. It is noted that the search area may be predefined or may be specified by a user of the image matching apparatus 2000.

[0019] After selecting the search method, the image matching apparatus 2000 uses the selected search method to search for the aerial image 20 that matches the ground image 10 from among two or more aerial images 20. It is noted that "the aerial image 20 matches the ground image 10" means that the aerial scene in the aerial image 20 matches the ground scene in the ground image 10. As described in detail later, the image matching apparatus 2000 may determine that the aerial image 20 matches the ground image 10 in the case where a feature value extracted from the aerial image 20 is substantially similar to a feature value extracted from the ground image 10: e.g., a degree of the similarity between the feature value of the aerial image 20 and the feature value of the ground image 10 is larger than or equal to a predefined threshold.

[0020] <Example of Advantageous Effect> 　　According to the image matching apparatus 2000, the search method for searching for an aerial image 20 that matches the ground image 10 is not fixed to a single method, but is selected from one of the multiple search methods. Thus, the present disclosure provides a novel technique to search for an aerial image that matches a given ground image.

[0021] As mentioned above, the search methods differ in their levels of detail in the search. The image matching apparatus 2000 can search for the aerial image 20 that matches the ground image 10 with greater accuracy by using the search method that provides a greater detail. However, search methods with greater detail tend to require more time, computational resources, or both. Thus, it is effective to select a search method that achieves a good balance between accuracy and computation time or the like.

[0022] To this end, the image matching apparatus 2000 uses the size of the field of view of the ground image 10, the similarity of the aerial scenes in a search area, or both as the selection criteria for determining an appropriate search method. These criteria represent the level of accuracy required for the search. As a result, the criteria may enable the image matching apparatus 2000 to select a search method that minimize time or other resource usage while still providing a sufficient level of detail to identify the aerial image 20 matching the ground image 10. Thus, the image matching apparatus 2000 is advantageous in that it can achieve a good balance between accuracy and computation time or the like.

[0023] Hereinafter, a more detailed explanation of the image matching apparatus 2000 will be described.

[0024] <Example of Functional Configuration> 　　Fig. 3 is a block diagram showing an example of the functional configuration of the image matching apparatus 2000. The image matching apparatus 2000 includes an acquiring unit 2020, a selecting unit 2040, and a searching unit 2060. The acquiring unit 2020 acquires the ground image 10. The selecting unit 2040 selects one of the search methods based on at least one of the selection criteria. The searching unit 2060 uses the selected search method to search for the aerial image 20 that matches the ground image 10.

[0025] <Example of Hardware Configuration> 　　The image matching apparatus 2000 may be implemented using one or more computers. Each of these computers can be a personal computer (PC), a server machine, a mobile device, or an integrated circuit, such as a system-on-chip (SoC). The computers may either be special-purpose computers designed specifically for the image matching apparatus 2000 or general-purpose computers.

[0026] The image matching apparatus 2000 can be realized by installing an application on the one or more computers. This application is a program designed to make the computers function as the image matching apparatus 2000. In other words, the program implements the functional components of the image matching apparatus 2000.

[0027] Fig. 4 is a block diagram illustrating an example hardware configuration of a computer 1000 used to implement the image matching apparatus 2000. As shown in Fig. 4, the computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output (I / O) interface 1100, and a network interface 1120.

[0028] The bus 1020 works as a data transmission channel, enabling the processor 1040, memory 1060, storage device 1080, I / O interface 1100, and network interface 1120 to exchange data. The processor 1040 may be a CPU (Central Processing Unit), MPU (Microprocessor Unit), GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), or DSP (Digital Signal Processor). The memory 1060 serves as a primary memory component, such as RAM (Random Access Memory) or ROM (Read-Only Memory). The storage device 1080 serves as a secondary memory component, such as a hard disk, SSD (Solid-State Drive), or memory card. The I / O interface 1100 connects the computer 1000 to peripheral devices, such as a keyboard, mouse, or display. The network interface 1120 connects the computer 1000 to a network, which can be a LAN (Local Area Network) or WAN (Wide Area Network).

[0029] The storage device 1080 may store the aforementioned program. The processor 1040 reads this program from the storage device 1080 and executes it, enabling the computer 1000 to implement the functional components of the image matching apparatus 2000.

[0030] The hardware configuration of the computer 1000 is not limited to the example shown in Fig. 4. For instance, as described above, the image matching apparatus 2000 may be implemented using multiple computers. In such cases, these computers can be connected to each other via a network.

[0031] <Example Application of Image Matching Apparatus 2000> 　　There are various possible applications of the image matching apparatus 2000. For example, the image matching apparatus 2000 can be used as a part of a system (hereinafter, a geo-localization system) that performs image geo-localization. Image geo-localization is a technique to determine the place at which an input image is captured. The geo-localization system may be implemented by one or more arbitrary computers such as ones depicted by Fig. 4. It is noted that the geo-localization system is merely an example of possible applications of the image matching apparatus 2000, and the application of the image matching apparatus 2000 is not restricted to being used in the geo-localization system.

[0032] Fig. 5 illustrates a geo-localization system 500 that includes the image matching apparatus 2000. The geo-localization system 500 includes the image matching apparatus 2000 and the location database 400. The location database 400 includes a plurality of aerial images 20 to each of which location information is attached. An example of the location information attached to an aerial image 20 may be GPS coordinates of the place captured on a specific point (e.g., center) of the corresponding aerial image 20.

[0033] The image matching apparatus 2000 receives a query that includes a ground image 10 from a client (e.g., user terminal). Then, the image matching apparatus 2000 searches the location database 400 for the aerial image 20 that matches the ground image 10 in the received query. If the image matching apparatus 2000 finds the aerial image 20 that matches the ground image 10, the image matching apparatus 2000 sends a response to the client. The response may include the aerial image 20 that is determined to match the ground image 10 and the location information corresponding to that aerial image 20. Since this location information indicates the location of the place captured in the aerial image 20 that matches the ground image 10, this location information can be used as data that indicates the location of the place at which the ground image 10 is captured.

[0034] It is noted that, as mentioned above, the aerial images 20 may be extracted from the source aerial image 30. In this case, the location database 400 may store one or more source aerial images 30 and their corresponding location information, instead of the aerial images 20 and their corresponding location information. The location information of the source aerial image 30 may indicate the GPS coordinates of two or more specific points (e.g., top-left corner and bottom-right corner) of the source aerial image 30. The image matching apparatus 2000 can compute the location information of each aerial image 20 using the location information of the source aerial image 30.

[0035] The geo-localization system 500 may allow the user to specify the search area. In this case, the query further includes information that specifies the search area. The image matching apparatus 2000 uses the aerial image 20 in which the places in the search area are captured. In the case where the aerial images 20 are extracted from the source aerial image 30, the image matching apparatus 2000 uses one or more source aerial images 30 in which the places in the search area are captured.

[0036] <Flow of Process> 　　Fig. 6 is a flowchart illustrating an example flow of processes performed by the image matching apparatus 2000. The acquiring unit 2020 acquires the ground image 10 (S102). The selecting unit 2040 selects one of the search methods based on at least one of the selection criteria (S104). The searching unit 2060 searches for the aerial image 20 that matches the ground image 10 using the selected search method (S106).

[0037] <Acquisition of Ground Image 10: S102> 　　The acquiring unit 2020 acquires the ground image 10 (S102). There are various ways to acquire the ground image 10. In some cases, the acquiring unit 2020 may receive the ground image 10 sent from another computer. In the example shown by Fig. 5, a query including the ground image 10 is sent from the user terminal to the image matching apparatus. The acquiring unit 2020 acquires the ground image 10 included in the query.

[0038] In other cases, the acquiring unit 2020 may retrieve, as the ground image 10, one of ground images from a storage unit to which the acquiring unit 2020 has access. For example, the user of the image matching apparatus 2000 specifies one of the ground images stored in the storage unit. The acquiring unit 2020 acquires the specified ground image as the ground image 10.

[0039] <Selection of Search method: S104> 　　The selecting unit 2040 selects one of the search methods based on one or more selection criteria (S104). Conceptually, in the case where the selection criteria indicate that a more detailed search is required, the search method providing greater detail is selected.

[0040] There may be various selection criteria that can be employed. An example of the selection criteria is the size of the field of view of the ground image 10. The size of the field of view of the ground image 10 represents the horizontal range of the scene captured in the ground image 10.

[0041] The smaller the size of the field of view of the ground image 10 is, the smaller the corresponding aerial scene that matches the ground scene in the ground image 10 is, necessitating a finer search for the aerial scene that matches the ground image 10. Thus, the selecting unit 2040 selects a search method with a smaller identifier i in the case where the size of the field of view of the ground image 10 is smaller.

[0042] For example, the range of the size F of the field of view (e.g., 0<=F<360) is divided into sections equal to the number of search methods. Suppose that there are three search methods and the range of F is 0<=F<360. In this case, the first search method is selected if the size F of the field of view of the ground image 10 satisfies 0<=F<120. The second search method is selected if F satisfies 120<=F<240. The third search method is selected if F satisfies 240<=F<360.

[0043] Another example of the selection criteria is a degree of similarity of aerial scenes in the search area. The degree of similarity of aerial scenes in the search area is considered higher if the landscape is consistent across all locations within the search area. For example, if the entire search area consists of a large residential neighborhood, the aerial scenes would be similar to each other, as each scene contains houses. In contrast, if the search area includes a variety of features such as houses, forests, and ponds, the aerial scenes would differ significantly across sections of the search area, leading to a lower degree of similarity of the aerial scenes.

[0044] Conceptually, the finer search is required if the degree of similarity of the aerial scenes in the search area is higher. Thus, the selecting unit 2040 selects a search method with a smaller identifier i if the similarity of the aerial scenes in the search area is higher.

[0045] Specifically, the range of the degree of similarity of the aerial scenes in the search area is divided into sections equal to the number of search methods. Suppose that there are four search methods, and the degree of similarity of the aerial scenes is represented by a score S, with a range from 0 to 100. In this case, the range of S is divided into four sections: 0<=S<25, 25<=S<50, 50<=S<75, and 75<=S<=100.

[0046] If the similarity S of the aerial scenes satisfies 0<=S<25, the selecting unit 2040 selects the first search method. If S satisfies 25<=S<50, the selecting unit 2040 selects the second search method. If S satisfies 50<=S<75, the selecting unit 2040 selects the third search method. If S satisfies 75<=S<=100, the selecting unit 2040 selects the fourth search method.

[0047] There may be various ways to compute the degree of similarity of the aerial scenes in the search area. For example, the selecting unit 2040 computes a degree of complexity of the aerial scenes in the search area as an index representing how low the degree of similarity of the aerial scenes in the search area is. The higher the degree of complexity of the aerial scenes in the search area is, the lower the degree of similarity of the aerial scenes in the search area is.

[0048] The selecting unit 2040 may compute the degree of complexity of the source aerial image 30 as the search area as the degree of complexity of the aerial scenes in the search area. In another example, the selecting unit 2040 computes the degree of complexity of a layout image of the search area, instead of the source aerial image, as the degree of complexity of the aerial scenes in the search area. An example of the layout image of the search area is a map image of the search area.

[0049] The degree of similarity of the aerial scenes in the search area can be computed as a value that increases as the degree of complexity of the aerial scenes in the search area decreases. For example, the selecting unit computes the reciprocal of the degree of complexity as the degree of similarity. It is noted that the degree of complexity of an image can be represented by various metrics, such as entropy.

[0050] In another example, the degree of similarity of the aerial scenes in the search area can be represented by the difference in the degrees of similarity of the aerial images 20 to the ground image 10. In this case, the image matching apparatus 2000 is configured to first use a default search method to search for the aerial image 20 that matches the ground image 10. The default search method may be a search method with the lowest detail in the search (i.e., a search method with the smallest identifier i). For example, if there are four search methods, the default search method is the fourth search method.

[0051] In this search using the default search method, the image matching apparatus 2000 computes the degree of similarity between the ground image 10 and the aerial image 20, referred to as a similarity score, for two or more aerial images 20. As mentioned in detail later, the similarity score can be computed based on the degree of similarity between the feature value of the ground image 10 and the feature value of aerial image 20.

[0052] Then, the image matching apparatus 2000 determines whether the number of the aerial images 20 whose similarity scores are close to each other is larger than a predefined threshold. When it is determined that the number of the aerial images 20 whose similarity scores are close to each other is larger than the predefined threshold, the selecting unit 2040 selects the search method whose level of detail in the search is one level higher than the default search method (i.e., the search method whose identifier i is one less than the identifier of the current search method). The image matching apparatus 2000 may repeatedly change the search method in the above-mentioned way until the number of the aerial images 20 whose similarity scores are close to each other becomes less than the predefined threshold.

[0053] <Search Using Selected Search Method: S106> 　　The searching unit 2060 searches for the aerial image 20 that matches the ground image 10 using the selected search method. As mentioned above, the search methods differ in the level of detail in the search. Hereinafter, examples of the search methods will be provided, along with an explanation of how the search is performed using these methods.

[0054] <<Example 1: Search Methods Different in Ways of Acquiring Aerial Images>> 　　The search methods may differ in the ways of acquiring the aerial images 20. As mentioned above, the aerial images 20 may be extracted using a sliding window 40 from one or more source aerial images 30. It is noted that the case where two or more source aerial images 30 are used occurs when the search area is not covered by a single source aerial image 30 but is instead covered by two or more source aerial images 30.

[0055] In such cases, more specifically, the search methods may differ in the strides used for the sliding window 40, as illustrated above with Fig. 2. In another example, the search methods may differ in the window sizes of the sliding window 40. In this case, the smaller the identifier i of the search method is, the smaller the window size used by the search method is.

[0056] Fig. 7 illustrates the case where the search methods differ in the window sizes of the sliding window 40. In Fig. 7, the first search method uses a sliding window 40 that is a square with window size W1 while the second search method uses a sliding window 40 that is a square with window size W2, where W1 is smaller than W2. Since W1 is smaller than W2, the first search method extracts more aerial images 20 than the second search method. By doing so, the first search method searches for the aerial scene that matches the ground scene in the ground image 10 with finer granularity than the second search method does.

[0057] In the example shown in Fig. 7, the stride is identical for the first and second search method. However, the search methods may differ in both the window sizes of the sliding window 40 and the strides for the sliding window 40. Suppose that there are three search methods. In this case, the pairs of stride and window size may be (S1, W1) for the first search method, (S2, W1) for the second search method, and (S2, W2) for the third search method.

[0058] In the case where the search methods differ in how the aerial images 20 are acquired, the search may follow the flow outlined in the flowchart in Fig. 8. Fig. 8 is a flowchart illustrating an example flow of the search performed by the searching unit 2060 in the case where the search methods differ in the ways of acquiring the aerial images 20.

[0059] Steps S202 to S210 constitutes a loop process L1, which is repeatedly performed until the entire search area has been searched. In Step S202, the searching unit 2060 determines whether the entire search area has been searched. In the case where the entire search area has been already searched, the searching unit 2060 terminates the search. This case occurs when no areal scene in the search area matches the ground scene in the ground image 10.

[0060] In the case where it is determined that the entire search area has not been searched yet, the searching unit 2060 performs Step 204. In Step 204, the searching unit 2060 determines the aerial image 20 to be acquired based on the selected search method. For example, the searching unit 2060 moves the sliding window by the stride that is defied by the selected search method, thereby determining the aerial image 20 to be extracted from the source aerial image 30. Then, the searching unit 2060 acquires the aerial image 20 that has been determined to be acquired (S206).

[0061] The searching unit 2060 determines whether the aerial image 20 matches the ground image 10 (S208). If it is determined that the aerial image 20 matches the ground image 10 (S208: YES), the searching unit 2060 terminates the loop process L1 to terminate the search. If it is determined that the aerial image 20 does not match the ground image 10 (S208: NO), the searching unit 2060 proceeds to the next iteration of the loop process L1 (S210, S202).

[0062] Whether the aerial image 20 matches the ground image 10 can be determined using the feature value of the aerial image 20 and the feature value of the ground image 10. Specifically, the searching unit 2060 computes the feature value of the ground image 10 and the feature value of the aerial image 20. Then, the searching unit 2060 computes the degree of similarity between the feature value of the aerial image 20 and the feature value of the ground image 10 (i.e., the similarity score). The searching unit 2060 determines that the aerial image 20 matches the ground image 10 if the similarity score is larger than or equal to a predefined threshold. On the other hand, the searching unit 2060 determines that the aerial image 20 does match the ground image 10 if the similarity score is less than the predefined threshold.

[0063] The feature values may be extracted using a pre-trained machine learning-based model, such as a neural network. Hereinafter, a model that extracts a feature value from the ground image 10 is referred to as the ground feature extracting model, while a model that extracts a feature value from the aerial image 20 is referred to the aerial feature extracting model.

[0064] The searching unit 2060 inputs the ground image 10 into the ground feature extracting model, thereby obtaining the feature value of the ground image 10 that is output by the ground feature extracting model. Similarly, the searching unit 2060 inputs the aerial image 20 into the aerial feature extracting model, thereby obtaining the feature value of the aerial image 20 that is output by the aerial feature extracting model.

[0065] <<Example 2: Search Methods Different in Rotation Angle Intervals of Aerial Image>> 　　The searching unit 2060 may compare each aerial image 20 with the ground image 10 multiple times by rotating each aerial image 20 at different angles. In this case, the search methods may differ in the rotation angle intervals of the aerial image 20. The smaller the identifier i of the search method is, the smaller the rotation angle interval used by the search method is. This means that the search method with a smaller identifier i compares each aerial images 20 with the ground image 10 in greater detail.

[0066] Fig. 9 illustrates a case where the search methods differ in the rotation angle intervals of the aerial images 20. It is noted that the aerial image 20 is extracted in a circle shape to ensure the aerial image 20 remains the same shape regardless of its rotation angle.

[0067] In Fig. 9, the rotation angle interval used by the first search method is 45 degrees. As a result, each aerial image 20 is compared with the ground image 10 eight times, at the following rotation angles: 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, and 315 degrees. On the other hand, the rotation angle interval used by the second search method is 90 degrees. As a result, each aerial image 20 is compared with the ground image 10 four times, at the following rotation angles: 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0068] In the case where the search methods differ in the rotation angle intervals, the search may follow the flow outlined in the flowchart in Fig. 10. Fig. 10 is a flowchart illustrating an example flow of the search performed by the searching unit 2060 in the case where the search methods differ in the rotation angle intervals of the aerial image 20.

[0069] Steps S302 to S314 constitutes a loop process L2, which is repeatedly performed until the entire search area has been searched. In Step S302, the searching unit 2060 determines whether the entire search area has been searched. In the case where the entire search area has been already searched, the searching unit 2060 terminates the search.

[0070] In the case where it is determined that the entire search area has not been searched yet, the searching unit 2060 performs Step 304. In Step 304, the searching unit 2060 acquires the aerial image 20 that has not been acquired yet.

[0071] It is noted that it is not necessary to extract the aerial image 20 from the source aerial image 30 in Example 2 of the search methods. For example, multiple aerial images 20 of the search area are stored in advance in a storage unit to which the image matching apparatus 2000 has access. In Step S304, the searching unit 2060 may acquire one of those multiple aerial images 20 that has not been acquired yet.

[0072] Steps S306 to S312 constitutes a loop process L3, which is repeatedly performed until the total rotation angle of the current aerial image 20 (i.e., the aerial image 20 acquired in the latest Step S304) exceeds a predefined threshold. The predefined threshold may be 360 degrees.

[0073] In Step S306, the searching unit 2060 determines whether the total rotation angle of the current aerial image 20 exceeds the predefined threshold. If the total rotation angle of the current aerial image 20 exceeds the predefined threshold, the searching unit 2060 stops comparing the current aerial image 20 with the ground image 10, and performing the next iteration of the loop process L2. This case occurs in the case where the current aerial image 20 does not match the ground image 10.

[0074] In the case where it is determined that the total rotation angle of the current aerial image 20 does exceed the predefined threshold in Step S306, the searching unit 2060 rotates the current aerial image 20 by the rotation angle that is determined based on the selected search method (S308). Let the rotation angle interval be denoted as A, and the current iteration number of the loop process L3 be denoted as Ni. The rotation angle of the aerial image 20 is then given by (Ni-1)*A. It is noted that the aerial image 20 is not rotated in the first iteration of the loop process L3.

[0075] The searching unit 2060 determines whether the rotated aerial image 20 matches the ground image 10 (S310). If it is determined that the rotated aerial image 20 matches the ground image 10 (S310: YES), the searching unit 2060 terminates the loop process L2 and L3 to terminate the search. If it is determined that the rotated aerial image 20 does not match the ground image 10 (S310: NO), the searching unit 2060 proceeds to the next iteration of the loop process L3 (S312, S306).

[0076] <<Example 3: Search Methods Different in Size of Feature Value>> 　　As mentioned above, the searching unit 2060 computes the feature value of the ground image 10 and the feature value of the aerial image 20 to determine whether they match each other. The search methods may differ in the size (i.e., the number of the dimensions) of the feature values computed for the ground image 10 and the aerial image 20. In this case, the smaller the identifier i of the search method is, the larger the size of the feature values computed by the search method is to compare the ground image 10 and the aerial image 20 more accurately. On the other hand, the larger the identifier i of the search method is, the smaller the size of the feature values computed by the search method is, thereby reducing computer resource consumption and search computation time.

[0077] Fig. 11 illustrates a case where the search methods differ in the size of the feature values. In Fig. 11, the first search method uses a ground feature extracting model 100 and an aerial feature extracting model 110 while the second search method uses a ground feature extracting model 120 and an aerial feature extracting model 130.

[0078] The feature value of the ground image 10 extracted by the ground feature extracting model 100 and the feature value of the aerial image 20 extracted by the aerial feature extracting model 110 have the same size D1 as each other. Similarly, the feature value of the ground image 10 extracted by the ground feature extracting model 120 and the feature value of the aerial image 20 extracted by the aerial feature extracting model 130 have the same size D2 as each other where D1 is larger than D2. Since D1 is larger than D2, the first search method can compute the similarity between the ground image 10 and the aerial image 20 more accurately than the second search method does.

[0079] In the case where the search methods differ in the size of the feature values, the search may follow the flow outlined in the flowchart in Fig. 12. Fig. 12 is a flowchart illustrating an example flow of the search performed by the searching unit 2060 in the case where the search methods differ in the size of the feature values.

[0080] The searching unit 2060 computes the feature value of the ground image 10 whose size is defined by the selected search method (S402). It is noted that, although the corresponding steps are not depicted, the feature value of the ground image 10 may be computed before the loop process L1 in the flowchart shown by Fig. 7 and before the loop process L2 in the flowchart shown by Fig. 9.

[0081] Steps S404 to S412 constitutes a loop process L4, which is repeatedly performed until the entire search area has been searched. In Step S404, the searching unit 2060 determines whether the entire search area has been searched. In the case where the entire search area has been already searched, the searching unit 2060 terminates the search.

[0082] In the case where it is determined that the entire search area has not been searched yet, the searching unit 2060 performs Step 406. In Step 406, the searching unit 2060 acquires the aerial image 20 that has not been acquired yet (S406). It is noted that it is not necessary to extract the aerial image 20 from the source aerial image 30 in Example 3 of the search methods.

[0083] The searching unit 2060 computes the feature value of the current aerial image 20 whose size is defined by the selected search method (S408). Then, the searching unit 2060 determines whether the aerial image 20 matches the ground image 10 using the feature values computed in S402 and S408 (S410).

[0084] If it is determined that the current aerial image 20 matches the ground image 10 (S410: YES), the searching unit 2060 terminates the loop process L4 to terminate the search. If it is determined that the current aerial image 20 does not match the ground image 10 (S410: NO), the searching unit 2060 proceeds to the next iteration of the loop process L4 (S412, S404).

[0085] <<Combination of Above-Mentioned Examples>> 　　The above-mentioned examples of search methods can be combined in various ways, allowing the search methods to differ in two or more of the factors described above. For example, the search methods may differ in the strides of the sliding window 40 and the rotation angle intervals of the aerial image 20 in the case where Examples 1 and 2 are combined. Suppose that there are three search methods. In this case, the pairs of stride and rotation angle interval may be (S1, A1) for the first search method, (S2, A1) for the second search method, and (S2, A2) for the third search method, where S1 is smaller than S2 and A1 is smaller than A2.

[0086] In the case where Examples 1 and 2 are combined, the search may follow the flow illustrated by Fig. 10 while the aerial image 20 to be acquired is determined based on the selected search method in Step S304. In the case where Examples 1 and 3 are combined, the search may follow the flow illustrated by Fig. 8 while the size of the feature values are defined by the selected search method. In the case where Examples 2 and 3 are combined, the search may follow the flow illustrated by Fig. 10 while the size of the feature values are defined by the selected search method. In the case where Examples 1, 2, and 3 are combined, the search may follow the flow illustrated by Fig. 10 while the aerial image 20 to be acquired is determined based on the selected search method in Step S304 and the size of the feature values are defined by the selected search method.

[0087] <Selection of Feature Data Type> 　　The searching unit 2060 may further select one of binary type and numeric type as the data type of the feature values based on the selected search method. In the case where binary type is selected as the data type of the feature values, each element of the feature values indicates a binary value (i.e., 0 or 1).

[0088] Numeric type may include integer types (e.g., an 8-bit integer and a 32-bit integer) and floating-point types (e.g., a 32-bit floating-point and a 64-bit floating-point). In the case where the numeric type employed in the image matching apparatus 2000 is 64-bit floating point, each element of the feature values indicates a 64-bit floating point value.

[0089] By using a numeric type for feature values, the searching unit 2060 can compute the degree of similarity between the feature value of the ground image 10 and the feature value of the aerial image 20 with greater accuracy compared to using a binary type. On the other hand, using a binary type for feature values enables the searching unit 2060 to consume fewer computational resources and less time in computing the degree of similarity than when using a numeric type.

[0090] Conceptually, the searching unit 2060 selects a binary type in the case where the selected search method provides greater detail in the search through the factors such as a smaller stride. This increased detail compensates for the lower level of representation in the feature values with the binary type.

[0091] In some cases, a predefined boundary of the search method identifier divides the search methods into the search methods that uses a binary type and the search methods that uses a numeric type. Suppose that the predefined boundary is two. In this case, search methods with an identifier i less than or equal to two use a binary type as the data type of the feature values, while search methods with an identifier i greater than two use a numeric type as the data type of the feature values.

[0092] The feature value with the binary type is derived by converting each element of the feature value with the numeric type into binary values (i.e., 0 or 1) based on a predefined threshold. Elements greater than the threshold are converted to 1 while elements less than or equal to the threshold are converted to 0. Thus, the ground feature extracting model and the aerial feature extracting model are configured to compute the feature value with the numeric type regardless of the data type defined by the search method.

[0093] Suppose that the selecting unit 2040 selects a searching method in which the data type of feature values is defined as the binary type. In this case, the searching unit 2060 inputs the ground image 10 into the ground feature extracting model to obtain the feature value of the ground image 10 with the numeric type. Then, the searching unit 2060 converts each element of the feature value of the ground image 10 into a binary value based on the predefined threshold to obtain the feature value of the ground image 10 with the binary type. Similarly, the searching unit 2060 inputs the aerial image 20 into the aerial feature extracting model to obtain the feature value of the aerial image 20 with the numeric type. Then, the searching unit 2060 converts each element of the feature value of the aerial image 20 into a binary value based on the predefined threshold to obtain the feature value of the aerial image 20 with the binary type.

[0094] In the case where the numeric data type is used, the searching unit 2060 computes the Euclidian distance between the feature value of the ground image 10 and the feature value of aerial image 20 to determine the similarity score. On the other hand, in the case where the binary data type is used, the searching unit 2060 computes the Hamming distance between the feature value of the ground image 10 and the feature value of aerial image 20 to determine the similarity score. Since the computational complexity of the Hamming distance is less than the computational complexity of the Euclidian distance, using a binary data type allows the image matching apparatus 2000 to reduce computation time and resource usage, as described above.

[0095] It is noted that the similarity score can be computed based on the distance between the feature values, such as the reciprocal of the distance.

[0096] <Output from Image Matching Apparatus> 　　The image matching apparatus 2000 may output information, referred to as result information, related to the outcome of the search. In the case where the searching unit 2060 finds an aerial image 20 that matches the ground image 10, the result information may include information about the aerial image 20, such as the aerial image 20 itself and the location information corresponding to the aerial image 20. If no aerial images 20 match the ground image 10, the result information may indicate that there is no aerial scene in the search area that matches the ground scene in the ground image 10. It is noted that the result information may be equivalent to the response from the geo-localization system 500 shown in Fig. 5.

[0097] There are various ways to output the result information. For example, the image matching apparatus 2000 may store the result information into a storage device. In another example, the image matching apparatus 2000 may output the result information to a display device, allowing the display device to show the contents of the result information. In another example, the image matching apparatus 2000 may send the result information to another computer, such as the user terminal.

[0098] While the present disclosure has been particularly shown and described with reference to example embodiments thereof, the present disclosure is not limited to these example embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the claims. And each embodiment can be appropriately combined with at least one of embodiments.

[0099] Each of the drawings or figures is merely an example to illustrate one or more example embodiments. Each figure may not be associated with only one particular example embodiment, but may be associated with one or more other example embodiments. As those of ordinary skill in the art will understand, various features or steps described with reference to any one of the figures can be combined with features or steps illustrated in one or more other figures, for example, to produce example embodiments that are not explicitly illustrated or described. Not all of the features or steps illustrated in any one of the figures to describe an example embodiment are necessarily essential, and some features or steps may be omitted. The order of the steps described in any of the figures may be changed as appropriate.

[0100] The program includes instructions (or software codes) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored in a non-transitory computer readable medium or a tangible storage medium. By way of example, and not a limitation, non-transitory computer readable media or tangible storage media can include a random-access memory (RAM), a read-only memory (ROM), a flash memory, a solid-state drive (SSD) or other types of memory technologies, a CD-ROM, a digital versatile disc (DVD), a Blu-ray disc or other types of optical disc storage, and magnetic cassettes, magnetic tape, magnetic disk storage or other types of magnetic storage devices. The program may be transmitted on a transitory computer readable medium or a communication medium. By way of example, and not a limitation, transitory computer readable media or communication media can include electrical, optical, acoustical, or other forms of propagated signals.

[0101] The whole or part of the example embodiments disclosed above can be described as, but not limited to, the following supplementary notes. <Supplementary notes> (Supplementary Note 1) 　　An image matching apparatus comprising: 　　at least one memory that is configured to store instructions; and 　　at least one processor that is configured to execute the instructions to: 　　acquire a ground image; 　　select one of two or more search methods based on at least one of selection criteria that includes a size of a field of view of the ground image, a degree of similarity of aerial scenes in a search area, or both; and 　　search, using the selected search method, for an aerial image that matches the ground image from among two or more of the aerial images in each of which an aerial scene of a part of the search area is captured. (Supplementary Note 2) 　　The image matching apparatus according to supplementary note 1, 　　wherein each aerial image is extracted using a sliding window from a source aerial image that is larger than the aerial image and in which aerial scenes of a part or a whole of the search area is captured; 　　wherein the two or more search methods differ in strides of the sliding window or sizes of the sliding window. (Supplementary Note 3) 　　The image matching apparatus according to supplementary note 1, 　　wherein the search for the aerial image includes comparing, for each aerial image, the aerial image with the ground image two or more times by rotating the aerial image at different angles; 　　wherein the two or more search methods differ in rotation angle intervals of the aerial image. (Supplementary Note 4) 　　The image matching apparatus according to supplementary note 1, 　　wherein the search for the aerial image includes computing a feature value of the ground image and a feature value of the aerial image; and 　　wherein the two or more search methods differ in sizes of the feature value of the ground image and sizes of the feature value of the aerial image. (Supplementary Note 5) 　　The image matching apparatus according to supplementary note 1, 　　wherein the two or more search methods include a first search method and a second search method, the first search method providing a greater detail in the search than the second methods, 　　wherein the search for the aerial image includes: 　　　　computing a feature value of the ground image and a feature value of the aerial image; and 　　　　computing a degree of similarity between the feature value of the ground image and the feature value of the aerial image, 　　wherein the search using the first search method includes: 　　　　computing the feature value of the ground image each element of which is a binary type and the feature value of the aerial image each element of which is a binary type; and 　　　　computing a Hamming distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image, and 　　wherein the search using the second search method includes: 　　　　computing the feature value of the ground image each element of which is a numeric type and the feature value of the aerial image each element of which is a numeric type; and 　　　　computing a Euclidian distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image. (Supplementary Note 6) 　　The image matching apparatus according to supplementary note 1, 　　wherein a possible range of the size of the field of view of the ground image is divided into two or more sections, each section being associated with a respective search method, the section including smaller values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　determining the section that includes the size of the field of view of the acquired ground image; and 　　　　selecting the search method associated with the determined section. (Supplementary Note 7) 　　The image matching apparatus according to supplementary note 1, 　　wherein a possible rage of the degree of similarity of aerial scenes in the search area is divided into two or more sections, each section being associated with a respective search method, the section including larger values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　computing the degree of similarity of aerial scenes in the search area; 　　　　determining the section that includes the computed degree of similarity of aerial scenes in the search area; and 　　　　selecting the search method associated with the determined section. (Supplementary Note 8) 　　An image matching method comprising: 　　acquiring a ground image; 　　selecting one of two or more search methods based on at least one of selection criteria that includes a size of a field of view of the ground image, a degree of similarity of aerial scenes in a search area, or both; and 　　searching, using the selected search method, for an aerial image that matches the ground image from among two or more of the aerial images in each of which an aerial scene of a part of the search area is captured. (Supplementary Note 9) 　　The image matching method according to supplementary note 8, 　　wherein each aerial image is extracted using a sliding window from a source aerial image that is larger than the aerial image and in which aerial scenes of a part or a whole of the search area is captured; 　　wherein the two or more search methods differ in strides of the sliding window or sizes of the sliding window. (Supplementary Note 10) 　　The image matching method according to supplementary note 8, 　　wherein the search for the aerial image includes comparing, for each aerial image, the aerial image with the ground image two or more times by rotating the aerial image at different angles; 　　wherein the two or more search methods differ in rotation angle intervals of the aerial image. (Supplementary Note 11) 　　The image matching method according to supplementary note 8, 　　wherein the search for the aerial image includes computing a feature value of the ground image and a feature value of the aerial image; and 　　wherein the two or more search methods differ in sizes of the feature value of the ground image and sizes of the feature value of the aerial image. (Supplementary Note 12) 　　The image matching method according to supplementary note 8, 　　wherein the two or more search methods include a first search method and a second search method, the first search method providing a greater detail in the search than the second methods, 　　wherein the search for the aerial image includes: 　　　　computing a feature value of the ground image and a feature value of the aerial image; and 　　　　computing a degree of similarity between the feature value of the ground image and the feature value of the aerial image, 　　wherein the search using the first search method includes: 　　　　computing the feature value of the ground image each element of which is a binary type and the feature value of the aerial image each element of which is a binary type; and 　　　　computing a Hamming distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image, and 　　wherein the search using the second search method includes: 　　　　computing the feature value of the ground image each element of which is a numeric type and the feature value of the aerial image each element of which is a numeric type; and 　　　　computing a Euclidian distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image. (Supplementary Note 13) 　　The image matching method according to supplementary note 8, 　　wherein a possible range of the size of the field of view of the ground image is divided into two or more sections, each section being associated with a respective search method, the section including smaller values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　determining the section that includes the size of the field of view of the acquired ground image; and 　　　　selecting the search method associated with the determined section. (Supplementary Note 14) 　　The image matching method according to supplementary note 8, 　　wherein a possible rage of the degree of similarity of aerial scenes in the search area is divided into two or more sections, each section being associated with a respective search method, the section including larger values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　computing the degree of similarity of aerial scenes in the search area; 　　　　determining the section that includes the computed degree of similarity of aerial scenes in the search area; and 　　　　selecting the search method associated with the determined section. (Supplementary Note 15) 　　A non-transitory computer-readable medium storing a program that causes a computer to execute: 　　acquiring a ground image; 　　selecting one of two or more search methods based on at least one of selection criteria that includes a size of a field of view of the ground image, a degree of similarity of aerial scenes in a search area, or both; and 　　searching, using the selected search method, for an aerial image that matches the ground image from among two or more of the aerial images in each of which an aerial scene of a part of the search area is captured. (Supplementary Note 16) 　　The non-transitory computer-readable medium according to supplementary note 15, 　　wherein each aerial image is extracted using a sliding window from a source aerial image that is larger than the aerial image and in which aerial scenes of a part or a whole of the search area is captured; 　　wherein the two or more search methods differ in strides of the sliding window or sizes of the sliding window. (Supplementary Note 17) 　　The non-transitory computer-readable medium according to supplementary note 15, 　　wherein the search for the aerial image includes comparing, for each aerial image, the aerial image with the ground image two or more times by rotating the aerial image at different angles; 　　wherein the two or more search methods differ in rotation angle intervals of the aerial image. (Supplementary Note 18) 　　The non-transitory computer-readable medium according to supplementary note 15, 　　wherein the search for the aerial image includes computing a feature value of the ground image and a feature value of the aerial image; and 　　wherein the two or more search methods differ in sizes of the feature value of the ground image and sizes of the feature value of the aerial image. (Supplementary Note 19) 　　The non-transitory computer-readable medium according to supplementary note 15, 　　wherein the two or more search methods include a first search method and a second search method, the first search method providing a greater detail in the search than the second methods, 　　wherein the search for the aerial image includes: 　　　　computing a feature value of the ground image and a feature value of the aerial image; and 　　　　computing a degree of similarity between the feature value of the ground image and the feature value of the aerial image, 　　wherein the search using the first search method includes: 　　　　computing the feature value of the ground image each element of which is a binary type and the feature value of the aerial image each element of which is a binary type; and 　　　　computing a Hamming distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image, and 　　wherein the search using the second search method includes: 　　　　computing the feature value of the ground image each element of which is a numeric type and the feature value of the aerial image each element of which is a numeric type; and 　　　　computing a Euclidian distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image. (Supplementary Note 20) 　　The non-transitory computer-readable medium according to supplementary note 15, 　　wherein a possible range of the size of the field of view of the ground image is divided into two or more sections, each section being associated with a respective search method, the section including smaller values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　determining the section that includes the size of the field of view of the acquired ground image; and 　　　　selecting the search method associated with the determined section. (Supplementary Note 21) 　　The image matching method according to supplementary note 8, 　　wherein a possible rage of the degree of similarity of aerial scenes in the search area is divided into two or more sections, each section being associated with a respective search method, the section including larger values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　computing the degree of similarity of aerial scenes in the search area; 　　　　determining the section that includes the computed degree of similarity of aerial scenes in the search area; and 　　　　selecting the search method associated with the determined section.

[0102] 10 ground image 20 aerial image 30 source aerial image 40 sliding window 100, 120 ground feature extracting model 110, 130 aerial feature extracting model 400 location database 500 geo-localization system 1000 computer 1020 bus 1040 processor 1060 memory 1080 storage device 1100 input / output interface 1120 network interface 2000 image matching apparatus 2020 acquiring unit 2040 selecting unit 2060 searching unit

Claims

1. An image matching apparatus comprising: 　　at least one memory that is configured to store instructions; and 　　at least one processor that is configured to execute the instructions to: 　　acquire a ground image; 　　select one of two or more search methods based on at least one of selection criteria that includes a size of a field of view of the ground image, a degree of similarity of aerial scenes in a search area, or both; and 　　search, using the selected search method, for an aerial image that matches the ground image from among two or more of the aerial images in each of which an aerial scene of a part of the search area is captured.

2. The image matching apparatus according to claim 1, 　　wherein each aerial image is extracted using a sliding window from a source aerial image that is larger than the aerial image and in which aerial scenes of a part or a whole of the search area is captured; 　　wherein the two or more search methods differ in strides of the sliding window or sizes of the sliding window.

3. The image matching apparatus according to claim 1, 　　wherein the search for the aerial image includes comparing, for each aerial image, the aerial image with the ground image two or more times by rotating the aerial image at different angles; 　　wherein the two or more search methods differ in rotation angle intervals of the aerial image.

4. The image matching apparatus according to claim 1, 　　wherein the search for the aerial image includes computing a feature value of the ground image and a feature value of the aerial image; and 　　wherein the two or more search methods differ in sizes of the feature value of the ground image and sizes of the feature value of the aerial image.

5. The image matching apparatus according to claim 1, 　　wherein the two or more search methods include a first search method and a second search method, the first search method providing a greater detail in the search than the second methods, 　　wherein the search for the aerial image includes: 　　　　computing a feature value of the ground image and a feature value of the aerial image; and 　　　　computing a degree of similarity between the feature value of the ground image and the feature value of the aerial image, 　　wherein the search using the first search method includes: 　　　　computing the feature value of the ground image each element of which is a binary type and the feature value of the aerial image each element of which is a binary type; and 　　　　computing a Hamming distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image, and 　　wherein the search using the second search method includes: 　　　　computing the feature value of the ground image each element of which is a numeric type and the feature value of the aerial image each element of which is a numeric type; and 　　　　computing a Euclidian distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image.

6. The image matching apparatus according to claim 1, 　　wherein a possible range of the size of the field of view of the ground image is divided into two or more sections, each section being associated with a respective search method, the section including smaller values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　determining the section that includes the size of the field of view of the acquired ground image; and 　　　　selecting the search method associated with the determined section.

7. The image matching apparatus according to claim 1, 　　wherein a possible rage of the degree of similarity of aerial scenes in the search area is divided into two or more sections, each section being associated with a respective search method, the section including larger values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　computing the degree of similarity of aerial scenes in the search area; 　　　　determining the section that includes the computed degree of similarity of aerial scenes in the search area; and 　　　　selecting the search method associated with the determined section.

8. An image matching method performed by a computer, comprising: 　　acquiring a ground image; 　　selecting one of two or more search methods based on at least one of selection criteria that includes a size of a field of view of the ground image, a degree of similarity of aerial scenes in a search area, or both; and 　　searching, using the selected search method, for an aerial image that matches the ground image from among two or more of the aerial images in each of which an aerial scene of a part of the search area is captured.

9. The image matching method according to claim 8, 　　wherein each aerial image is extracted using a sliding window from a source aerial image that is larger than the aerial image and in which aerial scenes of a part or a whole of the search area is captured; 　　wherein the two or more search methods differ in strides of the sliding window or sizes of the sliding window.

10. The image matching method according to claim 8, 　　wherein the search for the aerial image includes comparing, for each aerial image, the aerial image with the ground image two or more times by rotating the aerial image at different angles; 　　wherein the two or more search methods differ in rotation angle intervals of the aerial image.

11. The image matching method according to claim 8, 　　wherein the search for the aerial image includes computing a feature value of the ground image and a feature value of the aerial image; and 　　wherein the two or more search methods differ in sizes of the feature value of the ground image and sizes of the feature value of the aerial image.

12. The image matching method according to claim 8, 　　wherein the two or more search methods include a first search method and a second search method, the first search method providing a greater detail in the search than the second methods, 　　wherein the search for the aerial image includes: 　　　　computing a feature value of the ground image and a feature value of the aerial image; and 　　　　computing a degree of similarity between the feature value of the ground image and the feature value of the aerial image, 　　wherein the search using the first search method includes: 　　　　computing the feature value of the ground image each element of which is a binary type and the feature value of the aerial image each element of which is a binary type; and 　　　　computing a Hamming distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image, and 　　wherein the search using the second search method includes: 　　　　computing the feature value of the ground image each element of which is a numeric type and the feature value of the aerial image each element of which is a numeric type; and 　　　　computing a Euclidian distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image.

13. The image matching method according to claim 8, 　　wherein a possible range of the size of the field of view of the ground image is divided into two or more sections, each section being associated with a respective search method, the section including smaller values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　determining the section that includes the size of the field of view of the acquired ground image; and 　　　　selecting the search method associated with the determined section.

14. The image matching method according to claim 8, 　　wherein a possible rage of the degree of similarity of aerial scenes in the search area is divided into two or more sections, each section being associated with a respective search method, the section including larger values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　computing the degree of similarity of aerial scenes in the search area; 　　　　determining the section that includes the computed degree of similarity of aerial scenes in the search area; and 　　　　selecting the search method associated with the determined section.

15. A non-transitory computer-readable medium storing a program that causes a computer to execute: 　　acquiring a ground image; 　　selecting one of two or more search methods based on at least one of selection criteria that includes a size of a field of view of the ground image, a degree of similarity of aerial scenes in a search area, or both; and 　　searching, using the selected search method, for an aerial image that matches the ground image from among two or more of the aerial images in each of which an aerial scene of a part of the search area is captured.

16. The non-transitory computer-readable medium according to claim 15, 　　wherein each aerial image is extracted using a sliding window from a source aerial image that is larger than the aerial image and in which aerial scenes of a part or a whole of the search area is captured; 　　wherein the two or more search methods differ in strides of the sliding window or sizes of the sliding window.

17. The non-transitory computer-readable medium according to claim 15, 　　wherein the search for the aerial image includes comparing, for each aerial image, the aerial image with the ground image two or more times by rotating the aerial image at different angles; 　　wherein the two or more search methods differ in rotation angle intervals of the aerial image.

18. The non-transitory computer-readable medium according to claim 15, 　　wherein the search for the aerial image includes computing a feature value of the ground image and a feature value of the aerial image; and 　　wherein the two or more search methods differ in sizes of the feature value of the ground image and sizes of the feature value of the aerial image.

19. The non-transitory computer-readable medium according to claim 15, 　　wherein the two or more search methods include a first search method and a second search method, the first search method providing a greater detail in the search than the second methods, 　　wherein the search for the aerial image includes: 　　　　computing a feature value of the ground image and a feature value of the aerial image; and 　　　　computing a degree of similarity between the feature value of the ground image and the feature value of the aerial image, 　　wherein the search using the first search method includes: 　　　　computing the feature value of the ground image each element of which is a binary type and the feature value of the aerial image each element of which is a binary type; and 　　　　computing a Hamming distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image, and 　　wherein the search using the second search method includes: 　　　　computing the feature value of the ground image each element of which is a numeric type and the feature value of the aerial image each element of which is a numeric type; and 　　　　computing a Euclidian distance between the feature value of the ground image and the feature value of the aerial image to compute the degree of similarity between the feature value of the ground image and the feature value of the aerial image.

20. The non-transitory computer-readable medium according to claim 15, 　　wherein a possible range of the size of the field of view of the ground image is divided into two or more sections, each section being associated with a respective search method, the section including smaller values being associated with the search method that provides greater detail in the search, 　　wherein the selection of the search method includes: 　　　　determining the section that includes the size of the field of view of the acquired ground image; and selecting the search method associated with the determined section.

Images