An applied layered normalized image descriptor network

Abstract

Techniques for using and training a descriptor network are disclosed. An image can be received and provided to a descriptor network. The descriptor network can generate an image descriptor based on the image. The image descriptor can include a set of elements distributed between a primary vector comprising a first subset of the set of elements and a secondary vector comprising a second subset of the set of elements. The second subset of the set of elements can include more elements than the first subset of the set of elements. A hierarchical normalization can be imposed on the image descriptor by normalizing the primary vector to a primary normalization amount and normalizing the secondary vector to a secondary normalization amount. The secondary normalization amount can be less than the primary normalization amount.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 019,211, filed May 1, 2020, entitled “IMAGE DESCRIPTOR NETWORK WITH IMPOSED HIERARCHICAL NORMALIZATION,” the entire contents of which are incorporated herein by reference for all purposes. BACKGROUND

[0003] In image processing, a descriptor of an image (also referred to as an “image descriptor”) is a description of visual features of an image, including possibilities of color, shape, texture, motion, etc. of the image. Image descriptors and algorithms that produce image descriptors have a variety of applications in computer vision. For example, image descriptors provide a robust means for determining similarity between two images by generating an image descriptor for each of the images and computing a distance or difference between the two descriptors. SUMMARY

[0004] The present disclosure generally relates to techniques for generating image descriptors of images. More particularly, embodiments of the present disclosure provide techniques for training and using a descriptor network to accurately generate image descriptors having a primary and secondary vector.

[0005] The following provides summaries of various embodiments of the invention as an example list. As used in the following, any reference to a list of examples should be understood as a reference to each of the examples in that list (e.g., “Examples 1-4” should be understood as “Examples 1, 2, 3, or 4”).

[0006] Example 1 is a computer-implemented method comprising: receiving a first image; providing the first image to a descriptor network as input; generating, using the descriptor network, a first image descriptor based on the first image, the first image descriptor comprising a first set of elements distributed between: a first primary vector comprising a first subset of the first set of elements; and a first secondary vector comprising a second subset of the first set of elements, wherein the second subset of the first set of elements comprises more elements than the first subset of the first set of elements; and imposing hierarchical normalization on the first image descriptor by: normalizing the first primary vector to a primary normalization amount; and normalizing the first secondary vector to a secondary normalization amount, wherein the secondary normalization amount is less than the primary normalization amount.

[0007] Example 2 is the computer-implemented method of Example 1, further comprising: receiving a second image; providing the second image as input to the descriptor network; generating, using the descriptor network, a second image descriptor based on the second image, the second image descriptor comprising a second set of elements distributed between: a second primary vector comprising a first subset of the second set of elements; and a second secondary vector comprising a second subset of the second set of elements, wherein the second subset of the second set of elements comprises more elements than the first subset of the second set of elements; and imposing the hierarchical normalization on the second image descriptor by: normalizing the second primary vector to the primary normalization amount; and normalizing the second secondary vector to the secondary normalization amount.

[0008] Example 3 is the computer-implemented method of Example 2, further comprising: determining whether the first image matches the second image by: computing, based on the first primary vector and the second primary vector, a primary distance between the first image and the second image; and determining whether the primary distance is greater than an upper threshold.

[0009] Example 4 is the computer-implemented method of Example 3, wherein determining whether the first image matches the second image further comprises: determining that the primary distance is greater than the upper threshold; and determining that the first image does not match the second image.

[0010] Example 5 is the computer-implemented method of Example 3, wherein determining whether the first image matches the second image further comprises: determining that the primary distance is not greater than the upper threshold; and determining whether a secondary distance between the first image and the second image is greater than a lower threshold.

[0011] Example 6 is the computer-implemented method of Example 3, wherein computing, based on the first primary vector and the second primary vector, the primary distance between the first image and the second image comprises: computing a sum of squares of differences between the first subset of the first set of elements and the first subset of the second set of elements.

[0012] Example 7 is the computer-implemented method of Example 5, wherein determining whether the first image matches the second image further comprises: determining that the primary distance is greater than the lower threshold; computing, based on the first secondary vector and the second secondary vector, a secondary distance between the first image and the second image; and determining whether a sum of the primary distance and the secondary distance is greater than the upper threshold.

[0013] Example 8 is the computer-implemented method of example 7, wherein determining whether the first image matches the second image further comprises: determining that the sum of the primary distance and the secondary distance is greater than the upper threshold; and determining that the first image does not match the second image.

[0014] Example 9 is the computer-implemented method of example 7, wherein determining whether the first image matches the second image further comprises: determining that the sum of the primary distance and the secondary distance is not greater than the upper threshold; and determining that the first image matches the second image.

[0015] Example 10 is the computer-implemented method of example 7, wherein computing the secondary distance between the first image and the second image based on the first secondary vector and the second secondary vector comprises: computing a sum of squares of differences between the second subset of the first set of elements and the second subset of the second set of elements.

[0016] Example 11 is the computer-implemented method of examples 1-10, wherein: the primary normalization quantity is equal to 1 - a; and the secondary normalization quantity is equal to a, where a is less than 0.5.

[0017] Example 12 is the computer-implemented method of example 11, wherein a is equal to 1 / 8, 1 / 16, or 1 / 32.

[0018] Example 13 is the computer-implemented method of examples 1-12, wherein the descriptor network is a neural network having a set of weights that are modifiable through a training process.

[0019] Example 14 is a method of training a descriptor network, the method comprising: receiving a set of image pairs; and for each image pair in the set of image pairs: providing a first training image from the image pair to a descriptor network as input; generating, using the descriptor network, a first image descriptor based on the first training image, the first image descriptor comprising a first set of elements distributed between: a first primary vector comprising a first subset of the first set of elements; and a first secondary vector comprising a second subset of the first set of elements, wherein the second subset of the first set of elements comprises more elements than the first subset of the first set of elements; and imposing hierarchical normalization on the first image descriptor by: normalizing the first primary vector to a primary normalization amount; and normalizing the first secondary vector to a secondary normalization amount, wherein the secondary normalization amount is less than the primary normalization amount; providing a second training image from the image pair to the descriptor network as input; generating, using the descriptor network, a second image descriptor based on the second training image, the second image descriptor comprising a second set of elements distributed between: a second primary vector comprising a first subset of the second set of elements; and a second secondary vector comprising a second subset of the second set of elements, wherein the second subset of the second set of elements comprises more elements than the first subset of the second set of elements; and imposing the hierarchical normalization on the second image descriptor by: normalizing the second primary vector to the primary normalization amount; and normalizing the second secondary vector to the secondary normalization amount; computing, based on the first primary vector and the second primary vector, a primary distance between the first training image and the second training image; computing, based on the first secondary vector and the second secondary vector, a secondary distance between the first training image and the second training image; and modifying the descriptor network based on the primary distance and the secondary distance.

[0020] Example 15 is a system comprising: one or more processors; and a computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the method of any of Examples 1-14.

[0021] Example 16 is a non-transitory machine-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any of Examples 1-14. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. In the drawings:

[0023] Figure 1A An example system including a descriptor network is shown.

[0024] Figure 1B An example system including a descriptor network is shown.

[0025] Figure 2 An example system including a descriptor network is shown.

[0026] Figure 3A An example system for training a descriptor network is shown.

[0027] Figure 3B An example system for training a descriptor network is shown.

[0028] Figure 4 A method of using a descriptor network is shown.

[0029] Figure 5 A method of training a descriptor network is shown.

[0030] Figure 6 An example of hierarchical normalization is shown.

[0031] Figure 7 An example flowchart of a method for determining whether two images match is shown.

[0032] Figure 8 A graphical representation of the example flowchart shown in Figure 7 is shown.

[0033] Figure 9 An example flowchart of a method for determining whether two images match is shown.

[0034] Figure 10 A graphical representation of the example flowchart shown in Figure 9 is shown.

[0035] Figure 11 An example flowchart of a method for finding the image closest to a reference image from a set of images is shown.

[0036] Figure 12 A graphical representation of the example flowchart shown in Figure 11 is shown.

[0037] Figure 13This diagram illustrates an example flowchart of a method for finding the image that is closest to a reference image from a set of images.

[0038] Figure 14 Show Figure 13 The example flowchart shown is a graphical representation.

[0039] Figure 15 Additional graphical representations show the areas that can be filtered out using the primary distance.

[0040] Figure 16 Additional graphical representations show the areas that can be filtered out using the primary distance.

[0041] Figure 17 The experimental results for the descriptor network are shown in the figure.

[0042] Figure 18 The experimental results for the descriptor network are shown in the figure.

[0043] Figure 19 The experimental results for the descriptor network are shown in the figure.

[0044] Figure 20 The experimental results for the descriptor network are shown in the figure.

[0045] Figure 21 An example image descriptor with multiple primary vectors is shown.

[0046] Figure 22 A simplified computer system is shown. Detailed Implementation

[0047] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this specification, illustrate embodiments of the disclosure and, together with the detailed description, explain the principles of the disclosure. No further structural details of the disclosure are attempted to be shown except as necessary for a basic understanding of the disclosure and the various possible implementations.

[0048] Figure 1A An example system according to some embodiments of the present disclosure includes a descriptor network 100A for generating an image descriptor 110A based on an image 102. The descriptor network 100A may be a neural network, such as a convolutional neural network, with a specific architecture and associated weight set. When provided with an image 102 as input, the descriptor network 100A may be configured to generate an image descriptor 110A having N elements 116A. Each element in the elements 116A may indicate a visual feature in the image 102.

[0049] Figure 1BAn example system including a descriptor network 100B similar to descriptor network 100A is shown, the difference being that the generated image descriptor 110B includes a primary vector 112 and a secondary vector 114. In some embodiments, the primary vector 112 may include a first subset of elements 116B (from elements 116B-1 to elements 116B-M), and the secondary vector 114 may include a second subset of elements 116B (from elements 116B-M+1 to elements 116B-M+N). Thus, the first subset may include M elements, and the second subset may include N elements.

[0050] In some embodiments, a primary vector 112 and a secondary vector 114 are obtained by applying hierarchical normalization to the image descriptor 110B. This may include normalizing the primary vector 112 to a primary normalization amount and normalizing the secondary vector 114 to a secondary normalization amount, wherein the primary normalization amount is greater than the secondary normalization amount. By combining the application of hierarchical normalization with setting the size of the primary vector 112 (M) to be smaller than the size of the secondary vector 114 (N), the image descriptor 110B can be used to significantly improve performance in a variety of tasks, as described herein.

[0051] Figure 2 Example systems including a descriptor network 200 according to some embodiments of the present disclosure are illustrated. In the illustrated example, the descriptor network 200 is used to determine the similarity between images 202 and 203 by determining the distance between image descriptors 210 and 211 generated based on images 202 and 203, respectively. The descriptor network 200 may be similar to descriptor network 100B, such that image descriptors 210 and 211 each include a primary vector and a secondary vector. In various embodiments, images 202 and 203 may be provided to the descriptor network 200 as input sequentially (e.g., image 202 may be provided at a first time, while image 203 may be provided at a second time after the first time), or they may be provided simultaneously to two different instances of the descriptor network 200 (e.g., instructions for implementing the weights and model architecture associated with the descriptor network 200 may be loaded onto two separate processors or processor cores).

[0052] The total distance D between image descriptors 210 and 211 total The principal distance D between image descriptors can be calculated. major (Total distance D calculated based solely on the principal vectors of image descriptors 210 and 211) total Secondary distance D between the part and the image descriptor minor (The total distance D is calculated based solely on the secondary vectors of image descriptors 210 and 211) totalThe sum of the parts of the image descriptor 210. In the example shown, the Euclidean distance is used for the principal vectors (x1, x2, ..., x) of the image descriptor 210. M ) and the principal vector (x') of image descriptor 211 M ,x'2,…,x' M The primary distance is calculated using elements of the image descriptor 210, and the secondary distance is calculated using the secondary vector (x) based on the image descriptor 210. M+1 ,x M+2 ,…,x M+N ) and the secondary vector (x') of image descriptor 211 M+1 ,x' M+2 ,…,x' M+N The elements of the distance are used to calculate the secondary distance. Other distance metrics (such as cosine distance) can be used to calculate the primary and secondary distances.

[0053] Figure 3A An example system for training a descriptor network 300A according to some embodiments of the present disclosure is shown. The descriptor network 300A may be similar to the descriptor network 100A. During a single training iteration of multiple training iterations, the descriptor network 300A may be provided with training images 302 and 303. In some embodiments, it may be known that training images 302 and 303 are similar or dissimilar images. For example, training image 303 may be a slightly modified version of training image 302. When training images 302 and 303 are provided as input (e.g., sequentially), the descriptor network 300A may be configured to generate image descriptors 310A and 311A, respectively. The total distance D can then be calculated based on the image descriptors 310A and 311A. total .

[0054] By, for example, modifying the weights of the descriptor network 300A, the total distance D total It can be used to train a descriptor network 300A. In some embodiments, the weights of the descriptor network 300A can be modified to increase or decrease the total distance D toward the desired value. total For example, if training images 302 and 303 are known to be similar, the weights of descriptor network 300A can be modified to make the total distance D... total Towards zero reduction. As another example, if it is known that training images 302 and 303 are dissimilar images, the weights of descriptor network 300A can be modified to make the total distance D decrease. total Increase toward 1. In some embodiments, the weights of the descriptor network 300A can be modified using a weight modifier 318A, which can employ, for example, backpropagation techniques to adjust the weights.

[0055] Figure 3BExample systems for training descriptor networks 300B according to some embodiments of the present disclosure are shown. Descriptor network 300B may be similar to descriptor network 100B. Similar to... Figure 3A As described, during a single training iteration of multiple training iterations, the descriptor network 300B can provide training images 302 and 303. When training images 302 and 303 are provided as input (e.g., sequentially), the descriptor network 300B can generate image descriptors 310B and 311B, each with a primary vector and a secondary vector. The total distance D can then be calculated based on the image descriptors 310B and 311B. total .

[0056] Similar to Figure 3A As described above, by, for example, modifying the weights of the descriptor network 300B, the total distance D total It can be used to train a descriptor network 300B. In some embodiments, the weights of the descriptor network 300B can be modified to increase or decrease the total distance D toward the desired value. total For example, if training images 302 and 303 are known to be similar, the weights of descriptor network 300B can be modified to make the total distance D... total Towards zero reduction. As another example, if it is known that training images 302 and 303 are dissimilar images, the weights of descriptor network 300B can be modified to make the total distance D decrease. total Increase toward 1. In some embodiments, a weight modifier 318B can be used to modify the weights of the descriptor network 300B, which can employ, for example, backpropagation techniques to adjust the weights.

[0057] Figure 4 Method 400 using a descriptor network (e.g., descriptor networks 100A, 100B, 200, 300A, 300B) according to some embodiments of this disclosure is illustrated. During execution of method 400, one or more steps of method 400 may be omitted, and the steps of method 400 need not be performed in the order shown. One or more steps of method 400 may be executed by one or more processors. Method 400 may be implemented as a computer-readable medium or computer program product including instructions that, when executed by one or more computers, cause one or more computers to perform the steps of method 400. Such a computer program product may be transmitted over a data carrier signal carrying the computer program product via a wired or wireless network.

[0058] At step 402, a first image (e.g., images 102, 202, 203, 302, 303) is received. The first image may be a grayscale image, a multi-channel image (e.g., an RGB image), or other possibilities. The first image may be the original image or a portion thereof.

[0059] At step 404, the first image is provided to the descriptor network.

[0060] At step 406, the descriptor network generates first image descriptors (e.g., image descriptors 110A, 110B, 210, 211, 310A, 311A, 310B, 311B) based on the first image. The first image descriptor may include a first set of elements. The first image descriptor may include a first primary vector (e.g., primary vector 112) comprising a first subset of the first set of elements and a second primary vector (e.g., secondary vector 114) comprising a second subset of the first set of elements. In some embodiments, the second subset of the first set of elements includes more elements than the first subset of the first set of elements (e.g., N > M).

[0061] At step 408, hierarchical normalization is applied to the first image descriptor. In some embodiments, applying hierarchical normalization to the first image descriptor may include normalizing a first primary vector of the first image descriptor to a primary normalization amount and normalizing a second primary vector of the first image descriptor to a secondary normalization amount.

[0062] At step 410, a second image (e.g., images 102, 202, 203, 302, 303) is received. The second image may be a grayscale image, a multi-channel image (e.g., an RGB image), or other possibilities. The second image may be the original image or a portion thereof.

[0063] At step 412, the second image is provided to the descriptor network.

[0064] At step 414, the descriptor network generates second image descriptors (e.g., image descriptors 110A, 110B, 210, 211, 310A, 311A, 310B, 311B) based on the second image. The second image descriptor may include a second set of elements. The second image descriptor may include a second primary vector (e.g., primary vector 112) comprising a first subset of the second set of elements and a second secondary vector (e.g., secondary vector 114) comprising the second subset of the second set of elements. In some embodiments, the second subset of the second set of elements includes more elements than the first subset of the second set of elements (e.g., N > M).

[0065] At step 416, hierarchical normalization is applied to the second image descriptor. In some embodiments, applying hierarchical normalization to the second image descriptor may include normalizing the second primary vector of the second image descriptor to a primary normalization amount and normalizing the second secondary vector of the second image descriptor to a secondary normalization amount.

[0066] At step 418, it is determined whether the first image matches the second image based on the first image descriptor and the second image descriptor. In some embodiments, step 418 includes one or more of steps 420 to 426.

[0067] At step 420, the principal distance between the first image and the second image is calculated based on the first principal vector and the second principal vector. In some embodiments, calculating the principal distance includes calculating the Euclidean distance and / or cosine distance between the first principal vector and the second principal vector. In some embodiments, calculating the principal distance includes calculating the sum of squares of the differences between a first subset of the first set of elements and a first subset of the second set of elements.

[0068] At step 422, it is determined whether the first image matches the second image based on the major distance. In some embodiments, determining that the first image matches the second image includes determining that the major distance is greater than an upper threshold. In some embodiments, determining that the first image does not match the second image includes determining that the major distance is less than a lower threshold.

[0069] In some embodiments, if no determination is made at step 422 regarding whether the images match, method 400 terminates. In some embodiments, it may not be possible to determine whether the first image matches the second image based solely on the primary distance (e.g., the primary distance is between an upper threshold and a lower threshold). In such embodiments, method 400 proceeds to step 424 and calculates the secondary distance.

[0070] At step 424, a secondary distance between the first image and the second image is calculated based on the first and second target vectors. In some embodiments, calculating the secondary distance includes calculating the Euclidean distance and / or cosine distance between the first and second target vectors. In some embodiments, calculating the secondary distance includes calculating the sum of squares of the differences between a second subset of the first set of elements and a second subset of the second set of elements.

[0071] At step 426, it is determined whether the first image matches the second image based on the primary distance and the secondary distance. In some embodiments, the sum of the primary distance and the secondary distance (or the total distance) is calculated. In some embodiments, determining that the first image matches the second image includes determining that the sum of the primary distance and the secondary distance is greater than an upper threshold. In some embodiments, determining that the first image does not match the second image includes determining that the sum of the primary distance and the secondary distance is less than an upper threshold.

[0072] Figure 5 Method 500 for training a descriptor network (e.g., descriptor networks 100A, 100B, 200, 300A, 300B) according to some embodiments of the present disclosure is illustrated. During execution of method 500, one or more steps of method 500 may be omitted, and the steps of method 500 need not be performed in the order shown. One or more steps of method 500 may be executed by one or more processors. Method 500 may be implemented as a computer-readable medium or computer program product including instructions that, when executed by one or more computers, cause the one or more computers to perform the steps of method 500. Such a computer program product may be transmitted over a data carrier signal carrying the computer program product via a wired or wireless network.

[0073] At step 502, a set of image pairs (e.g., images 202, 203, 302, 303) is received. In various embodiments, depending on the size of the training dataset, this set of image pairs may include 10 image pairs, 1000 image pairs, 1,000,000 image pairs, etc. Each image in each image pair in this set of image pairs may be a grayscale image, a multi-channel image (e.g., an RGB image), etc. Each image may be the original image or a portion of the original image.

[0074] In some embodiments, steps 504 to 520 are performed for each image pair in the set of image pairs. At step 504, a first training image from the image pair is provided to the descriptor network.

[0075] At step 506, the descriptor network generates first image descriptors (e.g., image descriptors 110A, 110B, 210, 211, 310A, 311A, 310B, 311B) based on the first training images. The first image descriptor may include a first set of elements. The first image descriptor may include a first primary vector (e.g., primary vector 112) comprising a first subset of the first set of elements and a second primary vector (e.g., secondary vector 114) comprising a second subset of the first set of elements. In some embodiments, the second subset of the first set of elements includes more elements than the first subset of the first set of elements (e.g., N>M).

[0076] At step 508, hierarchical normalization is applied to the first image descriptor. In some embodiments, applying hierarchical normalization to the first image descriptor may include normalizing a first primary vector of the first image descriptor to a primary normalization amount and normalizing a first secondary vector of the first image descriptor to a secondary normalization amount.

[0077] At step 510, a second training image from the image pair is provided to the descriptor network.

[0078] At step 512, the descriptor network generates second image descriptors (e.g., image descriptors 110A, 110B, 210, 211, 310A, 311A, 310B, 311B) based on the second training images. The second image descriptors may include a second set of elements. The second image descriptors may include a second primary vector (e.g., primary vector 112) comprising a first subset of the second set of elements and a second secondary vector (e.g., secondary vector 114) comprising the second subset of the second set of elements. In some embodiments, the second subset of the second set of elements includes more elements than the first subset of the second set of elements (e.g., N > M).

[0079] At step 514, hierarchical normalization is applied to the second image descriptor. In some embodiments, applying hierarchical normalization to the second image descriptor may include normalizing the second primary vector of the second image descriptor to a primary normalization amount and normalizing the second secondary vector of the second image descriptor to a secondary normalization amount.

[0080] At step 516, the principal distance between the first image and the second image is calculated based on the first principal vector and the second principal vector, similar to step 420. In some embodiments, it is determined whether the first training image matches the second training image based on the principal distance.

[0081] At step 518, a secondary distance between the first image and the second image is calculated based on the first and second target vectors, similar to step 424. In some embodiments, it is determined whether the first training image matches the second training image based on the secondary distance.

[0082] At step 520, the descriptor network is modified based on the primary and / or secondary distances. In some embodiments, the weights of the descriptor network are modified to increase or decrease the primary and / or secondary distances (e.g., the sum of distances) when the same image pairs are provided as input to the descriptor network. The weights of the descriptor network can be modified by a weight modifier (e.g., weight modifiers 318A, 318B) that can perform backpropagation techniques to adjust the weights of the descriptor network.

[0083] In some embodiments, the descriptor network can be trained sequentially by first training the primary vector and then training the secondary vector. For example, a set of image pairs can be used to train the weights of the descriptor network that help compute the elements of the primary vector, while ignoring the elements of the secondary vector. Once trained, the weights of the descriptor network that help compute the elements of the primary vector may be fixed. Then, the weights of the descriptor network that help compute the elements of the secondary vector can be trained using the same set of image pairs or a different set of image pairs. In some embodiments, the elements of the primary vector may be ignored when training the elements of the secondary vector. In some embodiments, both the primary and secondary vectors may be considered when training the weights of the descriptor network that help compute the elements of the secondary vector. In some embodiments, the weights of the descriptor network that help compute the elements of both the primary and secondary vectors may be trained simultaneously.

[0084] Figure 6 Examples of hierarchical normalization according to some embodiments of this disclosure are shown. Instead of normalizing all elements to the value 1, the elements are divided into a primary vector and a secondary vector, and all elements in the primary vector are normalized to the value 1-α, and all elements in the secondary vector are normalized to the value α. In the example shown, α equals 1 / 16. Therefore, the primary vector is normalized to 15 / 16, and the secondary vector is normalized to 1 / 16. In the example shown, the primary vector comprises 8 elements, and the secondary vector comprises 24 elements.

[0085] Figure 7 An example flowchart of a method for determining whether two images match according to some embodiments of the present disclosure is shown. In the example shown, Euclidean distance is used to calculate a primary distance and a secondary distance. At step 702, the primary distance is calculated. At step 704, it is determined whether the primary distance is greater than an upper threshold. If the primary distance is greater than the upper threshold, the images are determined not to match. Otherwise, the method proceeds to step 706. At step 706, it is determined whether the primary distance is less than a lower threshold (equal to the upper threshold - 4α). If the primary distance is less than the lower threshold, the images are determined to match. Otherwise, the method proceeds to step 708.

[0086] At step 708, the secondary distance is calculated. At step 710, it is determined whether the sum of the primary and secondary distances is less than an upper threshold. If the sum of the primary and secondary distances is less than the upper threshold, the image is determined to match. Otherwise (e.g., if the sum of the primary and secondary distances is greater than the upper threshold), the image is determined not to match.

[0087] Figure 8 Some embodiments according to this disclosure are shown. Figure 7 The example flowchart shown is a graphical representation. Figure 8As shown, when the primary distance is greater than the upper threshold or less than the lower threshold, a match can be determined based solely on the primary distance. Otherwise, a match will be determined based on both the primary and secondary distances.

[0088] Figure 9 An example flowchart of a method for determining whether two images match according to some embodiments of the present disclosure is shown. In the example shown, cosine distance is used to calculate the primary distance and the secondary distance. At step 902, the primary distance is calculated by calculating the inner product between the primary vectors of the two images. At step 904, it is determined whether the primary distance is less than a lower threshold. If the primary distance is less than the lower threshold, the images are determined not to match. Otherwise, the method proceeds to step 906. At step 906, it is determined whether the primary distance is greater than an upper threshold (equal to the lower threshold + 2α). If the primary distance is greater than the upper threshold, the images are determined to match. Otherwise, the method proceeds to step 908.

[0089] At step 908, the secondary distance is calculated by computing the inner product of the secondary vectors of the two images. At step 910, it is determined whether the sum of the primary and secondary distances is greater than a median threshold (equal to the average between the upper and lower thresholds). If the sum of the primary and secondary distances is greater than the median threshold, the images are determined to match. Otherwise (e.g., if the sum of the primary and secondary distances is less than the median threshold), the images are determined to not match.

[0090] Figure 10 Some embodiments according to this disclosure are shown. Figure 9 The example flowchart shown is a graphical representation. Figure 10 As shown, when the primary distance is greater than the upper threshold or less than the lower threshold, a match can be determined based solely on the primary distance. Otherwise, a match will be determined based on both the primary and secondary distances.

[0091] Figure 11 An example flowchart illustrates a method for finding the image closest to a reference image from a set of images according to some embodiments of the present disclosure. In the illustrated example, Euclidean distance is used to calculate the primary and secondary distances. After generating a primary vector for image descriptors for the set of images and the reference image, at step 1102, the primary vector is used to calculate the primary distance between each image in the set of images and the reference image. At step 1104, the minimum primary distance is identified. At step 1106, all images having a primary distance less than the minimum distance + 4α are selected.

[0092] At step 1108, after generating secondary vectors for image descriptors for the selected image and the reference image, the secondary vectors are used to calculate the secondary distance between each selected image and the reference image. At step 1110, the closest image is selected by identifying the minimum total distance (which is the sum of the primary and secondary distances).

[0093] Figure 12 Some embodiments according to this disclosure are shown. Figure 11 The example flowchart shown is a graphical representation. Figure 12 As shown, when the primary distance is used to filter out images with a primary distance higher than the threshold, only the secondary distance needs to be used to search for small regions.

[0094] Figure 13 An example flowchart illustrates a method for finding the image closest to a reference image from a set of images according to some embodiments of the present disclosure. In the illustrated example, cosine distance is used to calculate the primary and secondary distances. After generating a primary vector for image descriptors for the set of images and the reference image, at step 1302, the primary distance between each image in the set of images and the reference image is calculated using the primary vector by computing the inner product. At step 1304, the maximum primary distance is identified. At step 1306, all images having a primary distance greater than the maximum distance - α are selected.

[0095] At step 1308, after generating secondary vectors for image descriptors for the selected and reference images, the secondary vectors are used to calculate the secondary distance between each selected image and the reference image by computing the inner product. At step 1310, the closest image is selected by identifying the maximum total distance (which is the sum of the primary and secondary distances).

[0096] Figure 14 Some embodiments according to this disclosure are shown. Figure 13 The example flowchart shown is a graphical representation. Figure 14 As shown, when the primary distance is used to filter out images with a primary distance below the threshold, only the secondary distance needs to be used to search for small regions.

[0097] Figure 15 Additional graphical representations of areas that can be filtered out using the primary distance are shown according to some embodiments of this disclosure. In the illustrated example, Euclidean distance is used to calculate the primary and secondary distances.

[0098] Figure 16 Additional graphical representations of regions that can be filtered out using the primary distance are shown according to some embodiments of this disclosure. In the illustrated example, cosine distance is used to calculate both the primary and secondary distances.

[0099] Figure 17 The diagram illustrates experimental results for a descriptor network according to some embodiments of the present disclosure. The diagram also shows precision as a function of recall.

[0100] Figure 18 A graph illustrating experimental results for a descriptor network according to some embodiments of this disclosure is shown. The graph shows the true positive rate as a function of the false positive rate. Fine-tuning was first performed over 300 epochs via uniform training, followed by non-uniform fine-tuning training over 30 epochs.

[0101] Figure 19 The diagram illustrates experimental results for a descriptor network according to some embodiments of the present disclosure. The diagram also shows precision as a function of recall.

[0102] Figure 20 A graph illustrating experimental results for a descriptor network according to some embodiments of the present disclosure is shown. The graph shows the true positive rate as a function of the false positive rate.

[0103] Figure 21 An example image descriptor 2100 with multiple primary vectors is shown according to some embodiments of the present disclosure. While descriptor networks are generally described herein as generating image descriptors with primary and secondary vectors, embodiments of the present disclosure further include descriptor networks configured to generate image descriptors with multiple primary vectors. In the illustrated example, image descriptor 2100 includes a first primary vector, a second primary vector, a third primary vector, and secondary vectors. In some embodiments, the first primary vector may include fewer elements than the second primary vector, the second primary vector may include fewer elements than the third primary vector, and the third primary vector may include fewer elements than the secondary vector.

[0104] When generating image descriptor 2100, the following hierarchical normalization can be adopted: the first principal vector can be normalized to the first principal normalization value, the second principal vector can be normalized to the second principal normalization value, the second principal normalization value is less than the first principal normalization value, the third principal vector can be normalized to the third principal normalization value, the third principal normalization value is less than the second principal normalization value, and the secondary vector can be normalized to the secondary normalization value, the secondary normalization value is less than the third principal normalization value.

[0105] Figure 22 An example computer system 2200 comprising various hardware elements is illustrated according to some embodiments of the present disclosure. The computer system 2200 may be incorporated into or integrated into the device described herein, and / or may be configured to perform some or all of the steps of the methods provided by the various embodiments. For example, in various embodiments, the computer system 2200 may be configured to perform method 400 or 500. It should be noted that...Figure 22 This is merely to provide a general overview of the various components, any or all of which may be used as appropriate. Therefore, Figure 22 It extensively demonstrates how individual system elements can be implemented in a relatively separate or relatively more integrated manner.

[0106] In the illustrated example, computer system 2200 includes communication medium 2202, one or more processors 2204, one or more input devices 2206, one or more output devices 2208, communication subsystem 2210, and one or more memory devices 2212. Computer system 2200 can be implemented using various hardware implementations and embedded system technologies. For example, one or more components of computer system 2200 can be implemented as a field-programmable gate array (FPGA), such as... or LATTICE Commercial field-programmable gate arrays, system-on-a-chip (SoC), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), microcontrollers, and / or hybrid devices such as SoCFPGA.

[0107] Various hardware components of computer system 2200 can be coupled via communication medium 2202. For clarity, communication medium 2202 is shown as a single connection, but it should be understood that communication medium 2202 may include various numbered and types of communication media for transmitting data between hardware components. For example, communication medium 2202 may include one or more wires (e.g., conductive traces, paths or leads on a printed circuit board (PCB) or integrated circuit (IC), microstrip, stripline, coaxial cable), one or more optical waveguides (e.g., optical fiber, strip waveguide), and / or one or more wireless connections or links (e.g., infrared wireless communication, radio communication, microwave wireless communication), etc.

[0108] In some embodiments, the communication medium 2202 may include one or more buses connecting pins of hardware components of the computer system 2200. For example, the communication medium 2202 may include a bus (referred to as a system bus) connecting the processor 2204 to the main memory 2214, and a bus (referred to as an expansion bus) connecting the main memory 2214 to an input device 2206 or an output device 2208. The system bus may consist of several components, including an address bus, a data bus, and a control bus. The address bus can carry memory addresses from the processor 2204 to address bus circuitry associated with the main memory 2214 so that the data bus can access and carry back the data contained at the memory address to the processor 2204. The control bus can carry commands from the processor 2204 and return status signals from the main memory 2214. Each bus may include multiple wires for carrying multiple bits of information, and each bus may support serial or parallel data transmission.

[0109] Processor 2204 may include one or more central processing units (CPUs), graphics processing units (GPUs), neural network processors or accelerators, digital signal processors (DSPs), etc. The CPU may be in the form of a microprocessor, manufactured on a single IC chip with a metal-oxide-semiconductor field-effect transistor (MOSFET) structure. Processor 2204 may include one or more multi-core processors, where each core can simultaneously read and execute program instructions with other cores.

[0110] Input device 2206 may include one or more various user input devices, such as a mouse, keyboard, microphone, and various sensor input devices, such as image capture devices, pressure sensors (e.g., barometers, tactile sensors), temperature sensors (e.g., thermometers, thermocouples, thermistors), motion sensors (e.g., accelerometers, gyroscopes, tilt sensors), and light sensors (e.g., photodiodes, photodetectors, charge-coupled devices). Input device 2206 may also include devices for reading and / or receiving removable storage devices or other removable media. Such removable media may include optical discs (e.g., Blu-ray discs, DVDs, CDs), memory cards (e.g., CompactFlash cards, Secure Digital (SD) cards, Memory Sticks), floppy disks, Universal Serial Bus (USB) flash drives, external hard disk drives (HDDs), or solid-state drives (SSDs).

[0111] Output device 2208 may include one or more of a variety of devices for converting information into a human-readable form, such as, but not limited to, display devices, speakers, printers, etc. Output device 2208 may also include devices for writing to removable storage devices or other removable media, such as the devices described with reference to input device 2206. Output device 2208 may also include various actuators for causing physical movement of one or more components. Such actuators may be hydraulic, pneumatic, or electric, and may be controlled by computer system 2200.

[0112] The communication subsystem 2210 may include hardware components for connecting the computer system 2200 to systems or devices located outside the computer system 2200, such as via a computer network. In various embodiments, the communication subsystem 2210 may include wired communication devices coupled to one or more input / output ports (e.g., universal asynchronous receiver-transmitter (UART)), optical communication devices (e.g., optical modems), infrared communication devices, or radio communication devices (e.g., wireless network interface controllers), optical communication devices, and wireless communication devices coupled to one or more input / output ports. Equipment, IEEE 802.11 equipment, Wi-Fi equipment, Wi-Max equipment, cellular equipment, etc.

[0113] Memory device 2212 may include various data storage devices of computer system 2200. For example, memory device 2212 may include various types of computer memory with different response times and capacities, ranging from memory with faster response times and lower capacities (such as processor registers and caches (e.g., L0, L1, L2)) to memory with medium response times and medium capacities (such as random access memory) to memory with lower response times and lower capacities (such as solid-state drives and hard disk drives). Although processor 2204 and memory device 2212 are shown as separate components, it should be understood that processor 2204 may include different levels of on-processor memory, such as processor registers and caches, which may be used by a single processor or shared among multiple processors.

[0114] Memory device 2212 may include main memory 2214, which can be directly accessed by processor 2204 via the memory bus of communication medium 2202. For example, processor 2204 can continuously read and execute instructions stored in main memory 2214. Therefore, various software elements can be loaded into main memory 2214 and... Figure 22The processor 2204 shown reads and executes data. Typically, main memory 2214 is volatile memory, losing all data when power is off, and therefore requires power to retain the stored data. Main memory 2214 may further include a small portion of non-volatile memory containing software (e.g., firmware, such as BIOS) for reading other software stored in memory device 2212 into main memory 2214. In some embodiments, the volatile memory of main memory 2214 is implemented as random access memory (RAM), such as dynamic RAM (DRAM), and the non-volatile memory of main memory 2214 is implemented as read-only memory (ROM), such as flash memory, erasable programmable read-only memory (EPROM), or electrically erasable programmable read-only memory (EEPROM).

[0115] Computer system 2200 may include software elements, shown as currently residing in main memory 2214, which may include an operating system, device drivers, firmware, compilers, and / or other code, such as one or more application programs, which may include computer programs provided by various embodiments of this disclosure. As an example only, one or more steps of any of the methods discussed above may be implemented as instructions 2216 executed by computer system 2200. In one example, such instructions 2216 may be received by computer system 2200 using communication subsystem 2210 (e.g., via wireless or wired signals carrying instructions 2216), carried by communication medium 2202 to memory device 2212, stored in memory device 2212, read into main memory 2214, and executed by processor 2204 to perform one or more steps of the method. In another example, instruction 2216 may be received by computer system 2200 using input device 2206 (e.g., via a reader for removable media), carried by communication medium 2202 to memory device 2212, stored in memory device 2212, read into main memory 2214, and executed by processor 2204 to perform one or more steps of the method.

[0116] In some embodiments of this disclosure, instruction 2216 is stored on a computer-readable storage medium, or simply on a computer-readable medium. Such a computer-readable medium may be non-transitory and therefore may be referred to as a non-transitory computer-readable medium. In some cases, the non-transitory computer-readable medium may be incorporated into computer system 2200. For example, the non-transitory computer-readable medium may be one of memory devices 2212, as shown in Figure 22, in which instruction 2216 is stored. In some cases, the non-transitory computer-readable medium may be separable from computer system 2200. In one example, the non-transitory computer-readable medium may be a removable medium provided to input device 2206, such as the medium described in reference input device 2206, as... Figure 22 As shown, instruction 2216 is provided to input device 2206. In another example, the non-transitory computer-readable medium may be a component of a remote electronic device (such as a mobile phone) that can wirelessly transmit data signals carrying instruction 2216 to computer system 2200 using communication subsystem 2210, such as... Figure 22 As shown, instruction 2216 is provided to communication subsystem 2210.

[0117] Instruction 2216 can take any suitable form that is read and / or executed by computer system 2200. For example, instruction 2216 can be source code (written in a human-readable programming language such as Java, C, C++, C#, Python), object code, assembly language, machine code, microcode, executable code, etc. In one example, instruction 2216 is provided to computer system 2200 in the form of source code, and a compiler is used to convert instruction 2216 from source code into machine code, which is then read into main memory 2214 for execution by processor 2204. As another example, instruction 2216 is provided to computer system 2200 in the form of an executable file with machine code, which can be immediately read into main memory 2214 for execution by processor 2204. In various examples, instruction 2216 may be provided to computer system 2200 in encrypted or unencrypted form, compressed or uncompressed form, as initialization for an installation package or broader software deployment, etc.

[0118] In one aspect of the invention, a system (e.g., computer system 2200) is provided to perform methods according to various embodiments of the present disclosure. For example, some embodiments may include a system comprising one or more processors (e.g., processor 2204) communicatively coupled to a non-transitory computer-readable medium (e.g., memory device 2212 or main memory 2214). The non-transitory computer-readable medium may store instructions (e.g., instruction 2216) therein that, when executed by the one or more processors, cause the one or more processors to perform the methods described in the various embodiments.

[0119] In another aspect of the invention, a computer program product including instructions (e.g., instruction 2216) is provided to perform methods according to various embodiments of the present disclosure. The computer program product may be tangibly embodied in a non-transitory computer-readable medium (e.g., memory device 2212 or main memory 2214). The instructions may be configured to cause one or more processors (e.g., processor 2204) to perform the methods described in the various embodiments.

[0120] In another aspect of this disclosure, a non-transitory computer-readable medium (e.g., memory device 2212 or main memory 2214) is provided. The non-transitory computer-readable medium may store instructions (e.g., instruction 2216) therein that, when executed by one or more processors (e.g., processor 2204), cause one or more processors to perform the methods described in various embodiments.

[0121] The methods, systems, and devices discussed above are examples. Various configurations can omit, substitute, or add various processes or components as needed. For example, in alternative configurations, the method may be performed in a different order than described, and / or different stages may be added, omitted, and / or combined. Furthermore, the features described with respect to certain configurations can be combined in various other configurations. Different aspects and elements of the configurations can be combined in a similar manner. Moreover, technology is constantly evolving, and therefore many elements are examples and do not limit the scope of this disclosure or the claims.

[0122] Specific details are provided in the description to offer a comprehensive understanding of the example configurations, including their implementations. However, the configurations can also be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary detail to avoid obscure configurations. This description provides only example configurations and does not limit the scope, applicability, or configuration of the claims. Rather, the foregoing description of the configurations will provide a supporting description for those skilled in the art to implement the techniques described. Various changes can be made to the function and arrangement of the elements without departing from the spirit or scope of this disclosure.

[0123] After describing several example configurations, various modifications, alternative structures, and equivalents can be used without departing from the spirit of this disclosure. For example, the foregoing elements may be components of a larger system in which other rules may take precedence over or otherwise modify the application of this technology. Furthermore, several steps may be taken before, during, or after considering the foregoing elements. Therefore, the foregoing description does not limit the scope of the claims.

[0124] As used herein and in the appended claims, the singular forms “a,” “an,” and “the” have plural meanings unless the context clearly specifies otherwise. Thus, for example, a reference to “user” includes a reference to one or more such users, and a reference to “processor” includes a reference to one or more processors and their equivalents known to those skilled in the art, and so on.

[0125] Furthermore, the terms “comprise,” “comprising,” “containing,” “including,” “including,” and “includes” are used in this specification and the following claims to describe the presence of the stated feature, whole, component, or step, but do not exclude the presence or addition of one or more other features, wholes, components, steps, actions, or groups.

[0126] It can also be understood that the examples and embodiments described herein are for illustrative purposes only, and various modifications or alterations based on these examples and embodiments will be suggested to those skilled in the art and will be included within the spirit and scope of this application and the appended claims.

Claims

1. A computer-implemented method, comprising: Receive the first image; The first image is provided as input to the descriptor network; Using the descriptor network, a first image descriptor is generated based on the first image, the first image descriptor comprising a first set of elements distributed among the following: The first principal vector comprises a first subset of the elements of the first set; as well as The first vector is required, which includes a second subset of the elements of the first set, wherein the second subset of the elements of the first set includes more elements than the first subset of the elements of the first set; and The first image descriptor is subjected to hierarchical normalization through the following operations: Normalize the first principal vector to the principal normalizer; and The first vector is normalized to a secondary normalization, wherein the secondary normalization is less than the primary normalization.

2. The computer-implemented method according to claim 1 further includes: Receive the second image; The second image is provided as input to the descriptor network; Using the descriptor network, a second image descriptor is generated based on the second image, the second image descriptor comprising a second set of elements distributed among the following: The second principal vector comprises a first subset of the elements of the second set. as well as The second vector comprises a second subset of the elements of the second set, wherein the second subset of the elements of the second set includes more elements than the first subset of the elements of the second set; and The hierarchical normalization is applied to the second image descriptor by the following operation: Normalize the second principal vector to the principal normalization factor; and The second vector is normalized to the second normalization value.

3. The computer-implemented method according to claim 2 further includes: The following steps are used to determine whether the first image matches the second image: Based on the first principal vector and the second principal vector, calculate the principal distance between the first image and the second image; and Determine whether the main distance is greater than the upper threshold.

4. The computer-implemented method according to claim 3, wherein, Determining whether the first image matches the second image further includes: Determine that the primary distance is greater than the upper threshold; and It is determined that the first image and the second image do not match.

5. The computer-implemented method according to claim 3, wherein, Determining whether the first image matches the second image further includes: Determine that the primary distance is not greater than the upper threshold; and Determine whether the primary distance is greater than the lower threshold.

6. The computer-implemented method according to claim 3, wherein, Calculating the principal distance between the first image and the second image based on the first principal vector and the second principal vector includes: Calculate the sum of squares of the differences between the first subset of the first group of elements and the first subset of the second group of elements.

7. The computer-implemented method according to claim 5, wherein, Determining whether the first image matches the second image further includes: Determine that the primary distance is greater than the lower threshold; Based on the first key vector and the second key vector, calculate the secondary distance between the first image and the second image; and Determine whether the sum of the primary distance and the secondary distance is greater than the upper threshold.

8. The computer-implemented method according to claim 7, wherein, Determining whether the first image matches the second image further includes: Determining that the sum of the primary distance and the secondary distance is greater than the upper threshold; and It is determined that the first image and the second image do not match.

9. The computer-implemented method according to claim 7, wherein, Determining whether the first image matches the second image further includes: Determine that the sum of the primary distance and the secondary distance is not greater than the upper threshold; and The first image is determined to match the second image.

10. The computer-implemented method according to claim 7, wherein, Calculating the secondary distance between the first image and the second image based on the first primary vector and the second primary vector includes: Calculate the sum of squares of the differences between the second subset of the first set of elements and the second subset of the second set of elements.

11. The computer-implemented method according to claim 1, wherein: The primary normalization factor is equal to 1-α; and The minor normalization factor is equal to α, where α is less than 0.

5.

12. The computer-implemented method according to claim 1, wherein, The descriptor network is a neural network with a set of weights that can be modified through the training process.

13. A system comprising: One or more processors; as well as A computer-readable medium storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including: Receive the first image; The first image is provided as input to the descriptor network; Using the descriptor network, a first image descriptor is generated based on the first image, the first image descriptor comprising a first set of elements distributed among the following: The first principal vector comprises a first subset of the elements of the first set; as well as The first vector is required, which includes a second subset of the elements of the first set, wherein the second subset of the elements of the first set includes more elements than the first subset of the elements of the first set; and The first image descriptor is subjected to hierarchical normalization through the following operations: Normalize the first principal vector to the principal normalizer; and The first vector is normalized to a secondary normalization, wherein the secondary normalization is less than the primary normalization.

14. The system according to claim 13, wherein, The operation also includes: Receive the second image; The second image is provided as input to the descriptor network; Using the descriptor network, a second image descriptor is generated based on the second image, the second image descriptor comprising a second set of elements distributed among the following: The second principal vector comprises a first subset of the elements of the second set; and The second vector comprises a second subset of the elements of the second set, wherein the second subset of the elements of the second set includes more elements than the first subset of the elements of the second set; and The hierarchical normalization is applied to the second image descriptor by the following operation: Normalize the second principal vector to the principal normalization factor; and The second vector is normalized to the second normalization value.

15. The system according to claim 14, wherein, The operation also includes: The following steps are used to determine whether the first image matches the second image: Based on the first principal vector and the second principal vector, calculate the principal distance between the first image and the second image; and Determine whether the main distance is greater than the upper threshold.

16. The system according to claim 15, wherein, Determining whether the first image matches the second image further includes: Determine that the primary distance is greater than the upper threshold; and It is determined that the first image and the second image do not match.

17. The system according to claim 15, wherein, Determining whether the first image matches the second image further includes: Determine that the primary distance is not greater than the upper threshold; and Determine whether the primary distance is greater than the lower threshold.

18. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: Receive the first image; The first image is provided as input to the descriptor network; Using the descriptor network, a first image descriptor is generated based on the first image, the first image descriptor comprising a first set of elements distributed among the following: The first principal vector comprises a first subset of the elements of the first set; as well as The first vector is required, which includes a second subset of the elements of the first set, wherein the second subset of the elements of the first set includes more elements than the first subset of the elements of the first set; and The first image descriptor is subjected to hierarchical normalization through the following operations: Normalize the first principal vector to the principal normalizer; and The first vector is normalized to a secondary normalization, wherein the secondary normalization is less than the primary normalization.

19. The non-transitory machine-readable medium according to claim 18, wherein, The operation also includes: Receive the second image; The second image is provided as input to the descriptor network; Using the descriptor network, a second image descriptor is generated based on the second image, the second image descriptor comprising a second set of elements distributed among the following: The second principal vector comprises a first subset of the elements of the second set; and The second vector comprises a second subset of the elements of the second set, wherein the second subset of the elements of the second set includes more elements than the first subset of the elements of the second set; and The hierarchical normalization is applied to the second image descriptor by the following operation: Normalize the second principal vector to the principal normalization factor; and The second vector is normalized to the second normalization value.

20. The non-transitory machine-readable medium according to claim 19, wherein, The operation also includes: The following steps are used to determine whether the first image matches the second image: Based on the first principal vector and the second principal vector, calculate the principal distance between the first image and the second image; and Determine whether the main distance is greater than the upper threshold.

Images