Image Descriptor Networks with Hierarchical Normalization

Abstract

Techniques for using and training a descriptor network are disclosed. An image may be received and provided to the descriptor network. The descriptor network may generate an image descriptor based on the image. The image descriptor may include a set of elements distributed between a major vector comprising a first subset of the set of elements and a minor vector comprising a second subset of the set of elements. The second subset of the set of elements may include more elements than the first subset of the set of elements. Hierarchical normalization may be applied to the image descriptor by normalizing the major vector to a major normalization amount and normalizing the minor vector to a minor normalization amount. The minor normalization amount may be less than the major normalization amount.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 019,211, filed May 1, 2020, and entitled "IMAGE DESCRIPTOR NETWORK WITH IMPOSED HIERARCHICAL NORMALIZATION," the entire contents of which are incorporated herein by reference for all purposes. [Background technology]

[0002] In image processing, an image descriptor, also called an "image descriptor", is a description of the visual features of an image, including its color, shape, texture, and motion, among other possibilities. Image descriptors and the algorithms that produce them have several applications in computer vision. For example, image descriptors provide a robust means for determining the similarity between two images, e.g., by generating an image descriptor for each image and calculating the distance or difference between the two descriptors. Summary of the Invention [Means for solving the problem]

[0003] The present disclosure generally relates to techniques for generating image descriptors for images. More specifically, embodiments of the present disclosure provide techniques for training and using a descriptor network to accurately generate image descriptors having principal and minor vectors.

[0004] A description of various embodiments of the invention is provided below as a list of examples. As used below, any reference to a series of examples is to be understood as a disjunctive reference to each of those examples (e.g., "Examples 1-4" is to be understood as "Examples 1, 2, 3, or 4").

[0005] Example 1 is a computer-implemented method comprising the steps of receiving a first image; providing the first image as an input to a descriptor network; and using the descriptor network to generate a first image descriptor based on the first image, the first image descriptor comprising a first set of elements distributed between a first principal vector comprising a first subset of the first set of elements and a first sub-vector comprising a second subset of the first set of elements, the second subset of the first set of elements comprising a first set of elements distributed between the first sub-vector comprising more elements than the first subset of the first set of elements; and applying hierarchical normalization to the first image descriptor by normalizing the first principal vector to a principal normalization amount and normalizing the first sub-vector to a sub-normalization amount, the sub-normalization amount being less than the principal normalization amount.

[0006] Example 2 is the computer-implemented method of Example 1, further comprising the steps of receiving a second image; providing the second image as an input to the descriptor network; and using the descriptor network to generate a second image descriptor based on the second image, the second image descriptor comprising a second set of elements distributed between a second principal vector comprising a first subset of the second set of elements and a second sub-vector comprising a second subset of the second set of elements, the second subset of the second set of elements comprising more elements than the first subset of the second set of elements; and applying hierarchical normalization to the second image descriptor by normalizing the second principal vector to a principal normalization quantity and normalizing the second sub-vector to a sub-normalization quantity.

[0007] Example 3 is the computer-implemented method of Example 2, further comprising determining whether the first image matches the second image by calculating a principal distance between the first image and the second image based on the first principal vector and the second principal vector, and determining whether the principal distance exceeds an upper threshold.

[0008] Example 4 is the computer-implemented method of Example 3, wherein the step of determining whether the first image matches the second image further includes the steps of determining that the main distance is above an upper threshold and determining that the first image does not match the second image.

[0009] Example 5 is the computer-implemented method described in Example 3, wherein the step of determining whether the first image matches the second image further includes the steps of determining that the main distance does not exceed an upper threshold and determining whether the main distance exceeds a lower threshold.

[0010] Example 6 is the computer-implemented method of Example 3, wherein calculating a principal distance between the first image and the second image based on the first principal vector and the second principal vector includes calculating a sum of squared differences between a first subset of the first set of elements and a first subset of the second set of elements.

[0011] Example 7 is a computer-implemented method described in Example 5, wherein the step of determining whether the first image matches the second image further includes the steps of determining that the main distance is greater than a lower threshold, calculating a secondary distance between the first image and the second image based on the first secondary vector and the second secondary vector, and determining whether the sum of the main distance and the secondary distance is greater than an upper threshold.

[0012] Example 8 is a computer-implemented method described in Example 7, wherein the step of determining whether the first image matches the second image further includes the steps of determining that the sum of the primary distance and the secondary distance exceeds an upper threshold, and determining that the first image does not match the second image.

[0013] Example 9 is a computer-implemented method described in Example 7, wherein the step of determining whether the first image matches the second image further includes the steps of determining that the sum of the primary distance and the secondary distance does not exceed an upper threshold, and determining that the first image matches the second image.

[0014] Example 10 is a computer-implemented method described in Example 7, wherein the step of calculating a sub-distance between the first image and the second image based on the first sub-vector and the second sub-vector includes a step of calculating a sum of squared differences between a second subset of the first set of elements and a second subset of the second set of elements.

[0015] Example 11 is the computer-implemented method of Examples 1-10, wherein the primary normalization amount is equal to 1-α, the secondary normalization amount is equal to α, and α is less than 0.5.

[0016] Example 12 is the computer-implemented method of example 11, wherein α is equal to 1 / 8, 1 / 16, or 1 / 32.

[0017] Example 13 is the computer-implemented method of Examples 1-12, wherein the descriptor network is a neural network having a set of weights that are modifiable through a training process.

[0018] Example 14 is a method of training a descriptor network, the method comprising: receiving a set of image pairs; for each image pair of the set of image pairs, providing a first training image from the image pair as an input to the descriptor network; generating a first image descriptor using the descriptor network based on the first training image, the first image descriptor including a first set of elements distributed between a first principal vector comprising a first subset of the first set of elements and a first sub-vector comprising a second subset of the first set of elements, the second subset of the first set of elements including more elements than the first subset of the first set of elements; normalizing the first principal vector to a principal normalization amount; and normalizing the first sub-vector to a sub-normalization amount, the sub-normalization amount being less than the principal normalization amount; and providing a second training image from the image pair as an input to the descriptor network. and generating a second training image descriptor based on the second image using the descriptor network, the second image descriptor comprising a second set of elements distributed between a second principal vector comprising a first subset of the second set of elements and a second secondary vector comprising a second subset of the second set of elements, the second subset of the second set of elements comprising more elements than the first subset of the second set of elements; applying hierarchical normalization to the second image descriptor by normalizing the second principal vector to a principal normalization amount and normalizing the second secondary vector to a secondary normalization amount; calculating a principal distance between the first training image and the second training image based on the first principal vector and the second principal vector; calculating a secondary distance between the first training image and the second training image based on the first secondary vector and the second secondary vector; and modifying the descriptor network based on the principal distance and the secondary distance.

[0019] Example 15 is a system comprising one or more processors and a computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform a method described in any of Examples 1-14.

[0020] Example 16 is a non-transitory machine-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to any of Examples 1-14. [Brief description of the drawings]

[0021] The accompanying drawings, which are included to provide a further understanding of the present disclosure, are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure, and together with the detailed description, serve to explain the principles of the present disclosure. No attempt is made to show structural details of the present disclosure in more detail than may be necessary for a fundamental understanding of the disclosure and various ways in which it may be practiced.

[0022] [Figure 1A] FIG. 1A illustrates an example system including a descriptor network.

[0023] [Figure 1B] FIG. 1B illustrates an example system including a descriptor network.

[0024] [Diagram 2] FIG. 2 illustrates an example system including a descriptor network.

[0025] [Figure 3A] FIG. 3A illustrates an example system for training a descriptor network.

[0026] [Figure 3B] FIG. 3B illustrates an exemplary system for training a descriptor network.

[0027] [Figure 4] FIG. 4 illustrates a method for using a descriptor network.

[0028] [Diagram 5] FIG. 5 illustrates a method for training a descriptor network.

[0029] [Figure 6] FIG. 6 illustrates an example of hierarchical normalization.

[0030] [Figure 7] FIG. 7 illustrates an exemplary flow chart showing a method for determining whether two images match.

[0031] [Figure 8] FIG. 8 illustrates a graphical representation of the exemplary flow chart illustrated in FIG.

[0032] [Figure 9] FIG. 9 illustrates an exemplary flow chart showing a method for determining whether two images match.

[0033] [Figure 10] FIG. 10 illustrates a graphical representation of the exemplary flow chart illustrated in FIG.

[0034] [Figure 11] FIG. 11 illustrates an exemplary flow chart showing a method for finding a closest image to a reference image from a set of images.

[0035] [Figure 12] FIG. 12 illustrates a graphical representation of the exemplary flow chart illustrated in FIG.

[0036] [Figure 13]FIG. 13 illustrates an exemplary flow chart showing a method for finding a closest image to a reference image from a set of images.

[0037] [Figure 14] FIG. 14 illustrates a graphical representation of the exemplary flow chart illustrated in FIG.

[0038] [Figure 15] FIG. 15 illustrates an additional graphical representation showing regions that can be filtered out using principal distance.

[0039] [Figure 16] FIG. 16 illustrates an additional graphical representation showing regions that can be filtered out using principal distance.

[0040] [Figure 17] FIG. 17 illustrates graphs showing experimental results of the descriptor network.

[0041] [Figure 18] FIG. 18 illustrates graphs showing experimental results of the descriptor network.

[0042] [Figure 19] FIG. 19 illustrates graphs showing experimental results of the descriptor network.

[0043] [Figure 20] FIG. 20 illustrates graphs showing experimental results of the descriptor network.

[0044] [Figure 21] FIG. 21 illustrates an example image descriptor having multiple principal vectors.

[0045] [Figure 22] FIG. 22 illustrates a simplified computer system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0046] DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS The accompanying drawings, which are included to provide a further understanding of the present disclosure, are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure, and together with the detailed description, serve to explain the principles of the present disclosure. No attempt is made to show structural details of the present disclosure in more detail than may be necessary for a fundamental understanding of the disclosure and various ways in which it may be practiced.

[0047] 1A illustrates an example system including a descriptor network 100A for generating an image descriptor 110A based on an image 102, according to some embodiments of the present disclosure. The descriptor network 100A may be a neural network, such as a convolutional neural network, having a particular architecture and a set of associated weights. The image 102 is provided as an input, while the descriptor network 100A may be configured to generate an image descriptor 110A having N elements 116A. Each of the elements 116A may represent a visual feature in the image 102.

[0048] 1B illustrates an example system including a descriptor network 100B, which is similar to the descriptor network 100A, except that the generated image descriptor 110B includes a major vector 112 and a minor vector 114. In some embodiments, the major vector 112 may include a first subset of elements 116B (elements 116B-1 to 116B-M) and the minor vector 114 may include a second subset of elements 116B (elements 116B-M+1 to 116B-M+N). Thus, the first subset may include M elements and the second subset may include N elements.

[0049] In some embodiments, the main vector 112 and the sub-vector 114 are obtained by applying hierarchical normalization to the image descriptor 110B. This may include normalizing the main vector 112 to a main normalization quantity and the sub-vector 114 to a sub-normalization quantity, the main normalization quantity being greater than the sub-normalization quantity. By setting the size of the main vector 112 (M) to be less than the size of the sub-vector 114 (N) in conjunction with applying hierarchical normalization, the image descriptor 110B can be used to significantly improve performance in various tasks, as described herein.

[0050] FIG. 2 illustrates an example system including a descriptor network 200 according to some embodiments of the present disclosure. In the illustrated example, the descriptor network 200 is used to determine similarity between images 202 and 203 by determining the distance between image descriptors 210 and 211 generated based on the images 202 and 203, respectively. The descriptor network 200 may be similar to the descriptor network 100B such that the image descriptors 210 and 211 each include a major vector and a minor vector. In various embodiments, the images 202 and 203 may be provided as inputs to the descriptor network 200 sequentially (e.g., the image 202 may be provided at a first time and the image 203 may be provided at a second time after the first time) or may be provided simultaneously to two different instances of the descriptor network 200 (e.g., instructions for implementing the model architecture and weightings associated with the descriptor network 200 may be loaded onto two separate processors or processor cores).

[0051] The total distance D between the image descriptors 210 and 211 total is the principal distance between image descriptors D major (The total distance D total (part of the image) and the sub-distance D minor (The total distance Dtotal In the illustrated embodiment, the Euclidean distance may be calculated as the sum of the principal vectors (x1, x2, ..., x M ) and the principal vector (x' M , x'2, …, x' M ) and the sub-vector (x M+1 , x M+2 , …, x M+N ) and the sub-vector (x' M+1 , x' M+2 , …, x' M+N ) and a minor distance based on the elements of (a) and (b). Other distance metrics, such as cosine distance, may also be used to calculate the major and minor distances.

[0052] 3A illustrates an example system for training a descriptor network 300A according to some embodiments of the present disclosure. The descriptor network 300A may be similar to the descriptor network 100A. During a single training iteration of the multiple training iterations, the descriptor network 300A may be provided with training images 302 and 303. In some embodiments, the training images 302 and 303 may be known to be similar or different images. For example, the training image 303 may be a slightly modified version of the training image 302. When the training images 302 and 303 are provided as inputs (e.g., sequentially), the descriptor network 300A may be configured to generate image descriptors 310A and 311A, respectively. The total distance D total may then be calculated based on the image descriptors 310A and 311A.

[0053] Total distance D total may be used to train the descriptor network 300A, for example, by modifying the weights of the descriptor network 300A. In some embodiments, the weights of the descriptor network 300A are totalmay be modified to increase or decrease the total distance D towards a desired value. For example, if the training images 302 and 303 are known to be similar images, the weights of the descriptor network 300A may be modified to increase or decrease the total distance D total As another example, if the training images 302 and 303 are known to be different images, the weights of the descriptor network 300A may be modified to reduce the total distance D total may be modified to increase the weights towards 1. In some embodiments, the weights of the descriptor network 300A may be modified using a weight modifier 318A, which may employ, for example, back-propagation techniques to adjust the weights.

[0054] 3B illustrates an example system for training a descriptor network 300B according to some embodiments of the present disclosure. The descriptor network 300B may be similar to the descriptor network 100B. Similar to that illustrated in FIG. 3A, during a single training iteration of multiple training iterations, the descriptor network 300B may be provided with training images 302 and 303. When the training images 302 and 303 are provided as inputs (e.g., sequentially), the descriptor network 300B may generate image descriptors 310B and 311B, each having a major vector and a minor vector. The total distance D total may then be calculated based on the image descriptors 310B and 311B.

[0055] Similar to that illustrated in FIG. 3A, the total distance D total may be used to train the descriptor network 300B, for example, by modifying the weights of the descriptor network 300B. In some embodiments, the weights of the descriptor network 300B are total may be modified to increase or decrease the total distance D towards a desired value. For example, if the training images 302 and 303 are known to be similar images, the weights of the descriptor network 300B may be modified to increase or decrease the total distance D totalAs another example, if the training images 302 and 303 are known to be different images, the weights of the descriptor network 300B may be modified to reduce the total distance D total may be modified to increase the weights towards 1. In some embodiments, the weights of the descriptor network 300B may be modified using a weight modifier 318B, which may employ, for example, back-propagation techniques to adjust the weights.

[0056] FIG. 4 illustrates a method 400 of using a descriptor network (e.g., descriptor networks 100A, 100B, 200, 300A, 300B) according to some embodiments of the present disclosure. One or more steps of the method 400 may be omitted during the implementation of the method 400, and the steps of the method 400 need not be performed in the order shown. One or more steps of the method 400 may be performed by one or more processors. The method 400 may be implemented as a computer-readable medium or computer program product comprising instructions that, when executed by one or more computers, cause the one or more computers to perform the steps of the method 400. Such a computer program product may be transmitted over a wired or wireless network within a data carrier signal carrying the computer program product.

[0057] In step 402, a first image (e.g., image 102, 202, 203, 302, 303) is received. The first image may be a grayscale image, a multi-channel image (e.g., an RGB image), among other possibilities. The first image may be an original image or a portion of an original image.

[0058] In step 404, a first image is provided to the descriptor network.

[0059] In step 406, the descriptor network generates a first image descriptor (e.g., image descriptor 110A, 110B, 210, 211, 310A, 311A, 310B, 311B) based on the first image. The first image descriptor may include a first set of elements. The first image descriptor may include a first main vector (e.g., main vector 112) that includes a first subset of the first set of elements and a first secondary vector (e.g., secondary vector 114) that includes a second subset of the first set of elements. In some embodiments, the second subset of the first set of elements includes more elements than the first subset of the first set of elements (e.g., N>M).

[0060] In step 408, a hierarchical normalization is applied to the first image descriptor. In some embodiments, applying a hierarchical normalization to the first image descriptor may include normalizing a first principal vector of the first image descriptor to a principal normalization quantity and normalizing a first minor vector of the first image descriptor to a minor normalization quantity.

[0061] In step 410, a second image (e.g., image 102, 202, 203, 302, 303) is received. The second image may be a grayscale image, a multi-channel image (e.g., an RGB image), among other possibilities. The second image may be the original image or a portion of the original image.

[0062] In step 412, a second image is provided to the descriptor network.

[0063] In step 414, the descriptor network generates a second image descriptor based on the second image (e.g., image descriptors 110A, 110B, 210, 211, 310A, 311A, 310B, 311B). The second image descriptor may include a second set of elements. The second image descriptor may include a second major vector (e.g., major vector 112) that includes a first subset of the second set of elements, and a second minor vector (e.g., minor vector 114) that includes a second subset of the second set of elements. In some embodiments, the second subset of the second set of elements includes more elements than the first subset of the second set of elements (e.g., N>M).

[0064] In step 416, a hierarchical normalization is applied to the second image descriptor. In some embodiments, applying a hierarchical normalization to the second image descriptor may include normalizing a second principal vector of the second image descriptor to a principal normalization quantity and normalizing a second minor vector of the second image descriptor to a minor normalization quantity.

[0065] In step 418, it is determined whether the first image matches the second image based on the first image descriptor and the second image descriptor. In some embodiments, step 418 includes one or more of steps 420-426.

[0066] In step 420, a principal distance between the first image and the second image is calculated based on the first principal vector and the second principal vector. In some embodiments, calculating the principal distance comprises calculating a Euclidean distance and / or a cosine distance between the first principal vector and the second principal vector. In some embodiments, calculating the principal distance comprises calculating a sum of squares of differences between a first subset of the first set of elements and a first subset of the second set of elements.

[0067] In step 422, it is determined whether the first image matches the second image based on the primary distance. In some embodiments, determining that the first image matches the second image includes determining that the primary distance is above an upper threshold. In some embodiments, determining that the first image does not match the second image includes determining that the primary distance is below a lower threshold.

[0068] In some embodiments, if any determination is made at step 422 regarding whether the images match, method 400 ends. In some embodiments, it may be impossible to determine whether a first image matches a second image based solely on the primary distance (e.g., the primary distance is between an upper and lower threshold). In such embodiments, method 400 proceeds to step 424, where the secondary distance is calculated.

[0069] In step 424, a sub-distance between the first image and the second image is calculated based on the first sub-vector and the second sub-vector. In some embodiments, calculating the sub-distance comprises calculating a Euclidean distance and / or a cosine distance between the first sub-vector and the second sub-vector. In some embodiments, calculating the sub-distance comprises calculating a sum of squares of differences between a second subset of the first set of elements and a second subset of the second set of elements.

[0070] In step 426, it is determined whether the first image matches the second image based on the primary and secondary distances. In some embodiments, the sum of the primary and secondary distances (or total distance) is calculated. In some embodiments, determining that the first image matches the second image includes determining that the sum of the primary and secondary distances is above an upper threshold. In some embodiments, determining that the first image does not match the second image includes determining that the sum of the primary and secondary distances is below an upper threshold.

[0071] FIG. 5 illustrates a method 500 of training a descriptor network (e.g., descriptor networks 100A, 100B, 200, 300A, 300B) according to some embodiments of the present disclosure. One or more steps of the method 500 may be omitted during the implementation of the method 500, and the steps of the method 500 need not be performed in the order shown. One or more steps of the method 500 may be performed by one or more processors. The method 500 may be implemented as a computer-readable medium or computer program product comprising instructions that, when executed by one or more computers, cause the one or more computers to perform the steps of the method 500. Such a computer program product may be transmitted via wired or wireless networks within a data carrier signal carrying the computer program product.

[0072] In step 502, a set of image pairs (e.g., images 202, 203, 302, 303) is received. In various embodiments, the set of image pairs may include 10 image pairs, 1,000 image pairs, or 1,000,000 image pairs, among other possibilities, depending on the size of the training dataset. Each image in each image pair of the set of image pairs may be, among other possibilities, a grayscale image, a multi-channel image (e.g., an RGB image). Each image may be an original image or a portion of an original image.

[0073] In some embodiments, steps 504-520 are performed for each image pair in the set of image pairs.In step 504, a first training image from the image pair is provided to the descriptor network.

[0074] In step 506, the descriptor network generates a first image descriptor (e.g., image descriptor 110A, 110B, 210, 211, 310A, 311A, 310B, 311B) based on the first training image. The first image descriptor may include a first set of elements. The first image descriptor may include a first principal vector (e.g., principal vector 112) that includes a first subset of the first set of elements, and a first secondary vector (e.g., secondary vector 114) that includes a second subset of the first set of elements. In some embodiments, the second subset of the first set of elements includes more elements than the first subset of the first set of elements (e.g., N>M).

[0075] In step 508, a hierarchical normalization is applied to the first image descriptor. In some embodiments, applying a hierarchical normalization to the first image descriptor may include normalizing a first principal vector of the first image descriptor to a principal normalization quantity and normalizing a first minor vector of the first image descriptor to a minor normalization quantity.

[0076] In step 510, a second training image from the image pair is provided to the descriptor network.

[0077] In step 512, the descriptor network generates a second image descriptor (e.g., image descriptor 110A, 110B, 210, 211, 310A, 311A, 310B, 311B) based on the second training image. The second image descriptor may include a second set of elements. The second image descriptor may include a second principal vector (e.g., principal vector 112) that includes a first subset of the second set of elements, and a second minor vector (e.g., minor vector 114) that includes a second subset of the second set of elements. In some embodiments, the second subset of the second set of elements includes more elements than the first subset of the second set of elements (e.g., N>M).

[0078] In step 514, a hierarchical normalization is applied to the second image descriptor. In some embodiments, applying a hierarchical normalization to the second image descriptor may include normalizing a second principal vector of the second image descriptor to a principal normalization quantity and normalizing a second minor vector of the second image descriptor to a minor normalization quantity.

[0079] In step 516, a principal distance between the first image and the second image is calculated based on the first principal vector and the second principal vector, similar to step 420. In some embodiments, based on the principal distance, it is determined whether the first training image matches the second training image.

[0080] In step 518, a sub-distance between the first image and the second image is calculated based on the first sub-vector and the second sub-vector, similar to step 424. In some embodiments, based on the sub-distance, it is determined whether the first training image matches the second training image.

[0081] In step 520, the descriptor network is modified based on the primary and / or secondary distances. In some embodiments, the weights of the descriptor network are modified to increase or decrease the primary and / or secondary distances (e.g., sum of distances) when identical image pairs are provided as inputs to the descriptor network. The weights of the descriptor network may be modified by a weight modifier (e.g., weight modifiers 318A, 318B), which may implement a back-propagation technique to adjust the weights of the descriptor network.

[0082] In some embodiments, the descriptor network may be trained sequentially by first training the main vector, followed by training the secondary vector. For example, the weights of the descriptor network that contribute to computing the elements of the main vector may be trained using a set of image pairs while ignoring the elements of the secondary vector. Once trained, the weights of the descriptor network that contribute to computing the elements of the main vector may be fixed. Thereafter, the weights of the descriptor network that contribute to computing the elements of the secondary vector may be trained using the same set of image pairs or a different set of image pairs. In some embodiments, the elements of the main vector may be ignored while training the elements of the secondary vector. In some embodiments, both the main vector and the secondary vector may be considered while training the weights of the descriptor network that contribute to computing the elements of the secondary vector. In some embodiments, the weights of the descriptor network that contribute to computing the elements of the main vector and the secondary vector may be trained simultaneously.

[0083] 6 illustrates an example of hierarchical normalization according to some embodiments of the present disclosure. Instead of normalizing all of the elements to a value of 1, the elements are split into a major vector and a minor vector, and all elements in the major vector are normalized to a value of 1-α, and all elements in the minor vector are normalized to a value of α. In the illustrated example, α is equal to 1 / 16. Thus, the major vector is normalized to 15 / 16, and the minor vector is normalized to 1 / 16. In the illustrated example, the major vector contains 8 elements, and the minor vector contains 24 elements.

[0084] FIG. 7 illustrates an exemplary flow chart showing a method for determining whether two images match according to some embodiments of the present disclosure. In the illustrated example, Euclidean distance is used to calculate the primary and secondary distances. In step 702, the primary distance is calculated. In step 704, it is determined whether the primary distance is greater than an upper threshold. If the primary distance is greater than an upper threshold, it is determined that the images do not match. Otherwise, the method proceeds to step 706. In step 706, it is determined whether the primary distance is less than a lower threshold (equal to the upper threshold minus 4α). If the primary distance is less than a lower threshold, it is determined that the images match. Otherwise, the method proceeds to step 708.

[0085] In step 708, the minor distance is calculated. In step 710, it is determined whether the sum of the major distance and the minor distance is less than the upper threshold. If the sum of the major distance and the minor distance is less than the upper threshold, it is determined that the images match. Otherwise (e.g., if the sum of the major distance and the minor distance is greater than the upper threshold), it is determined that the images do not match.

[0086] 8 illustrates a graphical representation of the exemplary flow chart illustrated in FIG. 7 according to some embodiments of the present disclosure. As shown in FIG. 8, matching can be determined based only on the primary distance when the primary distance is above an upper threshold or when the primary distance is below a lower threshold. Otherwise, matching is determined based on both the primary distance and the secondary distance.

[0087] FIG. 9 illustrates an exemplary flow chart showing a method for determining whether two images match according to some embodiments of the present disclosure. In the illustrated example, cosine distance is used to calculate the principal distance and the secondary distance. In step 902, the principal distance is calculated by calculating the dot product between the principal vectors of the two images. In step 904, it is determined whether the principal distance is less than a lower threshold. If the principal distance is less than a lower threshold, it is determined that the images do not match. Otherwise, the method proceeds to step 906. In step 906, it is determined whether the principal distance is greater than an upper threshold (equal to a lower threshold + 2α). If the principal distance is greater than an upper threshold, it is determined that the images match. Otherwise, the method proceeds to step 908.

[0088] In step 908, the minor distance is calculated by calculating the dot product between the minor vectors of the two images. In step 910, it is determined whether the sum of the major distance and the minor distance is above the central threshold (equal to the average between the upper and lower thresholds). If the sum of the major distance and the minor distance is above the central threshold, it is determined that the images match. Otherwise (e.g., if the sum of the major distance and the minor distance is less than the central threshold), it is determined that the images do not match.

[0089] 10 illustrates a graphical representation of the exemplary flow chart illustrated in FIG 9 according to some embodiments of the present disclosure. As shown in FIG 10, matching can be determined based only on the primary distance when the primary distance is above an upper threshold or when the primary distance is below a lower threshold. Otherwise, matching is determined based on both the primary distance and the secondary distance.

[0090] FIG. 11 illustrates an exemplary flowchart showing a method for finding the closest image to a reference image from a set of images according to some embodiments of the present disclosure. In the illustrated example, Euclidean distance is used to calculate the principal and minor distances. After the principal vectors of the image descriptors are generated for the set of images and the reference image, in step 1102, the principal distances are calculated between each of the set of images and the reference image using the principal vectors. In step 1104, the minimum principal distance is identified. In step 1106, all images having a principal distance less than the minimum distance + 4α are selected.

[0091] In step 1108, after sub-vectors of image descriptors have been generated for the selected and reference images, sub-distances are calculated between each of the selected images and the reference image using the sub-vectors. In step 1110, the closest image is selected by identifying the smallest total distance, which is the sum of the primary and secondary distances.

[0092] 12 illustrates a graphical representation of the exemplary flowchart illustrated in FIG 11, according to some embodiments of the present disclosure. As shown in FIG 12, only small regions need to be searched using the minor distance when the major distance is used to filter out images that have a major distance above a threshold.

[0093] FIG. 13 illustrates an exemplary flow chart showing a method for finding the closest image from a set of images to a reference image according to some embodiments of the present disclosure. In the illustrated example, the cosine distance is used to calculate the principal and minor distances. After the principal vectors of the image descriptors are generated for the set of images and the reference image, in step 1302, the principal distances are calculated between each of the set of images and the reference image using the principal vectors by calculating an inner product. In step 1304, the maximum principal distance is identified. In step 1306, all images having a principal distance greater than the maximum distance -α are selected.

[0094] In step 1308, after sub-vectors of image descriptors are generated for the selected and reference images, sub-distances are calculated between each of the selected images and the reference image using the sub-vectors by calculating dot products. In step 1310, the closest image is selected by identifying the maximum total distance, which is the sum of the primary and secondary distances.

[0095] 14 illustrates a graphical representation of the exemplary flowchart illustrated in FIG 13, according to some embodiments of the present disclosure. As shown in FIG 14, only small regions need to be searched using the minor distance when the major distance is used to filter out images that have a major distance below a threshold.

[0096] 15 illustrates an additional graphical representation showing regions that may be filtered out using principal distances, in accordance with some embodiments of the present disclosure. In the illustrated example, Euclidean distances are used to calculate principal and minor distances.

[0097] 16 illustrates an additional graphical representation showing regions that may be filtered out using the principal distance, according to some embodiments of the present disclosure. In the illustrated example, the cosine distance is used to calculate the principal and minor distances.

[0098] 17 illustrates a graph showing experimental results of the described descriptor network, according to some embodiments of the present disclosure. The illustrated graph shows precision as a function of recall.

[0099] 18 illustrates a graph showing experimental results of the described descriptor network according to some embodiments of the present disclosure. The graph shown shows the correct detection rate as a function of the false positive rate. Fine-tuning is performed by uniformly training over 300 reference time points, followed by non-uniform fine-tuning training over 30 reference time points.

[0100] 19 illustrates a graph showing experimental results of the described descriptor network, according to some embodiments of the present disclosure. The illustrated graph shows precision as a function of recall.

[0101] 20 illustrates a graph showing experimental results of the described descriptor network, according to some embodiments of the present disclosure. The graph shown shows the correct detection rate as a function of the false detection rate.

[0102] 21 illustrates an example image descriptor 2100 having multiple principal vectors, according to some embodiments of the present disclosure. Although a descriptor network is generally described herein to generate an image descriptor having a principal vector and a minor vector, embodiments of the present disclosure further include a descriptor network configured to generate an image descriptor having multiple principal vectors. In the illustrated example, the image descriptor 2100 includes a first principal vector, a second principal vector, a third principal vector, and a minor vector. In some embodiments, the first principal vector may include fewer elements than the second principal vector, which may include fewer elements than the third principal vector, which may include fewer elements than the minor vector.

[0103] When generating the image descriptor 2100, a hierarchical normalization can be applied as follows: the first principal vector can be normalized to a first principal normalization amount, the second principal vector can be normalized to a second principal normalization amount that is less than the first principal normalization amount, the third principal vector can be normalized to a third principal normalization amount that is less than the second principal normalization amount, and the secondary vectors can be normalized to a secondary normalization amount that is less than the third principal normalization amount.

[0104] FIG. 22 illustrates an exemplary computer system 2200 comprising various hardware elements according to some embodiments of the present disclosure. The computer system 2200 may be incorporated into or integrated with a device described herein and / or configured to perform some or all of the steps of the methods provided by various embodiments. For example, in various embodiments, the computer system 2200 may be configured to perform methods 400 or 500. It should be noted that FIG. 22 is intended to provide only a generalized illustration of the various components, any or all of which may be utilized, as desired. FIG. 22 thus broadly illustrates how individual system elements may be implemented in a relatively separate or more relatively integrated manner.

[0105] In the illustrated embodiment, computer system 2200 includes a communication medium 2202, one or more processors 2204, one or more input devices 2206, one or more output devices 2208, a communication subsystem 2210, and one or more memory devices 2212. Computer system 2200 may be implemented using a variety of hardware implementations and embedded system technologies. For example, one or more elements of computer system 2200 may be implemented as a field programmable gate array (FPGA), such as those commercially available from XILINX®, INTEL®, or LATTICE SEMICONDUCTOR®, a system on a chip (SoC), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a microcontroller, and / or a hybrid device such as a SoC FPGA, among other possibilities.

[0106] The various hardware elements of computer system 2200 may be coupled via a communication medium 2202. Although communication medium 2202 is illustrated as a single connection for purposes of clarity, it should be understood that communication medium 2202 may include various numbers and types of communication media for transferring data between the hardware elements. For example, communication medium 2202 may include one or more wires (e.g., conductive traces, paths, or conductors on a printed circuit board (PCB) or integrated circuit (IC), microstrip, stripline, coaxial cable), one or more optical waveguides (e.g., optical fiber, strip waveguide), and / or one or more wireless connections or links (e.g., infrared wireless communication, wireless communication, microwave wireless communication), among other possibilities.

[0107] In some embodiments, the communication medium 2202 may include a bus connecting one or more pins of the hardware elements of the computer system 2200. For example, the communication medium 2202 may include a bus connecting the processor 2204 and the main memory 2214, referred to as a system bus, and a bus connecting the main memory 2214 and the input devices 2206 or the output devices 2208, referred to as an expansion bus. The system bus may consist of several elements including an address bus, a data bus, and a control bus. The address bus may carry memory addresses from the processor 2204 to the address bus circuitry associated with the main memory 2214, for the data bus to access and return data contained in the memory addresses to the processor 2204. The control bus may carry commands from the processor 2204 and return status signals from the main memory 2214. Each bus may include multiple wires for carrying multiple bits of information, and each bus may support serial or parallel transmission of data.

[0108] The processor 2204 may include one or more central processing units (CPUs), graphic processing units (GPUs), neural network processors or accelerators, digital signal processors (DSPs), and / or the like. The CPU may take the form of a microprocessor, which is fabricated on a single IC chip of metal-oxide-semiconductor field-effect transistor (MOSFET) construction. The processor 2204 may include one or more multi-core processors, in which each core may read and execute program instructions concurrently with the other cores.

[0109] The input devices 2206 may include one or more of a variety of user input devices, such as a mouse, keyboard, microphone, and a variety of sensor input devices, such as an image capture device, a pressure sensor (e.g., barometer, tactile sensor), a temperature sensor (e.g., thermometer, thermocouple, thermistor), a movement sensor (e.g., accelerometer, gyroscope, tilt sensor), an optical sensor (e.g., photodiode, photodetector, charge-coupled device), and / or the like. The input devices 2206 may also include a device for reading and / or receiving a removable storage device or other removable media. Such removable media may include an optical disk (e.g., Blu-ray® disk, DVD, CD), a memory card (e.g., compact flash card, secure digital (SD) card, memory stick), a floppy disk, a universal serial bus (USB) flash drive, an external hard disk drive (HDD) or solid state drive (SSD), and / or the like.

[0110] The output devices 2208 may include one or more of a variety of devices that convert information into a human readable form, such as, but not limited to, a display device, a speaker, a printer, and / or the like. The output devices 2208 may also include a removable storage device or a device for writing to other removable media, such as those described with reference to the input devices 2206. The output devices 2208 may also include various actuators for causing physical movement of one or more components. Such actuators may be hydraulic, pneumatic, electric, and may be provided with control signals by the computer system 2200.

[0111] Communications subsystem 2210 may include hardware components for connecting computer system 2200 to systems or devices located outside of computer system 2200, such as via a computer network. In various embodiments, communications subsystem 2210 may include wired communications devices coupled to one or more input / output ports (e.g., a universal asynchronous receiver / transmitter (UART)), optical communications devices (e.g., an optical modem), infrared communications devices, wireless communications devices (e.g., wireless network interface controllers, Bluetooth devices, IEEE 802.11 devices, Wi-Fi devices, Wi-Max devices, cellular devices), among other possibilities.

[0112] The memory devices 2212 may include various data storage devices of the computer system 2200. For example, the memory devices 2212 may include various types of computer memory with various response times and capacities, from faster response times and lower capacity memory, such as processor registers and caches (e.g., L0, L1, L2), to medium response times and medium capacity memory, such as random access memory, to slower response times and lower capacity memory, such as solid state drives and hard drive disks. Although the processor 2204 and the memory devices 2212 are illustrated as separate elements, it should be understood that the processor 2204 may include varying levels of on-processor memory, such as processor registers and caches, that may be utilized by a single processor or shared among multiple processors.

[0113] The memory device 2212 may include a main memory 2214, which may be directly accessible by the processor 2204 via a memory bus of the communication medium 2202. For example, the processor 2204 may persistently read and execute instructions stored in the main memory 2214. Thus, various software elements may be loaded into the main memory 2214 and read and executed by the processor 2204 as illustrated in FIG. 22. Typically, the main memory 2214 is a volatile memory that loses all data when power is turned off and therefore requires power to save the stored data. The main memory 2214 may further include a small portion of non-volatile memory containing software (e.g., firmware such as a BIOS) used to read other software stored in the reading memory device 2212 into the main memory 2214. In some embodiments, the volatile memory of main memory 2214 is implemented as random access memory (RAM), such as dynamic RAM (DRAM), and the non-volatile memory of main memory 2214 is implemented as read-only memory (ROM), such as flash memory, erasable programmable read-only memory (EPROM), or electrically erasable programmable read-only memory (EEPROM).

[0114] Computer system 2200 may include software elements, currently shown as residing in main memory 2214, which may include one or more application programs, such as an operating system, device drivers, firmware, compilers, and / or other code, which may include computer programs provided by various embodiments of the present disclosure. Merely by way of example, one or more steps described with respect to any of the methods discussed above may be implemented as instructions 2216 executable by computer system 2200. In one example, such instructions 2216 may be received by computer system 2200 using communications subsystem 2210 (e.g., via wireless or wired signals carrying the instructions 2216), carried by communications medium 2202 to memory device 2212, stored within memory device 2212, read into main memory 2214, and executed by processor 2204 to perform one or more steps of the methods described. In another embodiment, the instructions 2216 may be received by the computer system 2200 using the input device 2206 (e.g., via a reader for removable media), carried by the communication medium 2202 to the memory device 2212, stored in the memory device 2212, read into the main memory 2214, and executed by the processor 2204 to perform one or more steps of the methods described.

[0115] In some embodiments of the disclosure, the instructions 2216 are stored on a computer-readable storage medium, or simply a computer-readable medium. Such a computer-readable medium may be non-transitory and therefore may be referred to as a non-transitory computer-readable medium. In some cases, the non-transitory computer-readable medium may be incorporated within the computer system 2200. For example, the non-transitory computer-readable medium may be one of the memory devices 2212, as shown in FIG. 22, and the instructions 2216 are stored within the memory device 2212. In some cases, the non-transitory computer-readable medium may be separate from the computer system 2200. In one example, the non-transitory computer-readable medium may be a removable medium provided to an input device 2206, such as those described with reference to the input device 2206, as shown in FIG. 22, and the instructions 2216 are provided to the input device 2206. In another embodiment, the non-transitory computer readable medium may be a component of a remote electronic device, such as a cellular telephone, that may wirelessly transmit data signals carrying the instructions 2216 to the computer system 2200 using a communications subsystem 2210, as shown in FIG. 22, where the instructions 2216 are provided to the communications subsystem 2210.

[0116] The instructions 2216 may take any suitable form for being read and / or executed by the computer system 2200. For example, the instructions 2216 may be source code (written in a human readable programming language such as Java, C, C++, C#, Python, etc.), object code, assembly language, machine code, microcode, executable code, and / or the like. In one embodiment, the instructions 2216 are provided to the computer system 2200 in the form of source code and a compiler is used to convert the instructions 2216 from source code to machine code, which may then be read into the main memory 2214 for execution by the processor 2204. As another example, the instructions 2216 are provided to the computer system 2200 in the form of an executable file, with the machine code, which may be immediately read into the main memory 2214 for execution by the processor 2204. In various embodiments, the instructions 2216 may be provided to the computer system 2200 in encrypted or decrypted form, in compressed or uncompressed form, as an installation package or initialization, among other possibilities, for wider software deployment.

[0117] In one aspect of the disclosure, a system (e.g., computer system 2200) is provided for performing methods according to various embodiments of the disclosure. For example, some embodiments may include a system comprising one or more processors (e.g., processor 2204) communicatively coupled to a non-transitory computer-readable medium (e.g., memory device 2212 or main memory 2214). The non-transitory computer-readable medium may have instructions (e.g., instructions 2216) stored therein that, when executed by the one or more processors, cause the one or more processors to perform the methods described in the various embodiments.

[0118] In another aspect of the disclosure, a computer program product including instructions (e.g., instructions 2216) is provided for performing methods according to various embodiments of the disclosure. The computer program product may be tangibly embodied in a non-transitory computer-readable medium (e.g., memory device 2212 or main memory 2214). The instructions may be configured to cause one or more processors (e.g., processor 2204) to perform the methods described in the various embodiments.

[0119] In another aspect of the disclosure, a non-transitory computer readable medium is provided, such as the memory device 2212 or the main memory 2214. The non-transitory computer readable medium may have instructions stored therein, such as instructions 2216, that, when executed by one or more processors, such as the processor 2204, cause the one or more processors to perform the methods described in various embodiments.

[0120] The methods, systems, and devices discussed above are examples. Various configurations may omit, substitute, or add various procedures or components, as appropriate. For example, in alternative configurations, the method may be performed in a different order than described, and / or various steps may be added, omitted, and / or combined. Also, features described with respect to one configuration may be combined in various other configurations. Different aspects and elements of the configurations may be combined in a similar manner. Also, technology evolves, and thus many of the elements are examples and do not limit the scope of the disclosure or the claims.

[0121] Specific details are given in the description to provide a thorough understanding of the example configurations, including the implementation. However, the configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques are shown without unnecessary detail to avoid obscuring the configurations. This description provides only example configurations and does not limit the scope, applicability, or configuration of the claims. Rather, the foregoing description of the configurations will provide those skilled in the art with an effective description for implementing the described techniques. Various changes may be made in the function and arrangement of elements without departing from the spirit or scope of the present disclosure.

[0122] Although several example configurations have been described, various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the disclosure. For example, the aforementioned elements may be components of a larger system, and other rules may take precedence over or otherwise modify the application of the present technology. Also, some steps may take place before, during, or after the aforementioned elements are discussed. Thus, the foregoing description is not intended to be binding on the scope of the claims.

[0123] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Thus, for example, reference to a "user" includes reference to one or more such users, and reference to a "processor" includes reference to one or more processors and equivalents thereof known to those skilled in the art, and so forth.

[0124] Additionally, the words "comprise," "comprising," "contains," "containing," "include," "including," and "includes," when used in this specification and the claims that follow, are intended to specify the presence of stated features, integers, components, or steps, but they do not exclude the presence or addition of one or more other features, integers, components, steps, acts, or groups.

[0125] It should also be understood that the examples and embodiments described herein are for illustrative purposes only, and that various modifications or changes in light thereof will be suggested to those skilled in the art and are within the spirit and scope of the present application and the appended claims.

Claims

1. 1. A computer-implemented method comprising: Receiving a first image; providing said first image as an input to a descriptor network; generating a first image descriptor based on the first image using the descriptor network, the first image descriptor comprising: a first major vector comprising a first subset of the first set of elements; a first sub-vector comprising a second subset of the first set of elements, the second subset of the first set of elements including more elements than the first subset of the first set of elements; a first set of elements distributed between normalizing the first principal vector to a principal normalization quantity; normalizing the first sub-vector to a sub-normalized amount, the sub-normalized amount being less than the main normalized amount; applying hierarchical normalization to the first image descriptor by 4. A computer-implemented method comprising:

2. Receiving a second image; and providing said second image as an input to said descriptor network; generating a second image descriptor based on the second image using the descriptor network, the second image descriptor comprising: a second major vector comprising a first subset of the second set of elements; a second sub-vector comprising a second subset of the second set of elements, the second subset of the second set of elements including more elements than the first subset of the second set of elements; a second set of elements distributed between normalizing said second principal vector to said principal normalization quantity; normalizing the second sub-vector to the sub-normalized quantity; applying the hierarchical normalization to the second image descriptor by The computer-implemented method of claim 1 , further comprising:

3. calculating a principal distance between the first image and the second image based on the first principal vector and the second principal vector; determining whether the primary distance exceeds an upper threshold; determining whether the first image matches the second image by The computer-implemented method of claim 2 , further comprising:

4. Determining whether the first image matches the second image further comprises: determining that the primary distance is greater than the upper threshold; determining that the first image does not match the second image; The computer-implemented method of claim 3 , comprising:

5. Determining whether the first image matches the second image further comprises: determining that the primary distance does not exceed the upper threshold; determining whether the primary distance is greater than a lower threshold; The computer-implemented method of claim 3 , comprising:

6. Calculating a principal distance between the first image and the second image based on the first principal vector and the second principal vector includes: calculating a sum of squares of differences between a first subset of the first set of elements and a first subset of the second set of elements; The computer-implemented method of claim 3 , comprising:

7. Determining whether the first image matches the second image further comprises: determining that the primary distance is greater than the lower threshold; calculating a sub-distance between the first image and the second image based on the first sub-vector and the second sub-vector; determining whether the sum of the primary distance and the secondary distance exceeds the upper threshold; The computer-implemented method of claim 5 , comprising:

8. Determining whether the first image matches the second image further comprises: determining that the sum of the primary distance and the secondary distance exceeds the upper threshold; determining that the first image does not match the second image; The computer-implemented method of claim 7 , comprising:

9. Determining whether the first image matches the second image further comprises: determining that the sum of the primary distance and the secondary distance does not exceed the upper threshold; determining that the first image matches the second image; The computer-implemented method of claim 7 , comprising:

10. Calculating a sub-distance between the first image and the second image based on the first sub-vector and the second sub-vector includes: calculating a sum of squares of the differences between a second subset of the first set of elements and a second subset of the second set of elements; The computer-implemented method of claim 7 , comprising:

11. The principal normalization quantity is equal to 1-α, the sub-normalization amount is equal to α, where α is less than 0.5; 10. The computer-implemented method of claim 1.

12. The computer-implemented method of claim 1 , wherein the descriptor network is a neural network having a set of weights that are modifiable through a training process.

13. 1. A system comprising: one or more processors; A computer-readable medium having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to: Receiving a first image; providing said first image as an input to a descriptor network; generating a first image descriptor based on the first image using the descriptor network, the first image descriptor comprising: a first major vector comprising a first subset of the first set of elements; a first sub-vector comprising a second subset of the first set of elements, the second subset of the first set of elements including more elements than the first subset of the first set of elements; a first set of elements distributed between normalizing the first principal vector to a principal normalization quantity; normalizing the first sub-vector to a sub-normalized amount, the sub-normalized amount being less than the main normalized amount; applying hierarchical normalization to the first image descriptor by A computer-readable medium for performing operations including: A system comprising:

14. The operation further comprises: Receiving a second image; and providing said second image as an input to said descriptor network; generating a second image descriptor based on the second image using the descriptor network, the second image descriptor comprising: a second major vector comprising a first subset of the second set of elements; a second sub-vector comprising a second subset of the second set of elements, the second subset of the second set of elements including more elements than the first subset of the second set of elements; a second set of elements distributed between normalizing said second principal vector to said principal normalization quantity; normalizing the second sub-vector to the sub-normalized quantity; applying the hierarchical normalization to the second image descriptor by The system of claim 13 , comprising:

15. The operation further comprises: calculating a principal distance between the first image and the second image based on the first principal vector and the second principal vector; determining whether the primary distance exceeds an upper threshold; determining whether the first image matches the second image by The system of claim 14 , comprising:

16. Determining whether the first image matches the second image further comprises: determining that the primary distance is greater than the upper threshold; determining that the first image does not match the second image; The system of claim 15 , comprising:

17. Determining whether the first image matches the second image further comprises: determining that the primary distance does not exceed the upper threshold; determining whether the primary distance is greater than a lower threshold; The system of claim 15 , comprising:

18. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to: Receiving a first image; providing said first image as an input to a descriptor network; generating a first image descriptor based on the first image using the descriptor network, the first image descriptor comprising: a first major vector comprising a first subset of the first set of elements; a first sub-vector comprising a second subset of the first set of elements, the second subset of the first set of elements including more elements than the first subset of the first set of elements; a first set of elements distributed between normalizing the first principal vector to a principal normalization quantity; normalizing the first sub-vector to a sub-normalized amount, the sub-normalized amount being less than the main normalized amount; applying hierarchical normalization to the first image descriptor by A non-transitory machine-readable medium for performing operations including:

19. The operation further comprises: Receiving a second image; and providing said second image as an input to said descriptor network; generating a second image descriptor based on the second image using the descriptor network, the second image descriptor comprising: a second major vector comprising a first subset of the second set of elements; a second sub-vector comprising a second subset of the second set of elements, the second subset of the second set of elements including more elements than the first subset of the second set of elements; a second set of elements distributed between normalizing said second principal vector to said principal normalization quantity; normalizing the second sub-vector to the sub-normalized quantity; applying the hierarchical normalization to the second image descriptor by 20. The non-transitory machine-readable medium of claim 18, comprising:

20. The operation further comprises: calculating a principal distance between the first image and the second image based on the first principal vector and the second principal vector; determining whether the primary distance exceeds an upper threshold; determining whether the first image matches the second image by 20. The non-transitory machine-readable medium of claim 19, comprising:

Images