Image Processing Device

Abstract

To solve the problem in which acceleration is hard to achieve and criteria to evaluate similarity between images is required to be determined in advance.SOLUTION: An image processing device for detecting a partial image similar to a query image from an input image comprises a neural network and processing means. The neural network is machine-learned using training data so as to output an estimated position of the partial image similar to the query image within the input image when the query image and the input image having the partial image similar to the query image in association with each other are inputted. The processing means inputs, into the learned neural network, the input image and the query image in association with each other, and calculates the position of the partial image similar to the query image within the input image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Template matching is a method for searching for a partial image similar to a smaller query image from a larger input image. In template matching, the query image is slid from the top left to the bottom right of the input image, and the similarity between the query image and the image of the part overlapping the query image is calculated.

[0003] Meanwhile, in the field of general object detection, a technique has been proposed or put into practical use that uses a Convolutional Neural Network (CNN) to detect specific classes such as people and cars from an image. For example, Patent Document 1 proposes a method of identifying all local regions in an image of interest by using a CNN while sliding the local region in small steps. In addition, a deep CNN has been proposed or put into practical use that simultaneously performs detection to find object-like regions in an image and classification to determine the class of the found regions. YOLO (You Look Only Once) and SSD (Single Shot MultiBox Detector) are examples of such deep CNNs. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-160921 A Summary of the Invention [Problem to be solved by the invention]

[0005] The above-mentioned template matching method is difficult to speed up, and a scale for evaluating the similarity between images must be determined in advance. The technology described in Patent Document 1 and deep CNNs such as YOLO and SSD can be used to speed up the process up to object detection. However, it is necessary to separately check whether the detected object is similar to the object in the query image. For example, when checking whether a specific person in the query image exists in an input image containing many people, it is necessary to calculate the similarity between the image of each person detected using the technology described in Patent Document 1 or deep CNNs such as YOLO and SSD and the image of the person in the query image to check whether the person images are images of the same person. Therefore, a scale for evaluating the similarity between images is still necessary, and it is difficult to speed up the process up to obtaining the final result.

[0006] An object of the present invention is to provide an image processing device that solves the above-mentioned problems, that is, the problems that it is difficult to increase the speed and that it is necessary to determine in advance a scale for evaluating the similarity between images. [Means for solving the problem]

[0007] An image processing device according to one aspect of the present invention includes: An image processing device that detects a partial image similar to a query image from an input image, a neural network that has been machine-learned using training data so as to output an estimated position of the partial image similar to the query image in an input image when a query image and an input image having a partial image similar to the query image are input in association with each other; a processing means for inputting an input image and a query image in association with each other to the trained neural network, and calculating a position of a partial image in the input image that is similar to the query image; The device is configured to include:

[0008] Further, an image processing method according to an aspect of the present invention includes: An image processing method for detecting a partial image similar to a query image from an input image, comprising: performing machine learning of a neural network using training data so as to output an estimated position of the partial image similar to the query image in the input image when a query image and an input image having a partial image similar to the query image are input in association with each other; An input image and a query image are input to the trained neural network in association with each other, and a position of a partial image similar to the query image in the input image is calculated. It is structured as follows.

[0009] In addition, a program according to one embodiment of the present invention includes: A computer detects a partial image similar to a query image from an input image. A process of performing machine learning of a neural network using training data so as to output an estimated position of a partial image similar to the query image in an input image when a query image and an input image having a partial image similar to the query image are input in association with each other; A process of inputting an input image and a query image in association with each other to the trained neural network, and calculating a position of a partial image in the input image that is similar to the query image; The present invention is configured to cause the Effect of the Invention

[0010] By having the above-described configuration, the present invention can quickly search for partial images similar to a query image from within an input image, and also eliminates the need to predetermine a scale for evaluating the similarity between images. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram of an image processing apparatus according to a first embodiment of the present invention. [Diagram 2] FIG. 11 is a block diagram of an image processing device according to a second embodiment of the present invention. [Diagram 3] FIG. 11 is a schematic diagram showing an overview of machine learning processing of a neural network in a second embodiment of the present invention. [Figure 4] 10 is a flowchart showing an example of the operation of a processing unit in an image processing device according to a second embodiment of the present invention. [Diagram 5] FIG. 11 is a diagram illustrating an example of the configuration of a neural network in an image processing device according to a second embodiment of the present invention. [Figure 6] FIG. 11 is a schematic diagram showing the connection relationship between a first convolution layer, a second convolution layer, and a third convolution layer of a neural network in an image processing device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Next, an embodiment of the present invention will be described in detail with reference to the drawings. [First embodiment] First, an image processing apparatus according to a first embodiment of the present invention will be described. In this embodiment, an overview of the image processing apparatus of the present invention will be described.

[0013] 1 is a block diagram of an image processing device 10 according to a first embodiment of the present invention. The image processing device 10 according to this embodiment is configured to detect a partial image similar to a query image from an input image. The image processing device 10 includes a trained neural network 11 and a processing means 12.

[0014] The neural network 11 undergoes machine learning processing using training data so that when a query image and an input image having a partial image similar to the query image are input in association with each other, the neural network 11 outputs an estimated position in the input image of the partial image similar to the query image.

[0015] The processing means 12 inputs the input image and the query image to the neural network 11 in association with each other, and calculates the position in the input image of a partial image similar to the query image.

[0016] In this manner, in the image processing device 10 according to the present embodiment, the neural network 11 is subjected to machine learning processing using teacher data so as to output an estimated position of the partial image similar to the query image in the input image when a query image and an input image having a partial image similar to the query image are input in association with each other. Then, the processing means 12 inputs the input image and the query image in association with each other to the neural network 11, and calculates the position of the partial image similar to the query image in the input image. Therefore, according to the image processing device 10, it is possible to quickly search for a partial image similar to the query image from within the input image, and it is not necessary to determine in advance a scale for evaluating the similarity between images.

[0017] [Second embodiment] Next, an image processing device according to a second embodiment of the present invention will be described. In this embodiment, the image processing device of the present invention will be specifically described.

[0018] 2 is a block diagram of an image processing device 20 according to a second embodiment of the present invention. The image processing device 20 according to this embodiment is configured to detect whether or not a specific person appearing in a query image is present in an input image in which many people are appearing. The image processing device 20 includes an image server 21, a communication interface (I / F) unit 22, an operation input unit 23, a screen display unit 24, a storage unit 25, and a calculation processing unit 26.

[0019] The camera 27 is an imaging device installed on the street or indoors where many people pass by for the purpose of crime prevention, surveillance, etc. The camera 27 may capture the same or different imaging areas from a fixed location in a fixed imaging direction. Alternatively, the camera 27 may be mounted on a moving body such as a car and capture the same or different imaging areas while moving. The number of cameras 27 is arbitrary. There may be one camera 27, or there may be two or more cameras 27 that capture different imaging areas. The camera 27 may be, for example, a color camera equipped with a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor having a pixel capacity of about several million pixels and capturing images continuously at a constant frame rate.

[0020] Image server 21 is connected to camera 27 by wire or wirelessly, and accumulates video (video) consisting of time-series images captured by camera 27 for a certain period of time. Image server 21 accumulates images captured by camera 27 in units of frame images, for example. Image server 21 also accumulates each frame image in association with a camera ID that uniquely identifies camera 27 that acquired the frame image and the capture time.

[0021] The communication I / F unit 22 is composed of a data communication circuit, and is configured to perform data communication with an external device wirelessly or by wire. The operation input unit 23 is composed of devices such as a keyboard and a mouse, and is configured to detect an operator's operation and output it to the arithmetic processing unit 26. The screen display unit 24 is composed of devices such as an LCD (Liquid Crystal Display), and is configured to display various information on the screen in response to instructions from the arithmetic processing unit 26.

[0022] The storage unit 25 is composed of storage devices such as a hard disk and a memory, and is configured to store processing information and a program 251 required for various processes in the arithmetic processing unit 26. The program 251 is a program that is read into the arithmetic processing unit 26 and executed to realize various processing units, and is read in advance from an external device or recording medium (not shown) via a data input / output function such as the communication I / F unit 22 and stored in the storage unit 25. The main processing information stored in the storage unit 25 includes an image DB (database) 252 and a neural network 253.

[0023] The image DB 252 is a database that stores images of the camera 27 acquired through the image server 21. The neural network 253 is configured in the same manner as the neural network 11 described with reference to FIG.

[0024] The arithmetic processing unit 26 has a processor such as an MPU and its peripheral circuits, and is configured to read and execute a program 251 from the storage unit 25 to realize various processing units by making the above hardware and the program 251 work together. The main processing units realized by the arithmetic processing unit 26 are an image acquisition unit 261, a learning unit 262, and a processing unit 263.

[0025] The image acquisition means 261 is configured to acquire images captured by the camera 27 from the image server 21 and store them in the image DB 252 .

[0026] The learning means 262 is a means for performing machine learning on the neural network 253. The learning means 262 performs machine learning processing on the neural network 253 using teacher data so as to output an estimated position in the input image of the partial image similar to the query image when a query image and an input image having a partial image similar to the query image are input in association with each other.

[0027] The processing means 263 is configured similarly to the processing means 12 described with reference to FIG.

[0028] Next, the operation of the image processing apparatus 20 will be described. The operation of the image processing apparatus 20 is roughly classified into an image collection operation, a learning operation, and a search operation. The image collection operation is executed by the image acquisition means 261. The learning operation is executed by the learning means 262. The search operation is executed by the processing means 263. Each operation will be described below.

[0029] <Image collection operation> The image acquisition means 261 periodically acquires the latest image from the image server 21 that stores the images captured by the camera 27, and performs a process of adding it to the image DB 252. The above process by the image acquisition means 261 is executed, for example, before the learning operation and the search operation are started, and is continuously executed during the learning operation and the search operation.

[0030] <Learning operation> The learning means 262 has a function of generating teacher data for learning and a function of performing machine learning on the neural network 253 using the generated teacher data.

[0031] In generating the teacher data, the learning means 262 creates a number of teacher data using, for example, a part of the images stored in the image DB 252. One teacher data is composed of one frame image of the camera 27 in which one or more persons are shown (hereinafter referred to as a large image), one image in which one person is shown (hereinafter referred to as a small image), and rectangular information representing the position on the large image of the person who matches the person shown in the small image. Here, the size (number of pixels) of the large image is set to a fixed value of H pixels in the vertical direction and W pixels in the horizontal direction. Also, the size (number of pixels) of the small image is set to a fixed value of h pixels in the vertical direction and w pixels in the horizontal direction. h < H and w < W. The rectangular information may be, for example, the coordinate values (pixel positions) of the four vertices of the circumscribed rectangle of the matching person. The learning means 262 may create the teacher data as described above by interactive processing via the operation input unit 23 and the screen display unit 24. Also, the learning means 262 may create the teacher data using various annotation tools. In addition to creating teacher data from the images stored in the image DB 252, teacher data created at another location may be used.

[0032] In the machine learning of the neural network 253, as shown in the schematic diagram of FIG. 3, the learning means 262 inputs the large image and the small image constituting the teacher data to the neural network 253 in association with each other ((1) in FIG. 3). When the large image and the small image are input in association with each other, the neural network 253 outputs the estimated position in the large image of a partial image similar to the small image and its probability (score) ((2) in FIG. 3). When the actual output is obtained from the neural network 253, the learning means 262 compares it with the rectangular information constituting the teacher data and evaluates how different the actual output is by an error function ((3) in FIG. 3). Then, the learning means 262 updates the weight values ​​of the neural network 253 in a direction to reduce the error function by using, for example, the backpropagation method ((4) in FIG. 3). The learning means 262 repeats the above process using a lot of teacher data, for example, until the error function becomes sufficiently small.

[0033] <Search behavior> FIG. 4 is a flowchart showing an example of the operation of the processing means 263. Referring to FIG. 4, the processing means 263 first acquires a query image (step S1). For example, the processing means 263 may acquire, as a query image, a partial image cut out with specified rectangular information from an image that is uniquely determined by a camera ID and an image capture time specified by an operator or an external device among images stored in the image DB 252. Alternatively, the processing means 263 may directly acquire a query image from an operator through the operation input unit 23 or the screen display unit 24. Alternatively, the processing means 263 may acquire a query image from an external device through the communication I / F unit 22. Here, the size of the query image may be the same as the size of the small image that constitutes the teacher data.

[0034] Next, the processing means 263 acquires an input image (step S2). For example, the processing means 263 may acquire, as the input image, an image that is uniquely determined by a camera ID and an image capture time specified by an operator or an external device, from among the images stored in the image DB 252. Alternatively, the processing means 263 may acquire the input image directly from the operator through the operation input unit 23 or the screen display unit 24. Alternatively, the processing means 263 may acquire the input image from an external device through the communication I / F unit 22. Here, the size of the input image may be the same as the size of the large image that constitutes the teacher data.

[0035] Next, the processing means 263 uses the trained neural network 253 to calculate the position of the partial image similar to the query image in the input image (step S3). Specifically, the processing means 263 first inputs the query image and the input image to the trained neural network 253 in association with each other, and obtains one or more pairs of the estimated position and score of the partial image similar to the query image in the input image output from the neural network 253. Next, the processing means 263 judges whether or not the maximum score of the one or more pairs is equal to or greater than a predetermined threshold. Next, if the maximum score is not equal to or greater than the threshold, the processing means 263 generates a search result indicating that the input image does not include a partial image similar to the query image. Furthermore, if the maximum score is equal to or greater than the threshold, the processing means 263 generates the estimated position and score related to the maximum score as search results.

[0036] Next, the processing means 263 displays the generated search results on the screen display unit 24 and / or outputs them to an external device via the communication I / F unit 22 (step S4).

[0037] Next, a configuration example of the neural network 253 will be described.

[0038] 5 is a diagram showing an example of the configuration of the neural network 253. The neural network 253 in this example includes a first input layer 31, a second input layer 32, a first convolution layer 33, a second convolution layer 34, a third convolution layer 35, an intermediate layer 36, and an output layer 37.

[0039] The first input layer 31 and the second input layer 32 constitute the input layer of the neural network 253. Of these, the first input layer 31 is a layer that propagates pixel information of an input image (large image) directly to the first convolution layer 33, and is composed of neurons whose number corresponds to the size of the input image. The second input layer 32 is a layer that propagates pixel information of a query image (small image) directly to the second convolution layer 34, and is composed of neurons whose number corresponds to the size of the query image.

[0040] The first convolution layer 33 and the second convolution layer 34 constitute the first intermediate layer of the neural network 253. Among them, the first convolution layer 33 is a layer that extracts features (feature map) of the input image input to the first input layer 31. The first convolution layer 33 is composed of neurons in a number corresponding to the size of the input image and the size of the local area (local receptive field) where convolution is performed. The size of the local area where convolution is performed may be, for example, 3×3 neurons, but is not limited thereto. In addition, the output value of each neuron of the first convolution layer 33 is calculated based on the result of convolution calculation of the output values ​​of multiple neurons of the corresponding local area of ​​the first input layer 31 and 3×3 weights (weight matrix) for the first convolution layer 33, and the bias and activation function for the first convolution layer 33.

[0041] The second convolution layer 34 is a layer that extracts features (feature map) of the query image input to the second input layer 32. The second convolution layer 34 is composed of neurons whose number corresponds to the size of the query image and the size of the local region in which convolution is performed. The size of the local region in which convolution is performed may be, for example, 3×3 neurons, but is not limited thereto. The output value of each neuron in the second convolution layer 34 is calculated based on the result of convolution calculation of the output values ​​of multiple neurons in the corresponding local region of the second input layer 32 and 3×3 weights (weight matrix) for the second convolution layer 34, and the bias and activation function for the second convolution layer 34.

[0042] The third convolution layer 35 is a layer that extracts features obtained by combining the features of the input image extracted by the first convolution layer 33 and the features of the query image extracted by the second convolution layer 34. Specifically, the third convolution layer 35 is composed of a number of neurons according to the size of the first convolution layer 33 and the size of the local region of the first convolution layer 33 where convolution is performed. The size of the local region where convolution is performed may be, for example, 3×3 neurons, but is not limited thereto. In addition, the second convolution layer 34 and the third convolution layer 35 are fully connected. Therefore, the output value of each neuron in the third convolution layer 35 is calculated based on the result of a convolution operation between the output values ​​of multiple neurons in the corresponding local region of the first convolution layer 33 and 3 × 3 weights (first weight matrix) for the third convolution layer 35, the result of a convolution operation between the output values ​​of all neurons in the second convolution layer 34 and 3 × 2 weights (second weight matrix) for the third convolution layer 35, and the bias and activation function for the third convolution layer 35.

[0043] 6 is a schematic diagram showing the connection relationship between the first convolution layer 33, the second convolution layer 34, and the third convolution layer 35. For convenience of explanation, the size of the first convolution layer 33 is 9 neurons vertically by 6 neurons horizontally, the size of the second convolution layer 34 is 3 neurons vertically by 2 neurons horizontally, and the size of the third convolution layer 35 is 7 neurons vertically by 4 neurons horizontally. Then, for example, the output of a 3×3 neuron having the second neuron 352 from the upper left corner of the first convolution layer 33 as its upper left vertex and the output of all the neurons of the second convolution layer 34, that is, the 3×2 neuron, are input to the second neuron 351 from the upper left corner of the third convolution layer 35. Neuron 351 calculates the result of a convolution operation between the output of the 3×3 neuron in the first convolution layer 33 and 3×3 weights (first weight matrix), and the result of a convolution operation between the output of the 3×2 neuron in the second convolution layer 34 and 3×2 weights (second weight matrix). Neuron 351 also calculates an output value of neuron 351 based on the sum of the two convolution operation results, a predetermined bias, and an activation function.

[0044] Here, all the weights mentioned above can be positive or negative.

[0045] Referring again to FIG. 5, the third convolutional layer 35 is followed by other intermediate layers 36, and the final stage is an output layer 37. That is, the output value of each neuron of the third convolutional layer 35 becomes the input value of the intermediate layer 36. The intermediate layer 36 and the output layer 37 function to estimate and output the position of a partial image similar to the query image in the input image based on a feature obtained by combining the features of the input image extracted by the third convolutional layer 35 and the features of the query image. The intermediate / output layer 38 composed of the intermediate layer 36 and the output layer 37 may be composed of a deep CNN that simultaneously performs, for example, detection to find an object-like region in an image and classification to determine the class of the found region. Examples of such deep CNN include YOLO and SSD. When YOLO or SSD is used, the output value of each neuron of the third convolutional layer 35 is treated as an input value of the input layer of YOLO or SSD.

[0046] The above is an example of the configuration of the neural network 253. However, the neural network 253 is not limited to the example of the configuration described above. For example, a fourth convolution layer may be used instead of the first convolution layer 33, the second convolution layer 34, and the third convolution layer 35. This fourth convolution layer is composed of neurons in a number corresponding to the size of the first input layer 31 and the size of the local area (e.g., 3×3 neurons) to be convolved. In addition, the second input layer 32 and the fourth convolution layer are fully connected. Therefore, the output value of each neuron of the fourth convolution layer is calculated based on the result of convolution calculation of the output values ​​of multiple neurons of the corresponding local area of ​​the first input layer 31 and the weights for the local area for the fourth convolution layer, the result of convolution calculation of the output values ​​of all neurons of the second input layer 32 and the weights for the neurons of the second input layer 32 for the fourth convolution layer, and the bias and activation function for the fourth convolution layer. Alternatively, at least one of the first convolutional layer 33, the second convolutional layer 34, and the third convolutional layer 35 of the neural network 253 may be composed of two or more convolutional layers.

[0047] As described above, in the image processing device 20 according to this embodiment, when a query image and an input image having a partial image similar to the query image are input in association with each other, the neural network 253 performs machine learning processing using teacher data by the learning means 262 so as to output an estimated position of the partial image similar to the query image in the input image. Then, the processing means 263 inputs the input image and the query image in association with each other to the trained neural network 253, and calculates the position of the partial image similar to the query image in the input image. Therefore, according to the image processing device 20, it is possible to quickly search for a partial image similar to the query image from the input image, and it is not necessary to determine in advance a scale for evaluating the similarity between images.

[0048] Moreover, the neural network 253 in this embodiment has a convolution layer (the first convolution layer 33, the second convolution layer 34, and the third convolution layer 35, or the fourth convolution layer) that extracts a composite feature obtained by combining the feature of the input image input to the first input layer 31 and the feature input to the second input layer 32. Therefore, the neural network 253 can extract features for each local region of the input image in terms of similarity with the query image. As a result, improved accuracy can be expected compared to a neural network that does not have a convolution layer that extracts such a composite feature.

[0049] Although the present invention has been described above with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. For example, in the above-mentioned embodiment, the present invention is applied to an image processing device that detects a partial image similar to a query image in which a specific person appears from an input image in which multiple people appear, but is not limited thereto. For example, the present invention may be applied to an image processing device that detects a partial image similar to a query image in which a specific object other than a person appears from an input image in which multiple objects other than a person (e.g., a car, a ship, a dog, etc.) appear. [Industrial Applicability]

[0050] The present invention is applicable to the field of detecting a specific object from an image, and in particular to the field of detecting a partial image similar to a smaller-sized image from a larger-sized image. [Explanation of symbols]

[0051] 10 Image processing device 11 Neural Networks 12 Processing means 20 Image processing device 21 Image Server 22 Communication I / F section 23 Operation input section 24 Screen display section 25 Memory section 26 Processing unit 31 First input layer 32 Second input layer 33 First convolution layer 34 Second convolution layer 35 3rd convolution layer 36 Middle Class 37 Output layer 38 Intermediate and output layers 251 Programs 252 Image DB 253 Neural Networks 261 Image Acquisition Method 262 Learning Methods 263 Processing Means 351, 352 Neuron

Claims

1. An image processing device that detects a partial image similar to a query image from an input image, a neural network that has been machine-learned using training data so as to output an estimated position of the partial image similar to the query image in an input image when a query image and an input image having a partial image similar to the query image are input in association with each other; a processing means for inputting an input image and a query image in association with each other to the trained neural network, and calculating a position of a partial image in the input image that is similar to the query image; Equipped with The neural network includes a first input layer to which the input image is input, a second input layer to which the query image is input, a convolution layer that extracts a composite feature obtained by combining a feature of the input image input to the first input layer with a feature input to the second input layer, and an intermediate layer and an output layer that calculate and output the estimated position based on the composite feature extracted by the convolution layer. Image processing device.

2. Each neuron of the convolution layer is configured to input an output value of a neuron in a local region of the first input layer and an output value of all neurons of the second input layer. The image processing device according to claim 1 .

3. The intermediate layer and the output layer are composed of a deep convolutional neural network that simultaneously detects object-like regions in an image and classifies the detected regions.

3. The image processing device according to claim 1 or 2.

4. The intermediate layer and the output layer are configured by YOLO (You Only Look Once).

4. An image processing device according to claim 1.

5. The intermediate layer and the output layer are composed of SSDs (Single Shot Multibox Detectors).

4. An image processing device according to claim 1.

6. An image processing method for detecting a partial image similar to a query image from an input image, comprising: performing machine learning of a neural network including a first input layer to which an input image is input, a second input layer to which the query image is input, a convolution layer that extracts a composite feature obtained by combining a feature of the input image input to the first input layer with a feature input to the second input layer, and an intermediate layer and an output layer that calculate and output the estimated position based on the composite feature extracted by the convolution layer, when a query image and an input image having a partial image similar to the query image are input in association with each other, using teacher data to output an estimated position of the partial image similar to the query image in the input image; An input image and a query image are input to the trained neural network in association with each other, and a position of a partial image similar to the query image in the input image is calculated. Image processing methods.

7. A computer detects a partial image similar to a query image from an input image. a process of performing machine learning on a neural network, the process including: a first input layer to which an input image is input using teacher data; a second input layer to which the query image is input; a convolution layer that extracts a composite feature obtained by combining a feature of the input image input to the first input layer with a feature input to the second input layer; and an intermediate layer and an output layer that calculate and output the estimated position based on the composite feature extracted by the convolution layer, so as to output an estimated position in the input image of a partial image similar to the query image when the query image and the input image having the partial image similar to the query image are input in association with each other; A process of inputting an input image and a query image in association with each other to the trained neural network, and calculating a position of a partial image in the input image that is similar to the query image; A program to carry out the above.

Images