Multi-stage cross-domain image retrieval method and system based on freehand sketch driving

Abstract

The invention provides a multi-stage cross-domain image retrieval method and system based on freehand sketch driving, and the method comprises the steps: a first stage is a freehand sketch conversion stage, a generation model is trained by using a freehand sketch and a white ground natural image, and a natural image of a freehand sketch to be queried is generated based on the generation model; the second stage is a feature extraction stage, and features of the generated natural image and a real natural image in a database are extracted; the final stage is a similarity calculation stage, similarity calculation is carried out on the generated natural image and the characteristics of the real natural image in the database, and the real natural image with the highest similarity of the freehand sketch to be inquired in the database can be obtained. According to the method, the free-hand sketch is firstly used for generating the natural image, and then the image retrieval is performed, so that the domain gap between the free-hand sketch and the natural image is reduced, and the retrieval precision of the image retrieval is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and more specifically, to a multi-stage cross-domain image retrieval method and system driven by hand-drawn sketches. Background Art

[0002] With the rapid popularization of touch-screen devices such as tablets, users can easily draw hand-drawn sketches. The problems related to hand-drawn sketches in the field of computer vision have also made rapid progress. Among them, image retrieval based on hand-drawn sketches has received a lot of attention. Sketch image retrieval is an advanced technology that searches the user's hand-drawn sketches against natural images in the database to find tasks corresponding to the category of hand-drawn sketches. Sketch image retrieval is more intuitive and can meet users' needs for visual information.

[0003] However, sketch image retrieval is a challenging task. Sketch image retrieval is to find images of the same category as the hand-drawn sketch given by the user in an existing database. This task is easy for humans but difficult for computers. First, due to the simplicity and abstractness of hand-drawn sketches, semantic differences may lead to understanding bias. Second, hand-drawn sketches usually lack details and contextual information, which may lead to the loss of key features during the retrieval process, affecting the accuracy of retrieval. In addition, the drawing quality and style differences of hand-drawn sketches will affect the retrieval results, because the sketches drawn by different users may be different. Some users use simple, sloppy lines to draw sketches, while some users can show the details and texture of the sketches when drawing sketches. This increases the complexity of matching. These shortcomings together limit the practical application of hand-drawn sketch image retrieval.

[0004] In order to solve this problem, people have conducted a lot of research. Some studies have tried to bridge the domain gap between hand-drawn sketches and natural images. A popular method is to build an excellent intermediate representation to convert natural images into edge images. The main drawback is that the domain difference between hand-drawn sketches and natural images is not easy to make up, because edge images are difficult to match to non-aligned hand-drawn sketches with large changes and ambiguity, and the retrieval is difficult. Or use a generative model to convert hand-drawn sketches into photo domains. One disadvantage of this method is that the hand-drawn sketches and natural images required for the training set are mapped one to one. Another widely used method is to learn an end-to-end semi-heterogeneous network. Convolutional neural networks are used in an end-to-end framework to learn domain transformation features from hand-drawn sketches and natural images. Although progress has been made, there are still some challenges. It is difficult for general convolutional neural networks to extract accurate and rich features, and some prior knowledge of samples cannot be well utilized, resulting in poor retrieval accuracy.

[0005] Obviously, the current direct use of hand-drawn sketches and natural images for retrieval can no longer meet the needs of practical applications. It is necessary to propose a new model that can reduce the gap between hand-drawn sketches and natural image domains and achieve high-precision sketch image retrieval. Summary of the invention

[0006] In view of the technical problems existing in the prior art, the present invention provides a multi-stage cross-domain image retrieval method and system driven by hand-drawn sketches, which is used to solve the problem of low retrieval accuracy and poor retrieval effect in existing sketch image retrieval tasks that directly use sketches and natural images for retrieval.

[0007] According to a first aspect of the present invention, a multi-stage cross-domain image retrieval method driven by hand-drawn sketches is provided, comprising:

[0008] Inputting the hand-drawn sketch to be queried into a generative model, and converting the hand-drawn sketch to be queried into a corresponding first natural image based on the generative model, wherein the generative model is trained based on a training set of hand-drawn sketches and corresponding natural images with white background;

[0009] Performing feature extraction on the first natural image and each second natural image in the database to obtain feature vectors of the first natural image and each second natural image respectively;

[0010] Calculating the similarity between the feature vector of the first natural image and the feature vector of each of the second natural images;

[0011] The second natural image having the greatest similarity to the first natural image is used as a query result corresponding to the hand-drawn sketch to be queried.

[0012] According to a second aspect of the present invention, a multi-stage cross-domain image retrieval system driven by hand-drawn sketches is provided, comprising:

[0013] a conversion module, configured to input the hand-drawn sketch to be queried into a generation model, and convert the hand-drawn sketch to be queried into a corresponding first natural image based on the generation model, wherein the generation model is trained based on a training set of hand-drawn sketches and corresponding natural images with a white background;

[0014] a feature extraction module, configured to extract features from the first natural image and each second natural image in the database, and obtain feature vectors of the first natural image and each second natural image respectively;

[0015] The query module is used to calculate the similarity between the feature vector of the first natural image and the feature vector of each of the second natural images; and use the second natural image with the greatest similarity to the first natural image as the query result corresponding to the hand-drawn sketch to be queried.

[0016] The present invention provides a multi-stage cross-domain image retrieval method and system based on hand-drawn sketch drive. In the hand-drawn sketch conversion stage, the hand-drawn sketch is converted into a natural image; in the feature extraction stage, the converted natural image and the real natural image in the database are subjected to feature extraction and normalization; in the similarity calculation stage, the similarity between the normalized vectors is calculated. Through the comprehensive use of these three stages, the multi-stage cross-domain image retrieval method of the present invention can effectively solve the problem of retrieval difficulties caused by the domain gap between hand-drawn sketches and natural images, and improve the retrieval accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a multi-stage cross-domain image retrieval method driven by hand-drawn sketches provided by an embodiment of the present invention;

[0018] Figure 2 A model architecture diagram of a multi-stage cross-domain image retrieval method driven by hand-drawn sketches provided in one embodiment of the present invention;

[0019] Figure 3 A schematic structural diagram of a hand-drawn sketch-driven multi-stage cross-domain image retrieval system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be arbitrarily combined with each other to form a feasible technical solution. This combination is not subject to the constraints of the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in this field to achieve. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0021] Sketch image retrieval is to extract the features of sketches and natural images, calculate the similarity between the features, and judge whether the two belong to corresponding images or images of the same category. Due to the simplicity and abstractness of hand-drawn sketches, semantic differences may lead to understanding deviations. Secondly, hand-drawn sketches usually lack details and contextual information, which may lead to the loss of key features during the retrieval process, affecting the accuracy of retrieval. In addition, the drawing quality and style differences of hand-drawn sketches will affect the retrieval results, because the sketches drawn by different users may be different. Some users use simple, sloppy lines to draw sketches, and some users can show the details and texture of the sketches when drawing sketches. This increases the complexity of matching. Therefore, it is necessary to propose a multi-stage cross-domain image retrieval method driven by hand-drawn sketches.

[0022] Figure 1 The present invention provides a flowchart of a multi-stage cross-domain image retrieval method driven by hand-drawn sketches, such as Figure 1 As shown, the method includes:

[0023] Step 1: input the hand-drawn sketch to be queried into a generation model, and convert the hand-drawn sketch to be queried into a corresponding first natural image based on the generation model, wherein the generation model is trained based on a training set of hand-drawn sketches and corresponding natural images with a white background.

[0024] It is understandable that some hand-drawn sketches and corresponding natural images with white background are collected as training sets, and a generative model is trained based on the training sets. Subsequently, as long as the hand-drawn sketches are input into the generative model, the natural images corresponding to the hand-drawn sketches can be generated. To distinguish them from the real natural images in the database, the natural images generated by the generative model are called the first natural images, and the real natural images in the database are called the second natural images.

[0025] The training of the generative model is to learn the mapping function between the two domains of the training sample hand-drawn sketch dataset X and the white background natural image dataset Y: G:X→Y and F:Y→X, where G represents the mapping function from the hand-drawn sketch dataset X to the white background natural image dataset Y, and F represents the mapping function from the white background natural image dataset Y to the hand-drawn sketch dataset X.

[0026] Among them, the training of the generative model includes: calculating the cycle-consistent adversarial loss between the first natural image of the hand-drawn sketch sample transformed by the generative model and the real natural image; based on the cycle-consistent adversarial loss, adjusting the parameters of the generative model, and continuously iterating the training until the first natural image generated by the generative model meets the objective function.

[0027] It can be understood that in the process of training the generative model, a cycle-consistent adversarial loss function is used to calculate the loss between the natural images generated by the generative model and the real natural images.

[0028] Among them, the adversarial loss is expressed as:

[0029]

[0030] Where X is the hand-drawn sketch dataset domain: X = [x 1 ,x 2 ,…,x N ], N is the total number of hand-drawn sketch samples, and Y is the domain of the white background natural image dataset: Y = [y 1 ,y 2 ,…,y M ], M is the total number of natural images with white background, and the data distribution is represented by x~P data (x) and y~P data (y); G represents the mapping function from the hand-drawn sketch dataset to the white background natural image dataset, and F represents the mapping function from the white background natural image dataset to the hand-drawn sketch dataset.

[0031] D X and D Y is the adversarial discriminator, where D X Aims to distinguish between hand-drawn sketches {x} and generated natural images {F(y)}, D Y The goal of G is to distinguish {y} from {G(x)}; the goal of G is to minimize G(x) and the true natural image y, against the opponent D who tries to maximize it Y ,Right now For the mapping function F and its adversarial discriminator D X There are also

[0032] The cycle consistency loss is expressed as:

[0033]

[0034] In the formula, for each image x in domain X, it should be x→G(x)→F(G(x))≈x, and for each image y in domain Y, it should also satisfy y→F(y)→G(F(y))≈y.

[0035] The cycle-consistent adversarial loss is expressed as:

[0036]

[0037] Here, λ controls the relative importance of the two objectives.

[0038] Preferably, λ=10, and the history of generated images is used to update the discriminator instead of using the latest image generated by the generator. An image buffer is saved, which stores the 50 images created previously, which can reduce model oscillation.

[0039] Based on the loss calculated by the cycle-consistent adversarial loss function, the model parameters of the generative model are adjusted. By continuously calculating the loss of the generative model and adjusting the parameters of the generative model, training is continuously performed until the trained generative model meets the objective function, and the trained generative model is obtained.

[0040] Among them, the objective function is:

[0041]

[0042] After the trained generative model is obtained, the hand-drawn sketch to be queried can be input into the generative model and converted into a corresponding first natural image. The first natural image refers to the one generated by the generative model, not a real natural image.

[0043] By generating a hand-drawn sketch into a corresponding natural image through a generative model, the hand-drawn sketch can be converted into a natural image, which can greatly reduce the domain gap between the hand-drawn sketch and the natural image. Through the cycle-consistent adversarial loss, the hand-drawn sketch can lose fewer detailed features and transform fewer features in the process of generating natural images.

[0044] Step 2: extract features from the first natural image and each second natural image in the database to obtain feature vectors of the first natural image and each second natural image respectively.

[0045] It is understandable that after the hand-drawn sketch is converted into the first natural image through the generative model, the feature extraction stage is entered. Since it is necessary to find a second natural image similar to the hand-drawn sketch from the database, after the hand-drawn sketch is converted into the first natural image, the feature vector of the first natural image and the feature vector of each second natural image are extracted respectively.

[0046] See also Figure 2 In the present invention, the feature extraction stage uses a ConvNeXt network to extract and normalize the first natural image and each second natural image respectively. The network weights are trained using real natural images in the database and some additional real images as validation sets, so that the trained feature extraction network is more suitable for sketch image retrieval tasks.

[0047] Among them, during feature extraction, the first natural image and the second natural image are respectively input into the ConvNeXt network, a feature matrix is ​​first generated, and convolution, average pooling and other operations are performed on the feature matrix. Finally, a multi-dimensional feature vector of the natural image is obtained through a fully connected layer.

[0048] In the process of training the ConvNeXt network, the loss of the feature extraction model is calculated based on the cross entropy loss function, wherein the cross entropy loss function is expressed as:

[0049]

[0050] In the formula, y represents the true feature vector of the sample, Represents the predicted feature vector output by the feature extraction model. When y=-1, it means that the sample belongs to the positive class; when y=0, it means that the sample belongs to the negative class.

[0051] Step 3: Calculate the similarity between the feature vector of the first natural image and the feature vector of each of the second natural images.

[0052] It can be understood that, through step 3, the feature vectors of the first natural image and each second natural image in the database are extracted, and the similarity between the feature vector of the first natural image and the feature vector of each second natural image is calculated.

[0053] The cosine similarity between the feature vector of the first natural image and the feature vector of each second natural image is calculated as the similarity between the first natural image and each second natural image.

[0054] The feature vector of the first natural image and the feature vector of the second natural image are both multi-dimensional feature vectors.

[0055] Calculating the cosine similarity between the feature vector of the first natural image and the feature vector of each second natural image, including:

[0056]

[0057] In the formula, μ i and i are the eigenvectors of the S domain and the R domain respectively, n is the dimension of the eigenvector, i is the index of the dimension, and the first natural image domain generated by the hand-drawn sketch is: S = [s 1 ,s 2 ,…,s P ], P is the number of first natural images, the real second natural image domain in the database: R = [r 1 ,r 2 ,…,r Q ], Q is the number of the second natural images.

[0058] Step 4: taking the second natural image having the greatest similarity to the first natural image as the query result corresponding to the hand-drawn sketch to be queried.

[0059] It can be understood that the similarity between the first natural image corresponding to the hand-drawn sketch and each second natural image in the database is calculated, and the second natural image in the database with the greatest similarity to the first natural image is used as the natural image corresponding to the hand-drawn sketch to be queried, thereby obtaining the query result.

[0060] In the present invention, Figure 2 As shown, in the hand-drawn sketch conversion stage, the hand-drawn sketch is input, and the corresponding natural image is generated by the generative model. Then, in the feature extraction stage, the generated natural image is compared with the real natural image in the database for feature extraction and normalization. In the similarity calculation stage, the normalized feature vector is used for similarity calculation using cosine similarity, and finally the image most similar to the query image given by the user is obtained.

[0061] The multi-stage combination of the present invention can greatly reduce the domain gap between hand-drawn sketches and natural images, solve the style difference and ambiguity problems, and generate natural images that can retain the specific style and semantic information of the original hand-drawn sketches. It can not only improve the retrieval speed, but also greatly improve the overall retrieval accuracy. The overall accuracy and reliability of the model are high, which is convenient for practical application.

[0062] It should be understood that the sequence numbers of the steps in the above embodiments do not imply a sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0063] See also Figure 3 , is a schematic diagram of the structure of a multi-stage cross-domain image retrieval system driven by hand-drawn sketches provided by the present invention, the system comprising:

[0064] A conversion module 301 is used to input the hand-drawn sketch to be queried into a generation model, and convert the hand-drawn sketch to be queried into a corresponding first natural image based on the generation model, wherein the generation model is trained based on a training set of hand-drawn sketches and corresponding natural images with white background;

[0065] A feature extraction module 302, configured to extract features from the first natural image and each second natural image in the database, and obtain feature vectors of the first natural image and each second natural image respectively;

[0066] The query module 303 is used to calculate the similarity between the feature vector of the first natural image and the feature vector of each of the second natural images; and use the second natural image with the greatest similarity to the first natural image as the query result corresponding to the hand-drawn sketch to be queried.

[0067] It can be understood that the multi-stage cross-domain image retrieval system driven by hand-drawn sketches provided by the present invention corresponds to the multi-stage cross-domain image retrieval method driven by hand-drawn sketches provided in the aforementioned embodiments. The relevant technical features of the multi-stage cross-domain image retrieval system driven by hand-drawn sketches can refer to the relevant technical features of the multi-stage cross-domain image retrieval method driven by hand-drawn sketches, which will not be repeated here.

[0068] The embodiment of the present invention provides a multi-stage cross-domain image retrieval method and system based on hand-drawn sketch drive. In the hand-drawn sketch conversion stage, the hand-drawn sketch is converted into a natural image; in the feature extraction stage, the feature extraction and normalization are performed on the converted natural image and the real natural image in the database; in the similarity calculation stage, the similarity between the normalized vectors is calculated. By first generating a natural image from the hand-drawn sketch and then performing image retrieval, the domain gap between the hand-drawn sketch and the natural image is reduced, and the retrieval accuracy of the image retrieval is significantly improved.

[0069] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0070] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0072] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0074] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0075] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A multi-stage cross-domain image retrieval method driven by hand-drawn sketches, characterized in that: include: Inputting the hand-drawn sketch to be queried into a generative model, and converting the hand-drawn sketch to be queried into a corresponding first natural image based on the generative model, wherein the generative model is trained based on a training set of hand-drawn sketches and corresponding natural images with white background; Performing feature extraction on the first natural image and each second natural image in the database to obtain feature vectors of the first natural image and each second natural image respectively; Calculating the similarity between the feature vector of the first natural image and the feature vector of each of the second natural images; The second natural image having the greatest similarity to the first natural image is used as a query result corresponding to the hand-drawn sketch to be queried.

2. The multi-stage cross-domain image retrieval method based on hand-drawn sketch driving according to claim 1 is characterized in that: The training of the generative model includes: Calculate the cycle-consistent adversarial loss between the first natural image output by the generative model and the real natural image of the hand-drawn sketch sample; Based on the cycle-consistent adversarial loss, the parameters of the generative model are adjusted, and the training is continuously iterated until the first natural image generated by the generative model satisfies the objective function.

3. The multi-stage cross-domain image retrieval method based on hand-drawn sketch driving according to claim 2 is characterized in that: The calculating of the cycle-consistent adversarial loss between the first natural image output by the generative model and the real natural image of the hand-drawn sketch sample comprises: The adversarial loss is expressed as: Where X is the hand-drawn sketch dataset domain: X = [x 1 ,x 2 ,…,x N ], N is the total number of hand-drawn sketch samples, and Y is the domain of the white background natural image dataset: Y = [y 1 ,y 2 ,…,y M ], M is the total number of natural images with white background, and the data distribution is represented by x~P data (x) and y~P data (y); G represents the mapping function from the hand-drawn sketch dataset to the white background natural image dataset, and F represents the mapping function from the white background natural image dataset to the hand-drawn sketch dataset; D X and D Y is the adversarial discriminator, where D X Aims to distinguish hand-drawn sketches {} from generated natural images {(y)}, D Y The goal of G is to distinguish {} from {()}; the goal of G is to minimize G(x) and the real natural image y, against the opponent D who tries to maximize it Y ,Right now For the mapping function F and its adversarial discriminator D X There are also The cycle consistency loss is expressed as: In the formula, for each image x in domain X, it should be x→G(x)→F(())≈x, and for each image y in domain Y, it should also satisfy y→F(t)→G(())≈t; The cycle-consistent adversarial loss is expressed as: Here, λ controls the relative importance of the two objectives.

4. The multi-stage cross-domain image retrieval method based on hand-drawn sketch driving according to claim 3 is characterized in that: The objective function is:

5. The multi-stage cross-domain image retrieval method based on hand-drawn sketch driving according to claim 1 is characterized in that: Performing feature extraction on the first natural image and each second natural image in the database to obtain feature vectors of the first natural image and each second natural image respectively includes: Based on the feature extraction model, feature extraction and normalization are performed on the first natural image and each second natural image in the database to obtain feature vectors of the first natural image and each second natural image respectively; Wherein, the feature extraction model is trained based on the second natural image and other real natural images in the database.

6. The multi-stage cross-domain image retrieval method based on hand-drawn sketch driving according to claim 5 is characterized in that: During the training process, the loss of the feature extraction model is calculated based on the cross entropy loss function, wherein the cross entropy loss function is expressed as: In the formula, L is the cross entropy loss, y represents the true feature vector of the sample, Represents the predicted feature vector output by the feature extraction model. When t=-1, it means that the sample belongs to the positive class; when t=0, it means that the sample belongs to the negative class.

7. The multi-stage cross-domain image retrieval method based on hand-drawn sketch driving according to claim 1 is characterized in that: Calculating the similarity between the feature vector of the first natural image and the feature vector of each of the second natural images includes: The cosine similarity between the feature vector of the first natural image and the feature vector of each of the second natural images is calculated as the similarity between the first natural image and each of the second natural images.

8. The multi-stage cross-domain image retrieval method based on hand-drawn sketch driving according to claim 7 is characterized in that: The feature vector is multi-dimensional, and calculating the cosine similarity between the feature vector of the first natural image and the feature vector of each of the second natural images includes: In the formula, μ i and i are the eigenvectors of the S domain and the R domain respectively, n is the dimension of the eigenvector, i is the index of the dimension, and the first natural image domain generated by the hand-drawn sketch is: S = [s 1 ,s 2 ,…,s P ], P is the number of first natural images, the real second natural image domain in the database: R = [r 1 ,r 2 ,…,r Q ], Q is the number of the second natural images.

9. A multi-stage cross-domain image retrieval method driven by hand-drawn sketches, characterized in that: include: a conversion module, configured to input the hand-drawn sketch to be queried into a generation model, and convert the hand-drawn sketch to be queried into a corresponding first natural image based on the generation model, wherein the generation model is trained based on a training set of hand-drawn sketches and corresponding natural images with a white background; a feature extraction module, configured to extract features from the first natural image and each second natural image in the database, and obtain feature vectors of the first natural image and each second natural image respectively; The query module is used to calculate the similarity between the feature vector of the first natural image and the feature vector of each of the second natural images; and use the second natural image with the greatest similarity to the first natural image as the query result corresponding to the hand-drawn sketch to be queried.

Images