Reverse image retrieval based on deep neural network (DNN) model and image feature detection model

Abstract

An electronic device and method for reverse image searching is provided. The electronic device receives an image. The electronic device extracts a first set of image features associated with the image via a DNN model and generates a first feature vector based on the first set of image features. The electronic device extracts a second set of image features associated with the image via an image feature detection model and generates a second feature vector based on the second set of image features. The electronic device generates a third feature vector based on a combination of the first and second feature vectors. The electronic device determines a similarity metric between the third feature vector and a fourth feature vector of each image of the set of pre-stored images and identifies the pre-stored images based on the similarity metric. The electronic device controls a display device to display information associated with the pre-stored images.

Description

[Technical Field]

[0001] Cross-reference to related applications / incorporation by reference This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 189,956, filed May 18, 2021, the contents of which are incorporated herein by reference in their entirety.

[0002] Various embodiments of the present disclosure relate to reverse image searching, and more particularly, to electronic devices and methods for reverse image searching based on deep neural network (DNN) models and image feature detection models. [Background technology]

[0003] Advances in information and communication technology have led to a variety of internet-based image retrieval systems (e.g., web search engines). Traditionally, a user can upload an input image to a web search engine as a search query. In such cases, the web search engine can provide an output image set from the internet (using a reverse image search method). The output image set can be similar to the input image. Such a reverse image search method can employ a machine learning model to determine an output image set that is similar to the input image. In some cases, the machine learning model may misclassify one or more objects in the input image, resulting in the output image set including undesirable or irrelevant images. Summary of the Invention [Problem to be solved by the invention]

[0004] The limitations and disadvantages of conventional approaches will become apparent to those skilled in the art by comparing the described system with certain aspects of the present disclosure illustrated in the remainder of this application and with reference to the drawings. [Means for solving the problem]

[0005] Provided are electronic devices and methods for reverse image search based on deep neural network (DNN) models and image feature detection models substantially as shown and / or described in connection with at least one of the figures and more fully set forth in the claims.

[0006] These and other features and advantages of the present disclosure will become apparent from a consideration of the following detailed description of the disclosure when taken in conjunction with the accompanying drawings, in which like reference characters refer to like elements throughout. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary network environment for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a block diagram illustrating an exemplary electronic device for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to an embodiment of the present disclosure. [Figure 3] FIG. 1 illustrates exemplary operations for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to embodiments of the present disclosure. [Figure 4] 1 is a flowchart illustrating an exemplary method for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0008] The disclosed electronic device and method for performing reverse image search based on a deep neural network (DNN) model and an image feature detection model to improve the accuracy of the reverse image search can be implemented as described below. An exemplary aspect of the present disclosure provides an electronic device implementing a deep neural network (DNN) model and an image feature detection model for reverse image search. The electronic device can receive a first image (e.g., an image for which a user needs to search for similar images). The electronic device can extract a first set of image features associated with the received first image using the deep neural network (DNN) model and generate a first feature vector associated with the received first image based on the extracted first image feature set. The electronic device can extract a second set of image features associated with the received first image using the image feature detection model and generate a second feature vector associated with the received first image based on the extracted second image feature set. Examples of image feature detection models include, but are not limited to, a Scale-Invariant Feature Transform (SIFT)-based model, a Speeded-Up Robust Feature (SURF)-based model, an Oriented FAST and Rotated BRIEF (ORB)-based model, or a Fast Library for Approximate Nearest Neighbors (FLANN)-based model. The image feature detection model can extract image features, some of which may have been misdetected and / or misclassified by the DNN model 108.

[0009] The electronic device may further generate a third feature vector associated with the received first image based on a combination of the generated first feature vector and the generated second feature vector. In one example, generating the third feature vector may be further based on applying a principal component analysis (PCA) transform to the combination of the generated first feature vector and the generated second feature vector. The electronic device may further determine a similarity metric between the generated third feature vector associated with the received first image and a fourth feature vector of each image in a pre-stored second set of images (e.g., images stored in a database). Examples of similarity metrics may include, but are not limited to, cosine distance similarity or Euclidean distance similarity. The electronic device may further identify a pre-stored third image (e.g., an image identical or similar to the received first image) from the pre-stored second set of images based on the determined similarity metric and control a display device to display information associated with the identified pre-stored third image.

[0010] The disclosed electronic device can automatically generate a third feature vector associated with the received first image based on a combination of the generated first feature vector and the generated second feature vector. As a result, the third feature vector can include a first set of image features that can be determined by a DNN model and a second set of image features that can be determined by an image feature detection model. The first set of image features includes high-level image features associated with the received first image (e.g., facial features such as eyes, nose, ears, and hair), and the second set of image features includes low-level image features associated with the received first image (e.g., facial edges, lines, and contours). Including both the high-level and low-level image features in the third feature vector can complement each other in identifying similar images. For example, if the received first image is an image that is under-represented in the training dataset of the DNN model, the first set of image features may not be sufficient to identify images similar to the received first image from a pre-stored second set of images. However, since the second image feature set can include low-level image features related to the received first image, including the second image feature set in the third feature vector can improve the accuracy of identifying images similar to the received first image from the pre-stored second image set. On the other hand, if the image quality is poor (e.g., in the case of low-resolution, blurry images), the first image feature set (i.e., high-level image features) may not be sufficient to identify similar images. In such cases, the second image features (i.e., low-level image features) may be more useful and accurate for identifying similar images.

[0011] FIG. 1 is a block diagram illustrating an exemplary network environment for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to an embodiment of the present disclosure. FIG. 1 illustrates a network environment 100. The network environment 100 may include an electronic device 102, a server 104, and a database 106. Also shown is a deep neural network (DNN) model 108 and an image feature detection model 110 implemented on the server 104. As shown in FIG. 1, the database 106 may store a training dataset 112. The electronic device 102, the server 104, and the database 106 may be communicatively coupled to each other via a communications network 114. Also shown is a user 116 associated with the electronic device 102. While FIG. 1 illustrates the electronic device 102 and the server 104 as two separate devices, in some embodiments, the entire functionality of the server 104 may be incorporated into the electronic device 102 without departing from the scope of the present disclosure.

[0012] The electronic device 102 may include suitable logic, circuitry, interfaces, and / or code that may be configured to identify and display a set of images similar to a first image based on implementing the DNN model 108 and the image feature detection model 110 on the first image. Examples of the electronic device 102 may include, but are not limited to, an image search engine, a server, a personal computer, a laptop, a computer workstation, a mainframe machine, a gaming device, a virtual reality (VR) / augmented reality (AR) / mixed reality (MR) device, a smartphone, a mobile phone, a computing device, a tablet, and / or any consumer electronics (CE) device.

[0013] The DNN model 108 may be a deep convolutional neural network model that can be trained to detect a first set of image features in a first image based on an image feature detection task. The DNN model 108 may be defined by hyperparameters, such as activation function(s), the number of weights, a cost function, a regularization function, an input size, and the number of layers. The DNN model 108 may be referred to as a computational network or a system of artificial neurons (also called nodes). The nodes of the DNN model 108 may be arranged in multiple layers as defined by the neural network topology of the DNN model 108. The multiple layers of the DNN model 108 may include an input layer, one or more hidden layers, and an output layer. Each layer of the multiple layers may include one or more nodes (or artificial neurons). The output of every node in the input layer may be connected to at least one node in the hidden layer(s). Similarly, the input of each hidden layer may be connected to the output of at least one node in another layer of the DNN model 108. The output of each hidden layer may be coupled to the input of at least one node in another layer of the DNN model 108. The node(s) in the final layer may receive input from at least one hidden layer and output a result. The number of layers and the number of nodes in each layer may be determined from hyperparameters of the DNN model 108. Such hyperparameters may be set before or during training of the DNN model 108 based on the training dataset 112.

[0014] Each node of the DNN model 108 may correspond to a mathematical function (e.g., a sigmoid function or a rectified linear unit) having a set of parameters that can be adjusted during training of the network. The set of parameters may include, for example, weight parameters and regularization parameters. Each node may calculate an output using a mathematical function based on one or more inputs from nodes in other layer(s) (e.g., previous layer(s)) of the DNN model 108. All or some of the nodes of the DNN model 108 may correspond to the same or different mathematical functions.

[0015] In training the DNN model 108, one or more parameters of each node of the DNN model 108 may be updated based on whether the output of the final layer for a given input (from the training dataset) matches the correct result based on a loss function of the DNN model 108. The above process may be repeated for the same or different inputs until a minimum of the loss function is achieved, minimizing the training error. Several training methods are known in the art, including gradient descent, stochastic gradient descent, batch gradient descent, gradient boosting, and metaheuristic methods.

[0016] In some embodiments, the DNN model 108 may include electronic data that may be implemented, for example, as a software component of an application executable on the electronic device 102 or the server 104. The DNN model 108 may rely on libraries, external scripts, or other logic / instructions for execution by a processing device, such as the electronic device 102 or the server 110. The DNN model 108 may include computer-executable code or routines that enable a computing device, such as the electronic device 102 or the server 104, to perform one or more operations to detect image features in an input image. Additionally or alternatively, the DNN model 108 may be implemented using hardware, including a processor, a microprocessor (e.g., that performs or controls one or more operations), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). For example, the electronic device 102 (or the server 104) may include an inference accelerator chip that accelerates the computation of the DNN model 108 for image feature detection tasks. In some embodiments, the DNN model 108 may be implemented using a combination of both hardware and software. Examples of the DNN model 108 may include, but are not limited to, an artificial neural network (ANN), a convolutional neural network (CNN), Regions with CNN (R-CNN), Fast R-CNN, Faster R-CNN, You Only Look Once (YOLO) network, a residual neural network (Res-Net), a feature pyramid network (FPN), a retina net, a single-shot detector (SSD), and / or combinations thereof.

[0017] The image feature detection model 110 may be an image processing algorithm configured to extract image features associated with a first image. The image feature detection model 110 may be defined by hyperparameters, such as the number of image features, an edge threshold, the number of weights, a cost function, an input size, and the number of layers. The hyperparameters of the image feature detection model 110 may be adjusted to move toward a global minimum of the cost function of the image feature detection model 110, and the weights may be updated accordingly. The image feature detection model 110 may include electronic data that may be implemented, for example, as a software component of an application executable on the electronic device 102 or the server 104. The image feature detection model 110 may rely on libraries, external scripts, or other logic / instructions for execution by a processing device, such as the electronic device 102 or the server 104. The image feature detection model 110 may include code and routines configured to enable a computing device, such as the electronic device 102 or the server 104, to perform one or more operations, such as extracting a set of image features associated with the first image. Additionally or alternatively, the image feature detection model 110 may be implemented using hardware, including a processor, microprocessor (e.g., performing or controlling one or more operations), field programmable gate array (FPGA), or application specific integrated circuit (ASIC). Alternatively, in some embodiments, the image feature detection model 110 may be implemented using a combination of hardware and software. Examples of the image feature detection model 110 may include, but are not limited to, a scale invariant feature transform (SIFT)-based model, a speeded up robust features (SURF)-based model, an oriented FAST and rotational BRIEF (ORB)-based model, or a fast library for approximate nearest neighbors (FLANN)-based model.

[0018] The server 104 may include suitable logic, circuitry, interfaces, and / or code that may be configured to store the DNN model 108 and the image feature detection model 110. The server 104 may use the DNN model 108 to generate a first feature vector associated with a first image and the image feature detection model 110 to generate a second feature vector associated with the first image. The server 104 may further store machine learning models different from the DNN model 108 and the image feature detection model 110. The stored machine learning models may be configured to determine first weights associated with the generated first feature vector and second weights associated with the generated second feature vector. In an exemplary embodiment, the server 104 is implemented as a cloud server and may perform operations via web applications, cloud applications, HTTP requests, repository operations, file transfers, and the like. Other implementations of the server 104 may include, but are not limited to, a database server, a file server, a web server, an application server, a mainframe server, or a cloud computing server.

[0019] In at least one embodiment, server 104 may be implemented as a number of distributed cloud-based resources using techniques known to those skilled in the art. Those skilled in the art will appreciate that the scope of the present disclosure may not be limited to the implementation of server 104 and electronic device 102 as two separate entities. In some embodiments, the functionality of server 104 may be incorporated, in whole or at least in part, into electronic device 102 without departing from the scope of the present disclosure.

[0020] The database 106 may include suitable logic, interfaces, and / or code that may be configured to store a training dataset 112 for the DNN model 108. The training dataset 112 may include a pre-stored set of training images and a predetermined tag assigned to each image in the pre-stored training image set. The predetermined tag assigned to a particular training image may include a label corresponding to an image feature that may have been predetermined for the particular training image. The DNN model 108 may be pre-trained for an image feature detection task based on the training dataset 112. In some embodiments, the database 106 may be further configured to store a second pre-stored set of images. The database 106 may be a relational or non-relational database. In some cases, the database 106 may be stored on a server (e.g., server 104), such as a cloud server, or cached on the electronic device 102. Additionally or alternatively, the database 106 may be implemented using hardware, including a processor, a microprocessor (e.g., performing or controlling one or more operations), a field programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). In some other cases, the database 106 may be implemented using a combination of hardware and software.

[0021] The communication network 114 may include any communication medium that allows the electronic devices 102, the server 104, and the database 106 to communicate with one another. Examples of the communication network 114 may include, but are not limited to, the Internet, a cloud network, a Long Term Evolution (LTE) network, a (wireless local area network) WLAN, a local area network (LAN), a power over telephone (POTS), and / or a metropolitan area network (MAN). The various devices in the network environment 100 may be configured to connect to the communication network 114 according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocols, or Bluetooth (BT) communication protocols, or combinations thereof.

[0022] During operation, the electronic device 102 can initiate a reverse image search query. In some embodiments, the reverse image search can be initiated based on user input received via a display device (shown in FIG. 2). The electronic device 102 can be configured to receive a first image as an image search query at the initiation of the reverse image search. For example, the first image can correspond to an image uploaded through an I / O device (shown in FIG. 2) of the electronic device 102 based on user input. The first image can relate to a still image with fixed foreground or background objects or an image extracted from a video. The electronic device 102 can be configured to extract a first set of image features associated with the received first image using the DNN model 108. The electronic device 102 can be configured to extract a second set of image features associated with the received first image using the image feature detection model 110. Details of the first and second image feature sets are shown, for example, in FIG. 3. Examples of the image feature detection model 110 may include, but are not limited to, a Scale Invariant Feature Transform (SIFT)-based model, a Speed-Up Robust Features (SURF)-based model, an Oriented FAST and Rotated BRIEF (ORB)-based model, or a Fast Library for Approximate Nearest Neighbors (FLANN)-based model.

[0023] The electronic device 102 may be further configured to generate a first feature vector associated with the received first image based on the extracted first image feature set. The electronic device 102 may be further configured to generate a second feature vector associated with the received first image based on the extracted second image feature set. The first feature vector associated with the received first image may be a vector that can include information about the first image feature set, and the second feature vector associated with the received first image may be a vector that can include information about the second image feature set. The electronic device 102 may be further configured to generate a third feature vector associated with the received first image based on a combination of the generated first feature vector and the generated second feature vector. The third feature vector may include, but is not limited to, the generated first feature vector and the generated second vector. In some embodiments, the generation of the third feature vector may be further based on applying a principal component analysis (PCA) transform to the combination of the generated first feature vector and the generated second feature vector. Details of the generation of the third feature vector are described, for example, with reference to FIG. 3.

[0024] The electronic device 102 may be configured to determine a similarity metric between the generated third feature vector associated with the received first image and the fourth feature vector of each image in a pre-stored second set of images (e.g., images stored in the database 106). Examples of similarity metrics may include, but are not limited to, cosine distance similarity or Euclidean distance similarity. The electronic device 102 may be configured to identify a pre-stored third image (e.g., an image that is identical or similar to the received first image) from the pre-stored second set of images based on the determined similarity metric. The electronic device 102 may further be configured to control a display device to display information related to the identified pre-stored third image. Details of determining the similarity metric and identifying the pre-stored third image are further described, for example, with reference to FIG. 3 .

[0025] FIG. 2 is a block diagram illustrating an exemplary electronic device for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to an embodiment of the present disclosure. The description of FIG. 2 is provided with reference to elements in FIG. 1. FIG. 2 illustrates a block diagram 200 of an electronic device 102. The electronic device 102 may include circuitry 202, memory 204, input / output (I / O) devices 206, and a network interface 208. The I / O devices 206 may further include a display device 210. The network interface 208 may connect the electronic device 102 to the server 104 and the database 106 via a communication network 114.

[0026] Circuitry 202 may include suitable logic, circuits, and / or interfaces that can be configured to execute program instructions associated with different operations performed by electronic device 102. Circuitry 202 may include one or more specialized processing units that can be implemented as independent processors. In some embodiments, one or more specialized processing units may be implemented as an integrated processor or a group of processors that can be configured to collectively perform the functions of one or more specialized processing units. Circuitry 202 may be implemented based on multiple processor technologies known in the art. Example implementations of circuitry 202 may be an X86-based processor, a graphics processing unit (GPU), a reduced instruction set computing (RISC) processor, an application-specific integrated circuit (ASIC) processor, a complex instruction set computing (CISC) processor, a microcontroller, a central processing unit (CPU), and / or other control circuitry.

[0027] The memory 204 may include suitable logic, circuitry, interfaces, and / or code that may be configured to store program instructions executed by the circuit 202. In at least one embodiment, the memory 204 may be configured to store the DNN model 108 and the image feature detection model 110. The memory 204 may be configured to store one or more of, but not limited to, a similarity metric, a machine learning model different from the DNN model 108 and the image feature detection model 110 (e.g., machine learning model 316 of FIG. 3 ), a first weight associated with the generated first feature vector, and a second weight associated with the generated second feature vector. In some embodiments, the memory 204 may store the first image and the identified pre-stored third image. Example implementations of the memory 204 may include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), a hard disk drive (HDD), a solid-state drive (SSD), a CPU cache, and / or a secure digital (SD) card.

[0028] The I / O device 206 may include suitable logic, circuitry, interfaces, and / or code that may be configured to receive input and provide output based on the received input. The I / O device 206 may include a variety of input and output devices that may be configured to communicate with the circuit 202. In one example, the electronic device 102 may receive (via the I / O device 206) user input including a reverse image search query. The reverse image search query may include a first image. In another example, the electronic device 102 may receive (via the I / O device 206) user input including a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector. The electronic device 102 may control the I / O device 206 to output the identified pre-stored third image. Examples of the I / O device 206 may include, but are not limited to, a touch screen, a keyboard, a mouse, a joystick, a display device (e.g., the display device 210), a microphone, or a speaker.

[0029] The display device 210 may include suitable logic, circuitry, and interfaces that can be configured to display the output of the electronic device 102. The display device 210 may be utilized to display information related to the identified pre-stored third image. In some embodiments, the display device 210 may be an externally coupled display device associated with the electronic device 102. The display device 210 may be a touch screen that allows the user 116 to provide user input via the display device 210. The touch screen may be at least one of a resistive touch screen, a capacitive touch screen, a thermal touch screen, or any other touch screen that can be used to provide input to the display device 210. The display device 210 may be implemented through a number of known technologies, such as, but not limited to, at least one of a liquid crystal display (LCD) display, a light emitting diode (LED) display, a plasma display, or an organic LED (OLED) display technology, or other display devices.

[0030] The network interface 208 may include suitable logic, circuitry, interfaces, and / or code that may be configured to facilitate communication between the electronic device 102, the server 104, and the database 106 over the communications network 114. The network interface 208 may be implemented using various known technologies to support wired or wireless communication of the electronic device 102 to and from the communications network 112. The network interface 208 may include, but is not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a coder-decoder (CODEC) chipset, a subscriber identity module (SIM) card, or a local buffer circuit.

[0031] The network interface 208 may be configured to communicate with a network, such as the Internet, an intranet, a wireless network, a cellular telephone network, a wireless local area network (LAN), or a metropolitan area network (MAN), via wired communication, wireless communication, or a combination thereof. The wireless communication may be configured to use one or more of a number of communication standards, protocols, and technologies, such as Global System for Mobile Communications (GSM), Extended Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (WiFi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, or IEEE 802.11n), Voice over Internet Protocol (VoIP), Light Fidelity (Li-Fi), Worldwide Interoperability for Microwave Access (Wi-MAX), protocols for email, instant messaging, and short message service (SMS).

[0032] The operation of circuit 202 is further described, for example, in Figures 3 and 4. It should be noted that electronic device 102 shown in Figure 2 may also include various other components or systems, the description of which is omitted from this disclosure for the sake of brevity.

[0033] Figure 3 illustrates exemplary operations for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to an embodiment of the present disclosure. The description of Figure 3 is provided with reference to elements of Figures 1 and 2. Figure 3 illustrates a block diagram 300 illustrating exemplary operations 302-314 for reverse image search based on the DNN model 108 and the image feature detection model 110. The exemplary operations may be performed by any computer system, such as, for example, the electronic device 102 of Figure 1 or the circuit 202 of Figure 2.

[0034] At 302, a first image may be received. In one embodiment, the circuit 202 may be configured to receive the first image. For example, a first image 302A may be received. The first image 302A may be received from a data source, such as persistent storage (such as memory 204) on the electronic device 102, an image capture device, a cloud server, or a combination thereof. The first image 302A may include an object of interest for which the user 116 may require similar or identical image results using a reverse image search. Alternatively, the first image 302A may correspond to an image from a series of images in a first video. The circuit 202 may be configured to extract the first image 302A from the first video. The first video may correspond to a video including an object of interest for which the user 116 may require similar or identical video results using a reverse image search. The first image 302A may represent an image with a fixed foreground or background. For example, as shown, the first image 302A may represent a scene from a movie (e.g., an image of Spider-Man as the object of interest, as shown in FIG. 3).

[0035] After receiving the first image 302A, the circuit 202 can input the received first image 302A to the DNN model 108 and the image feature detection model 110 for image feature extraction. The circuit 202 can use the DNN model 108 to extract a first set of image features associated with the first image 302A, for example, as described at 304. Additionally, the circuit 202 can use the image feature detection model 110 to extract a second set of image features associated with the first image 302A, for example, as described at 306. Operations 304 and 306 can be performed in any order without departing from the scope of this disclosure.

[0036] At 304, first image features can be extracted. In one embodiment, the circuit 202 can be configured to extract a first set of image features associated with the received first image 302A using a deep neural network (DNN) model (such as the DNN model 108). The extracted first set of image features can correspond to unique features associated with one or more objects in the received first image 302A. The DNN model 108 can be pre-trained for the image feature extraction task based on a training dataset 112 of a pre-stored set of training images assigned predetermined tags. The predetermined tags assigned to a particular training image can include labels corresponding to image features that can be predetermined for the particular training image. The circuit 202 can provide the first image 302A as an input to the DNN model 108 and receive the first image feature set (i.e., the image feature set associated with the first image 302A) as an output from the DNN model 108 based on the image feature detection task performed by the DNN model 108 on the first image 302A.

[0037] In some embodiments, the extracted first image feature set may include information required to classify each included object into a particular object class. Examples of the extracted first image feature set may include, but are not limited to, shape, texture, color, and other high-level image features. For example, as shown in FIG. 3 , the extracted first image feature 304A associated with the first image 302A may include color, which indicates a shade of gray on the object of interest (such as Spider-Man's face). For example, if the object of interest in the first image 302A is a person's face, such as Spider-Man or another person / character, the first image feature set 304A may include the shape of the eyes, the shape of the ears, the shape of the nose, and the shape / texture of other high-level facial details of the person / character. The detailed implementation of the extraction of the first image feature set by the DNN model 108 is believed to be well known to those skilled in the art, and therefore, a detailed description of the extraction of such a first image feature set is omitted from this disclosure for the sake of brevity.

[0038] The circuit 202 can be configured to generate a first feature vector associated with the received first image 302A based on the extracted first image feature set. Such a first feature vector can also be referred to as a unique first image feature set. The generated first feature vector can include multiple vector elements, each of which can correspond to an image feature from the extracted first image feature set. Each vector element of the first feature vector can store a value that can correspond to a specific first image feature from the first image feature set. For example, if the received first image 302A is a high-definition image (e.g., an image of 1024×1024 pixels), the first feature vector can be a 1×2048 vector having 2048 vector elements. The ith element of the first feature vector can represent the value of the ith first image feature.

[0039] At 306, a second set of image features may be extracted. In some embodiments, the circuit 202 may be configured to extract a second set of image features associated with the received first image 302A using an image feature detection model (such as the image feature detection model 110). The extracted second set of image features may correspond to specific unique features associated with one or more objects included in the received first image 302A. In some embodiments, the second set of image features may be image features that may have been misdetected or remain undetected by the DNN model 108 (e.g., at 304). In some embodiments, the extracted second set of image features may include information needed to optimally classify each object (in the first image 302A) into a particular object class. Examples of the second set of image features may include, but are not limited to, edges, lines, contours, and other low-level image features. Examples of image feature detection models 110 may include, but are not limited to, a Scale Invariant Feature Transform (SIFT)-based model, a Speeded Up Robust Features (SURF)-based model, an Oriented FAST and Rotated BRIEF (ORB)-based model, or a Fast Library for Approximate Nearest Neighbors (FLANN)-based model. Detailed implementations of these exemplary methods are believed to be well known to those skilled in the art, and therefore, detailed descriptions of such methods are omitted from this disclosure for the sake of brevity. For example, as shown in FIG. 3 , a second image feature set 306A associated with a first image 302A represents a second image feature set extracted based on a SIFT-based model, and a second image feature set 306B associated with the first image 302A represents a second image feature set extracted based on a SURF-based model. For example, if the face of a person, such as Spider-Man or any other person / character in the first image 302A, is the object of interest, the second image feature set 306A (or the second image feature set 306B) may include eye edges and contours, ear edges and contours, nose edges and contours, and other low-level facial details of the person / character.

[0040] The circuit 202 may be further configured to generate a second feature vector associated with the received first image 302A based on the extracted second image feature set. Such a second feature vector may also be referred to as a unique second image feature set. The generated second feature vector may include multiple vector elements, each of which may correspond to an image feature from the extracted second image feature set. Each vector element of the second feature vector may store a value corresponding to a specific second image feature from the second image feature set. For example, if the received first image 302A is a high-definition image (e.g., an image of 1024×1024 pixels), the second feature vector may be a 1×2048 vector having 2048 vector elements. The ith element of the second feature vector may represent the value of the ith second image feature.

[0041] At 308, the feature vectors can be combined. In one embodiment, the circuit 202 can be configured to generate a third feature vector associated with the received first image 302A based on combining the generated first feature vector and the generated second feature vector. In one embodiment, the circuit 202 can be configured to automatically combine the generated first feature vector and the generated second feature vector. For example, if the received first image 302A is a high-definition image (e.g., an image of 1024×1024 pixels), the first feature vector can be a 1×2048 vector having 2048 vector elements, and the second feature vector can also be a 1×2048 vector having 2048 vector elements. In such a case, the third feature vector can be a 1×4096 vector having 4096 vector elements.

[0042] In one embodiment, the circuit 202 can be configured to determine a first weight associated with a generated first feature vector and a second weight associated with a generated second feature vector by a machine learning model 316 (i.e., a machine learning model different from the DNN model 108 and the image feature detection model 110). The machine learning model 316 can be a regression model that can be trained based on a set of similarly sized feature vectors, where each vector can be tagged with a user-defined weight value. The machine learning model 316 can be trained based on a feature vector weight assignment task, where the machine learning model 316 can receive two similarly sized feature vector sets as inputs and output weights for each of the two similarly sized feature vector sets. The machine learning model 316 can be defined by hyperparameters, such as the number of weights, a cost function, an input size, and the number of layers. The hyperparameters of the machine learning model 316 can be adjusted to approach a global minimum of the cost function of the machine learning model 316, and the weights can be updated accordingly. The machine learning model 316 can be trained to output weight values ​​for an input set after several epochs of training based on feature information in a training dataset. The output may indicate a weight value for each input (eg, the first feature vector and the second feature vector) in the input set.

[0043] The machine learning model 316 may include electronic data that may be implemented, for example, as a software component of an application executable on the electronic device 102. The machine learning model 316 may rely on libraries, external scripts, or other logic / instructions for execution by a computing device including a processor, such as the circuit 202. The machine learning model 316 may include code and routines configured to enable a computing device including a processor, such as the circuit 202, to perform one or more operations for determining first weights associated with a first feature vector and determining second weights associated with a second feature vector. Additionally or alternatively, the machine learning model 316 may be implemented using hardware, including a processor, a microprocessor (e.g., performing or controlling one or more operations), a field programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). Alternatively, in some embodiments, the machine learning model 316 may be implemented using a combination of hardware and software.

[0044] The first weight associated with the generated first feature vector and the second weight associated with the generated second feature vector may each indicate a likelihood of reliability of the respective feature vector for identifying an object of interest in an image and, therefore, identifying similar images that may contain the object of interest. The first weight and the second weight may specify a confidence value for reliability (with a probability value between 0 and 1). Thus, a more reliable feature vector may have a higher weight value. For example, if the received first image 302A is a high-resolution image and the extracted first image feature set is more precise than the extracted second image feature set, the generated first feature vector may have a higher confidence value than the generated second feature vector. In such an example, the first weight associated with the generated first feature vector may have a higher weight value (e.g., 0.6) compared to the second weight (e.g., 0.4) associated with the generated second feature vector. In contrast, if the received first image 302A is a low-resolution image and the extracted second image feature set is more precise than the extracted first image feature set, the generated second feature vector may have a higher confidence value than the generated first feature vector. In such an example, the first weight associated with the generated first feature vector may have a lower weight value (e.g., a 0.4 value) compared to the second weight associated with the generated second feature vector (e.g., a 0.6 value). Furthermore, if the received first image 302A is a medium resolution image (e.g., a standard resolution image), the first weight associated with the generated first feature vector may have an equal weight value (e.g., 0.5) compared to the second weight associated with the generated second feature vector (e.g., a 0.5 value). The circuit 202 may be further configured to combine the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight, and further generate a third feature vector based on the combination.

[0045] In one embodiment, the circuit 202 can be configured to receive user input including a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector. In one example, the received user input can indicate the first weight associated with the generated first feature vector as “0.4,” such that the circuit 202 can be configured to determine the second weight associated with the generated second feature vector as “0.6.” In another example, the received user input can indicate the first weight associated with the generated first feature vector as “0.5” and the second weight associated with the generated second feature vector as “0.5.” The circuit 202 can be further configured to combine the generated first feature vector and the generated second feature vector based on the received user input and further generate a third feature vector based on the combination.

[0046] In one embodiment, the circuit 202 can be configured to classify, by the DNN model 108, the received first image 302A with a first image tag from a set of image tags associated with the DNN model 108. The first image tag can specify an image tag to which the received first image 302A can belong. For example, the received first image 302A can have an image tag such as a Spider-Man character. The circuit 202 can be further configured to determine a first image count associated with the first image tag in a training dataset (such as the training dataset 112) associated with the DNN model 108. For example, the circuit 202 can compare the first image tag with image tags of pre-stored training images from a set of pre-stored training images in the training dataset 112. If the image tag of a pre-stored training image in the training dataset 112 matches the first image tag, the circuit 202 increments the first image count associated with the first image tag by one. Similarly, the circuit 202 can determine a first image count based on a comparison of the image tag of each of the pre-stored training images in the training dataset 112 with the first image tag. The circuit 202 can be further configured to determine a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on the determined first image count associated with the first image tag. For example, if the first image count in the training dataset 112 is higher than a certain threshold (e.g., a threshold count or percentage of all images in the training dataset 112), the first weight associated with the generated first feature vector can have a higher weight value than the second weight associated with the generated second feature vector. In contrast, if the first image count in the training dataset 112 is lower than a threshold or nominal value (e.g., a value that can be easily ignored, such as a few hundred images in a training dataset 112 of one million images), the first weight associated with the generated first feature vector can have a lower weight value than the second weight associated with the generated second feature vector.The circuit 202 can be configured to combine the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight to generate a third feature vector, and further generate a third feature vector based on the combination.

[0047] In some embodiments, the circuit 202 can be configured to determine an image quality score associated with the received first image 302A based on at least one of the extracted first set of image features or the extracted second set of image features. The image quality score can indicate a qualitative value related to the fidelity of the received first image 302A. A higher image quality score can indicate a higher fidelity of the received first image 302A. The image quality score can correspond to, but is not limited to, sharpness, noise, dynamic range, tone reproduction, contrast, saturation, distortion, vignetting, exposure accuracy, chromatic aberration, lens flare, color moire, or artifacts associated with the received first image 302A. Sharpness can correspond to details related to image features associated with the received first image 302A. For example, if the pixel count or focus of the received first image 302A is high, the sharpness of the received first image 302A can be high. Noise may correspond to disturbances in the received first image 302A, such as unwanted pixel-level variations in the received first image 302A. Dynamic range may correspond to the amount of tonal difference between the lightest and darkest shades of light captured in the received first image 302A. Tone reproduction may correspond to the correlation between the amount of light captured in the received first image 302A and the amount of light to which the received first image 302A is exposed. Contrast may correspond to the amount of color variation in the received first image 302A. Saturation may correspond to the intensity of colors in the received first image 302A. Distortion may correspond to unwanted pixel variations in the received first image 302A. Vignetting may correspond to darkening, reduced sharpness, or reduced color saturation from the corners of the received first image 302A compared to the center of the received first image 302A. Exposure accuracy may correspond to capturing the received first image 302A at optimal brightness.Chromatic aberration may correspond to color distortion in the received first image 302A. Lens flare may correspond to the response of an image capture device to bright light. Color moiré may correspond to repetitive color stripes appearing in the received first image 302A. Artifacts related to the received first image 302A may correspond to any virtual objects that may be present in the received first image 302A.

[0048] The circuit 202 may be further configured to determine a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on the determined image quality score. For example, if the determined image quality score is high, the first weight associated with the generated first feature vector may be assigned a higher weight value than the second weight associated with the generated second feature vector. In contrast, if the determined image quality score is low or nominal, the first weight associated with the generated first feature vector may be assigned a lower weight value than the second weight associated with the generated second feature vector. In one embodiment, if the image quality score is above a threshold, the determined first weight may be higher than the determined second weight. The threshold may include image quality scores such as, for example, 0.4, 0.6, and 0.8. In one embodiment, the circuit 202 may be configured to receive a user input setting a threshold for the image quality score. In another embodiment, the circuit 202 may be configured to automatically set the threshold for the image quality score. The circuit 202 may be configured to combine the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight, and then may be further configured to generate a third feature vector based on combining the generated first feature vector and the generated second feature vector.

[0049] At 310, dimensionality can be reduced. In one embodiment, the circuit 202 can be configured to reduce the dimensionality of the generated third feature vector. In some embodiments, the circuit 202 can resize (or compress) the generated third feature vector to match the size of the input layer of the feature extractor and pass the resized generated third feature vector to the input layer of the feature extractor. This can reduce undesired or repetitive information from the generated third feature vector. The generation of the third feature vector can be further based on applying a principal component analysis (PCA) transform to the combination of the generated first feature vector and the generated second feature vector. For example, if the generated third vector is a "1 x 4096" vector having 4096 vector elements, after applying the PCA transform, the generated third vector can be reduced to a "1 x 256" vector having 256 vector elements. The detailed implementation of the PCA transform is believed to be well known to those skilled in the art, and therefore a detailed description of such a transform will be omitted from this disclosure for the sake of brevity.

[0050] At 312, a similarity metric can be determined. In one embodiment, the circuit 202 can be configured to determine a similarity metric between the generated third feature vector associated with the received first image 302A and a fourth feature vector for each image in the pre-stored set of second images. The pre-stored set of second images can be stored in the database 106. In one embodiment, the circuit 202 can be configured to generate the fourth feature vector for each image in the pre-stored set of second images. For example, the circuit 202 can apply the DNN model 108, the image feature detection model 110, or a combination of both to each image in the pre-stored set of second images to generate the fourth feature vector for each pre-stored second image. In another example, the fourth feature vector for each respective pre-stored second image can be predetermined and pre-stored in the database 106 along with the respective pre-stored second image. The similarity metric can correspond to a similarity measure for determining images from the pre-stored second set of images that are similar to the received first image 302A. In such a case, the generated third feature vector associated with the received first image 302A can be compared with the fourth feature vector of each image in the set of pre-stored second images based on the determined similarity metric to identify similar images. Examples of similarity metrics include, but are not limited to, cosine distance similarity or Euclidean distance similarity. In cosine distance similarity, the cosine distance between the generated third feature vector associated with the received first image 302A and the fourth feature vector of each image in the set of pre-stored second images can be determined. For example, if the fourth feature vector of a particular pre-stored second image has a small cosine distance to the generated third vector, the particular pre-stored second image can be identified as one of the images similar to the received first image 302A.

[0051] At 314, similar images can be identified. In an embodiment, the circuit 202 can be configured to identify the pre-stored third image as a similar image from the pre-stored second set of images based on the determined similarity metric. For example, the circuit 202 can compare the generated third feature vector associated with the received first image 302A with the fourth feature vector of each image in the pre-stored second set of images based on the similarity metric. If it is determined based on the similarity metric that the fourth feature vector of a particular pre-stored second image matches the generated third feature vector associated with the received first image 302A, the circuit 202 can identify the particular pre-stored second image from the pre-stored second set of images as a pre-stored third image (i.e., a similar image).

[0052] The circuit 202 may be further configured to control a display device (such as the display device 210) to display information related to the identified pre-stored third image. The information related to the identified pre-stored third image may include, but is not limited to, information such as the pre-stored third image itself, metadata related to the pre-stored third image, a feature map between the third feature vector and the fourth feature vector, a file size of the pre-stored third image, a storage location associated with the pre-stored third image, or a file download path associated with the pre-stored third image. In an embodiment, the identified pre-stored third image may correspond to a pre-stored second video. The pre-stored second video may be related to the first video. For example, the pre-stored third image may be one of the image frames from a set of image frames in the pre-stored second video. In an embodiment, the first image 302A may be extracted from the first video. The pre-stored third image can be related or similar to the received first image 302A, and the pre-stored second video can be related or similar to the first video.

[0053] 3 illustrates information 314A related to an identified pre-stored third image that can be identified based on the DNN model 108 and the image feature detection model 110 (e.g., a SIFT-based model). The information 314A may include a feature map between a generated third feature vector associated with the received first image 302A and a fourth feature vector associated with the identified pre-stored third image. The example illustrated in FIG. 3 also illustrates information 314B related to an identified pre-stored third image that can be identified based on the DNN model 108 and the image feature detection model 110 (e.g., a SURF-based model). The information 314B may also include a feature map between a generated third feature vector associated with the received first image 302A and a fourth feature vector associated with the identified pre-stored third image.

[0054] As described above, the disclosed electronic device 102 can automatically generate a third feature vector associated with the received first image 302A based on a combination of the generated first feature vector and the generated second feature vector. As a result, the third feature vector can include a first image feature set that can be determined by the DNN model 108 and a second image feature set that can be determined by the image feature detection model 110. The first image feature set can include high-level image features (e.g., facial features such as eyes, nose, ears, and hair) associated with the received first image 302A (e.g., an image of a person's face), and the second image feature set can include low-level image features (e.g., points, edges, lines, contours, or basic objects and shapes of a face). Including both the high-level and low-level image features in the third feature vector can complement each other in identifying similar images. In some scenarios, the first image feature set may not detect and extract all features that may be present in the received first image 302A. For example, some features may be misdetected or remain undetected by the DNN model 108. For example, if the received first image 302A is an image that is not well represented in the training dataset 112 of the DNN model 108, the first image feature set may not be sufficient to identify images similar to the received first image from the pre-stored second image set. However, since the second image feature set (i.e., the second image feature set determined by the image feature detection model 110) may include low-level image features related to the received first image 302A, including the second image feature set in the third feature vector can further improve the accuracy of identifying images similar to the received first image 302A from the pre-stored second image set. For example, if the image quality is poor (e.g., a blurry image with low resolution), the first image feature set (i.e., high-level image features) may not be sufficient to identify similar images. In such cases, the second image features (i.e., low-level image features) may be more useful and accurate for identifying similar images.

[0055] Figure 4 is a flowchart illustrating an exemplary method for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to an embodiment of the present disclosure. Figure 4 is described with reference to elements in Figures 1, 2, and 3. Figure 4 illustrates a flowchart 400. The method illustrated in flowchart 400 can be performed by any computer system, such as electronic device 102 or circuit 202. The method can start at 402 and proceed to 404.

[0056] At 404, a first image (such as first image 302A) may be received. In one or more embodiments, circuit 202 may be configured to receive first image 302A. Receiving first image 302A is further described, for example, in FIG. 3 (at 302).

[0057] At 406, a first image feature set (such as first image feature set 304A) associated with the received first image (e.g., first image 302A) may be extracted by a deep neural network (DNN) model (e.g., DNN model 108). In one or more embodiments, circuit 202 may be configured to extract first image feature set 304A associated with the received first image 302A by DNN model 108. Extraction of first image feature set 304A is further described, for example, in FIG. 3 (at 304).

[0058] At 408, a first feature vector associated with the received first image 302A can be generated based on the extracted set of first image features 304A. The circuit 202 can be configured to generate a first feature vector associated with the received first image 302A based on the extracted first image features 304A. The generation of the first feature vector is further described, for example, in FIG. 3 (at 304).

[0059] At 410, a second set of image features (such as the second image feature set 306A) associated with the received first image 302A can be extracted by an image feature detection model (e.g., the image feature detection model 110). In one or more embodiments, the circuit 202 can be configured to extract the second set of image features 306A associated with the received first image 302A by the image feature detection model 110. The image feature detection model 110 includes at least one of a Scale Invariant Feature Transform (SIFT)-based model, a Speeded Up Robust Features (SURF)-based model, an Oriented FAST and Rotated BRIEF (ORB)-based model, or a Fast Library for Approximate Nearest Neighbors (FLANN)-based model. Extraction of the second image feature set 306A is further described, for example, in FIG. 3 (at 306).

[0060] At 412, a second feature vector associated with the received first image 302A can be generated based on the extracted set of second image features 306A. In one or more embodiments, the circuit 202 can be configured to generate a second feature vector associated with the received first image 302A based on the extracted set of second image features 306A. Generation of the second feature vector is further described, for example, in FIG. 3 (at 306).

[0061] At 414, a third feature vector associated with the received first image 302A may be generated based on a combination of the generated first feature vector and the generated second feature vector. In one or more embodiments, the circuit 202 may be configured to generate a third feature vector associated with the received first image 302A based on a combination of the generated first feature vector and the generated second feature vector. The generation of the third feature vector may be further based on applying a principal component analysis (PCA) transform to the combination of the generated first feature vector and the generated second feature vector. The generation of the third feature vector is further described, for example, in FIG. 3 (at 308).

[0062] At 416, a similarity metric may be determined between the generated third feature vector associated with the received first image 302A and the fourth feature vector of each image in the pre-stored second set of images. In one or more embodiments, the circuit 202 may be configured to determine a similarity metric between the generated third feature vector associated with the received first image 302A and the fourth feature vector of each image in the pre-stored second set of images. In one example, the similarity metric may include, but is not limited to, at least one of cosine distance similarity or Euclidean distance similarity. Determining the similarity metric is further described, for example, in FIG. 3 (at 312).

[0063] At 418, a pre-stored third image (e.g., a pre-stored third image) may be identified from the set of pre-stored second images based on the determined similarity metric. In one or more embodiments, circuit 202 may be configured to identify the pre-stored third image from the set of pre-stored second images based on the determined similarity metric. Identifying the pre-stored third image is further described, for example, in FIG. 3 (at 314).

[0064] At 420, a display device (such as display device 210) may be controlled to display information related to the identified pre-stored third image. Circuit 202 may be configured to control display device 210 to display information related to the identified pre-stored third image. Control of display device 210 is further described, for example, in FIG. 3 (at 314). Control proceeds to the end.

[0065] Although flowchart 400 is depicted as discrete operations such as 404, 406, 408, 410, 412, 416, 418, and 420, the disclosure is not so limited. Thus, in some embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated depending on the particular implementation without departing from the essence of the disclosed embodiments.

[0066] Various embodiments of the present disclosure may provide a non-transitory computer-readable medium and / or storage medium having stored thereon instructions executable by a machine and / or a computer (e.g., electronic device 102). The instructions may cause the machine and / or computer to perform operations including receiving a first image (e.g., first image 302A). The operations may further include extracting, with a deep neural network (DNN) model (e.g., DNN model 108), a first set of image features (e.g., first image feature set 304A) associated with the received first image 302A. The operations may further include generating a first feature vector associated with the received first image 302A based on the extracted first image features 304A. The operations may further include extracting, with an image feature detection model (e.g., image feature detection model 110), a second set of image features (e.g., second image feature set 306A) associated with the received first image 302A. The operations may further include generating a second feature vector associated with the received first image 302A based on the extracted second image features 304A. The operations may further include generating a third feature vector associated with the received first image 302A based on a combination of the generated first feature vector and the generated second feature vector. The operations may further include determining a similarity metric between the generated third feature vector associated with the received first image and a fourth feature vector of each image in the set of pre-stored second images. The operations may further include identifying a pre-stored third image from the set of pre-stored second images based on the determined similarity metric. The operations may further include controlling a display device (e.g., display device 210) to display information associated with the identified pre-stored third image.

[0067] An exemplary aspect of the present disclosure may provide an electronic device (such as the electronic device 102 of FIG. 1 ) including a circuit (such as the circuit 202). The circuit 202 may be configured to receive a first image 302A. The circuit 202 may be configured to extract a first set of image features (such as a first image feature set 304A) associated with the received first image 302A using a deep neural network (DNN) model 108. The circuit 202 may be configured to generate a first feature vector associated with the received first image 302A based on the extracted first image features 304A. The circuit 202 may be configured to extract a second set of image features (such as a second image feature set 306A) associated with the received first image 302A using an image feature detection model 110. The circuit 202 may be configured to generate a second feature vector associated with the received first image 302A based on the extracted second image features 306A. The circuit 202 can be configured to generate a third feature vector associated with the received first image 302A based on a combination of the generated first feature vector and the generated second feature vector. The circuit 202 can be configured to determine a similarity metric between the generated third feature vector associated with the received first image 302A and a fourth feature vector of each image in the set of pre-stored second images. The circuit 202 can be configured to identify a pre-stored third image from the set of pre-stored second images based on the determined similarity metric. The circuit 202 can be configured to control the display device 210 to display information associated with the identified pre-stored third image.

[0068] According to an embodiment, the image feature detection model 110 may include at least one of, but is not limited to, a Scale Invariant Feature Transform (SIFT)-based model, a Speed-Up Robust Features (SURF)-based model, an Oriented FAST and Rotated BRIEF (ORB)-based model, or a Fast Library for Approximate Nearest Neighbors (FLANN)-based model.

[0069] According to an embodiment, generating the third feature vector may be further based on applying a principal component analysis (PCA) transform to a combination of the generated first feature vector and the generated second feature vector. According to an embodiment, the similarity metric may include at least one of, but is not limited to, cosine distance similarity or Euclidean distance similarity.

[0070] According to an embodiment, the circuit 202 may be further configured to determine a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector by a machine learning model (e.g., the machine learning model 316) different from the DNN model 108 and the image feature detection model 110. The circuit 202 may be further configured to combine the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight. The circuit 202 may be configured to generate a third feature vector based on combining the generated first feature vector and the generated second feature vector.

[0071] According to an embodiment, the circuit 202 may be further configured to receive a user input including a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector. The circuit 202 may be further configured to combine the generated first feature vector and the generated second feature vector based on the received user input. The circuit 202 may be configured to generate a third feature vector based on combining the generated first feature vector and the generated second feature vector.

[0072] According to an embodiment, the circuit 202 may be further configured to classify, by the DNN model 108, the received first image 302A to a first image tag from a set of image tags associated with the DNN model 108. The circuit 202 may be configured to determine a first image count associated with the first image tag within a training dataset (e.g., training dataset 112) associated with the DNN model 108. The circuit 202 may be further configured to determine a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on the determined first count of images associated with the first image tag. The circuit 202 may be further configured to combine the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight. The circuit 202 may be configured to generate a third feature vector based on combining the generated first feature vector and the generated second feature vector.

[0073] According to an embodiment, the circuit 202 can be configured to determine an image quality score associated with the received first image 302A based on at least one of the extracted first image feature set 304A or the extracted second image feature set 306A. The circuit 202 can be further configured to determine a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on the determined image quality score. The circuit 202 can be further configured to combine the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight. The circuit 202 can be configured to generate a third feature vector based on combining the generated first feature vector and the generated second feature vector. According to an embodiment, the image quality score can correspond to at least one of, but not limited to, sharpness, noise, dynamic range, tone reproduction, contrast, saturation, distortion, vignetting, exposure accuracy, chromatic aberration, lens flare, color moiré, or artifacts associated with the received first image 302A.

[0074] According to an embodiment, the circuit 202 may be further configured to extract the first image 302A from the first video and an identified pre-stored third image that may correspond to a pre-stored second video, where the pre-stored second video may be related to the first video.

[0075] The present disclosure can be implemented in hardware or a combination of hardware and software. The present disclosure can be implemented in a centralized manner in at least one computer system, or in a distributed manner where different elements can be distributed across several interconnected computer systems. Any computer system or other device adapted to perform the methods described herein can be suitable. The combination of hardware and software can be a general-purpose computer system that includes a computer program that, when loaded and executed, can control the computer system to perform the methods described herein. The present disclosure can be implemented in hardware, including portions of integrated circuits that also perform other functions.

[0076] The present disclosure may also be embodied in a computer program product, which includes all features that enable the implementation of the methods described herein and which is capable of executing these methods when loaded into a computer system. A computer program in this context means any expression, in any language, code or notation, of a set of instructions intended to cause a system having information processing capabilities to perform a particular function, either directly, or after a) conversion into another language, code or notation, or b) reproduction in a different content form, or both.

[0077] While the present disclosure has been described with reference to several embodiments, those skilled in the art will recognize that various modifications may be made and equivalents may be substituted without departing from the scope of the disclosure. Additionally, many modifications may be made to adapt a particular situation or material to the teachings of the disclosure without departing from the scope of the disclosure. Therefore, it is not intended that the disclosure be limited to the particular embodiments disclosed, but rather, it is intended to include all embodiments falling within the scope of the appended claims. [Explanation of symbols]

[0078] 102 Electronic equipment 104 Server 106 databases 108 Deep Neural Network (DNN) Models 110 Image Feature Detection Model 112 training dataset 114 Communication Network 116 users

Claims

1. 1. An electronic device, comprising: receiving a first image; extracting a first set of image features associated with the received first image using a deep neural network (DNN) model; generating a first feature vector associated with the received first image based on the extracted set of first image features; extracting a second set of image features associated with the received first image using an image feature detection model; generating a second feature vector associated with the received first image based on the extracted second set of image features; determining a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on a resolution of the received first image; combining the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight; generating a third feature vector associated with the received first image based on a combination of the generated first feature vector and the generated second feature vector; determining a similarity metric between the generated third feature vector associated with the received first image and a fourth feature vector of each image of a second set of pre-stored images; identifying a pre-stored third image from the set of pre-stored second images based on the determined similarity metric; controlling a display device to display information related to the identified pre-stored third image. a circuit configured to: An electronic device characterized by:

2. the image feature detection model comprises at least one of a Scale Invariant Feature Transform (SIFT)-based model, a Speeded Up Robust Features (SURF)-based model, an Oriented FAST and Rotated BRIEF (ORB)-based model, or a Fast Library for Approximate Nearest Neighbors (FLANN)-based model; The electronic device of claim 1 .

3. the generation of the third feature vector is further based on applying a principal component analysis (PCA) transform to the combination of the generated first feature vector and the generated second feature vector. The electronic device of claim 1 .

4. the similarity metric includes at least one of cosine distance similarity or Euclidean distance similarity; The electronic device of claim 1 .

5. The circuit comprises: determining a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector by a machine learning model different from the DNN model and the image feature detection model; The electronic device of claim 1 further configured to:

6. The circuit comprises: receiving user input including a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector; combining the generated first feature vector and the generated second feature vector based on the received user input; generating the third feature vector based on the combination of the generated first feature vector and the generated second feature vector; The electronic device of claim 1 further configured to:

7. The circuit comprises: classifying, with the DNN model, the received first image into a first image tag from a set of image tags associated with the DNN model; determining a first image count associated with the first image tag in a training data set associated with the DNN model; determining a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on the determined first image count associated with the first image tag; combining the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight; generating the third feature vector based on the combination of the generated first feature vector and the generated second feature vector; The electronic device of claim 1 further configured to:

8. The circuit comprises: determining an image quality score associated with the received first image based on at least one of the extracted first set of image features or the extracted second set of image features; determining a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on the determined image quality score; combining the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight; generating the third feature vector based on the combination of the generated first feature vector and the generated second feature vector; The electronic device of claim 1 further configured to:

9. the image quality score corresponds to at least one of sharpness, noise, dynamic range, tone reproduction, contrast, color saturation, distortion, vignetting, exposure accuracy, chromatic aberration, lens flare, color moiré, or artifacts associated with the received first image; 9. The electronic device of claim 8.

10. the circuitry is further configured to extract the first image from a first video and the identified pre-stored third image corresponding to a pre-stored second video related to the first video. The electronic device of claim 1 .

11. In an electronic device, receiving a first image; extracting a first set of image features associated with the received first image using a deep neural network (DNN) model; generating a first feature vector associated with the received first image based on the extracted set of first image features; extracting a second set of image features associated with the received first image using an image feature detection model; generating a second feature vector associated with the received first image based on the extracted second set of image features; determining a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on a resolution of the received first image; combining the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight; generating a third feature vector associated with the received first image based on a combination of the generated first feature vector and the generated second feature vector; determining a similarity metric between the generated third feature vector associated with the received first image and a fourth feature vector of each image of a second set of pre-stored images; identifying a pre-stored third image from the set of pre-stored second images based on the determined similarity metric; controlling a display device to display information related to the identified pre-stored third image; A method comprising:

12. the image feature detection model comprises at least one of a Scale Invariant Feature Transform (SIFT)-based model, a Speeded Up Robust Features (SURF)-based model, an Oriented FAST and Rotated BRIEF (ORB)-based model, or a Fast Library for Approximate Nearest Neighbors (FLANN)-based model; The method of claim 11.

13. the generation of the third feature vector is further based on applying a principal component analysis (PCA) transform to the combination of the generated first feature vector and the generated second feature vector. The method of claim 11.

14. the similarity metric includes at least one of cosine distance similarity or Euclidean distance similarity; The method of claim 11.

15. determining a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector by a machine learning model different from the DNN model and the image feature detection model; The method of claim 11 further comprising:

16. receiving user input including a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector; combining the generated first feature vector and the generated second feature vector based on the received user input; generating the third feature vector based on the combination of the generated first feature vector and the generated second feature vector; The method of claim 11 further comprising:

17. classifying, with the DNN model, the received first image into a first image tag from a set of image tags associated with the DNN model; determining a first image count associated with the first image tag in a training data set associated with the DNN model; determining a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on the determined first image count associated with the first image tag; combining the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight; generating the third feature vector based on the combination of the generated first feature vector and the generated second feature vector; The method of claim 11 further comprising:

18. determining an image quality score associated with the received first image based on at least one of the extracted first set of image features or the extracted second set of image features; determining a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on the determined image quality score; combining the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight; generating the third feature vector based on the combination of the generated first feature vector and the generated second feature vector; The method of claim 11 further comprising:

19. the image quality score corresponds to at least one of sharpness, noise, dynamic range, tone reproduction, contrast, color saturation, distortion, vignetting, exposure accuracy, chromatic aberration, lens flare, color moiré, or artifacts associated with the received first image; 20. The method of claim 18.

20. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by an electronic device, receiving a first image; extracting a first set of image features associated with the received first image using a deep neural network (DNN) model; generating a first feature vector associated with the received first image based on the extracted set of first image features; extracting a second set of image features associated with the received first image using an image feature detection model; generating a second feature vector associated with the received first image based on the extracted second set of image features; determining a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on a resolution of the received first image; combining the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight; generating a third feature vector associated with the received first image based on a combination of the generated first feature vector and the generated second feature vector; determining a similarity metric between the generated third feature vector associated with the received first image and a fourth feature vector of each image of a second set of pre-stored images; identifying a pre-stored third image from the set of pre-stored second images based on the determined similarity metric; controlling a display device to display information related to the identified pre-stored third image; 10. A non-transitory computer-readable medium for causing the electronic device to perform operations including:

Images