Reverse image search based on deep neural network (DNN) model and image feature detection model

Abstract

An electronic device and method for reverse image search are provided. The electronic device receives an image. It extracts a first set of image features associated with the image using a DNN model and generates a first feature vector based on the first set of image features. It then extracts a second set of image features associated with the image using an image feature detection model and generates a second feature vector based on the second set of image features. Finally, it generates a third feature vector based on a combination of the first and second feature vectors. The electronic device determines a similarity measure between the third feature vector and a fourth feature vector for each image in a pre-stored set of images and identifies the pre-stored images based on this similarity measure. The electronic device controls a display device to display information associated with the pre-stored images.

Description

[0001] Cross-reference to related applications / incorporation by reference

[0002] This application claims priority to U.S. Patent Application Serial No. 17 / 482,290, filed September 22, 2021, and to U.S. Provisional Patent Application Serial No. 63 / 189,956, filed May 18, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0003] Various embodiments of this disclosure relate to reverse image search. More specifically, various embodiments of this disclosure relate to electronic devices and methods for reverse image search based on deep neural network (DNN) models and image feature detection models. Background Technology

[0004] Advances in information and communication technologies have led to the development of various internet-based image search systems (e.g., web search engines). Typically, users upload an input image to a web search engine as a search query. In this case, the web search engine (using a reverse image search method) can provide a set of output images from the internet. This set of output images can be similar to the input image. This reverse image search method can employ machine learning models to determine the set of output images similar to the input image. In some cases, the machine learning model may misclassify one or more objects in the input image, which could result in the set of output images including unwanted or irrelevant images.

[0005] By comparing the described system with some aspects of this disclosure as set forth in the remainder of this application and with reference to the accompanying drawings, the limitations and disadvantages of conventional and traditional methods will become clear to those skilled in the art. Summary of the Invention

[0006] An electronic device and method for reverse image search based on a deep neural network (DNN) model and an image feature detection model are provided substantially as shown in at least one of the accompanying drawings and / or described in conjunction with at least one of the accompanying drawings.

[0007] These and other features and advantages of this disclosure will become apparent from a review of the following detailed description of this disclosure and the accompanying drawings, in which the same reference numerals refer to the same parts throughout. Attached Figure Description

[0008] Figure 1 This is a block diagram illustrating an exemplary network environment for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to embodiments of the present disclosure.

[0009] Figure 2 This is a block diagram illustrating an exemplary electronic device for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to embodiments of the present disclosure.

[0010] Figure 3 This is a diagram illustrating exemplary operation of reverse image search based on a deep neural network (DNN) model and an image feature detection model according to embodiments of the present disclosure.

[0011] Figure 4 This is a flowchart illustrating an exemplary method for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to embodiments of the present disclosure. Detailed Implementation

[0012] Implementations described below can be found in the disclosed electronic devices and methods for reverse image search based on deep neural network (DNN) models and image feature detection models, to enhance the accuracy of reverse image search. An exemplary aspect of this disclosure provides an electronic device that implements a deep neural network (DNN) model and an image feature detection model for reverse image search. The electronic device may receive a first image (such as an image for which a user needs to search for similar images). The electronic device may extract a first set of image features associated with the received first image using a deep neural network (DNN) model, and generate a first feature vector associated with the received first image based on the extracted first set of image features. The electronic device may extract a second set of image features associated with the received first image using an image feature detection model, and generate a second feature vector associated with the received first image based on the extracted second set of image features. Examples of image feature detection models may include, but are not limited to, models based on scale-invariant feature transform (SIFT), models based on accelerated robust features (SURF), models based on oriented FAST and rotated BRIEF (ORB), or models based on a fast approximate nearest neighbor library (FLANN). Image feature detection models can extract those image features, some of which may be incorrectly detected and / or misclassified by DNN models.

[0013] The electronic device can also generate a third feature vector associated with the received first image based on a combination of the generated first feature vector and the generated second feature vector. In an example, the generation of the third feature vector can also be based on the application of principal component analysis (PCA) transformation to the combination of the generated first and second feature vectors. The electronic device can also determine a similarity measure between the generated third feature vector associated with the received first image and a fourth feature vector of each of the pre-stored second images (such as images stored in a database). Examples of similarity measures may include, but are not limited to, cosine distance similarity or Euclidean distance similarity. The electronic device can also identify pre-stored third images (such as images identical or similar to the received first image) from the pre-stored set of second images based on the determined similarity measure, and control a display device to display information associated with the identified pre-stored third image.

[0014] The disclosed electronic device can automatically generate a third feature vector associated with a received first image based on a combination of a generated first feature vector and a generated second feature vector. The third feature vector can thus include a first set of image features that can be determined by a DNN model and a second set of image features that can be determined by an image feature detection model. The first set of image features can include higher-level image features (e.g., facial features such as eyes, nose, ears, and hair), and the second set of image features can include lower-level image features associated with the received first image (e.g., edges, lines, and contours of the face). The inclusion of both higher-level and lower-level image features in the third feature vector can complement each other for the recognition of similar images. For example, if the received first image is not adequately represented in the training dataset of the DNN model, the first set of image features may be insufficient to identify images similar to the received first image from a pre-stored set of second images. However, since the second set of image features can include lower-level image features associated with the received first image, including the second set of image features in the third feature vector can improve the accuracy of identifying images similar to the received first image from a pre-stored set of second images. On the other hand, if the image quality is poor (e.g., if it is a low-resolution and blurry image), then the first set of image features (i.e., higher-level image features) may be insufficient to identify similar images. In this case, the second set of image features (i.e., lower-level image features) may be more useful and accurate for identifying similar images.

[0015] Figure 1 This is a block diagram illustrating an exemplary network environment for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to embodiments of the present disclosure. Reference Figure 1 The diagram illustrates a network environment 100. Network environment 100 may include an electronic device 102, a server 104, and a database 106. A deep neural network (DNN) model 108 and an image feature detection model 110 implemented on server 104 are also shown. Figure 1 As shown, training dataset 112 can be stored on database 106. Electronic device 102, server 104, and database 106 can be communicatively coupled to each other via communication network 114. User 116 associated with electronic device 102 is also shown. Figure 1 In this disclosure, electronic device 102 and server 104 are shown as two separate devices; however, in some embodiments, the full functionality of server 104 may be incorporated into electronic device 102 without departing from the scope of this disclosure.

[0016] Electronic device 102 may include suitable logic, circuitry, interfaces, and / or code that can be configured to identify and display a set of images similar to the first image based on an implementation of DNN model 108 and image feature detection model 110 on the first image. Examples of electronic device 102 may include, but are not limited to, image search engines, servers, personal computers, laptops, computer workstations, mainframes, gaming devices, virtual reality (VR) / augmented reality (AR) / mixed reality (MR) devices, smartphones, mobile phones, computing devices, tablets, and / or any consumer electronics (CE) device.

[0017] DNN model 108 can be a deep convolutional neural network model, which can be trained on an image feature detection task to detect a first set of image features in a first image. DNN model 108 can be defined by its hyperparameters (e.g., one or more activation functions, the number of weights, a cost function, a regularization function, input size, number of layers, etc.). DNN model 108 can be referred to as a system or computational network of artificial neurons (also called nodes). The nodes of DNN model 108 can be arranged in multiple layers, as defined in the neural network topology of DNN model 108. The multiple layers of DNN model 108 can include an input layer, one or more hidden layers, and an output layer. Each of the multiple layers can include one or more nodes (or artificial neurons). The outputs of all nodes in the input layer can be coupled to at least one node in one or more hidden layers. Similarly, the input of each hidden layer can be coupled to the output of at least one node in the other layers of DNN model 108. The output of each hidden layer can be coupled to the input of at least one node in the other layers of DNN model 108. The nodes in the last layer can receive input from at least one hidden layer to output a result. The number of layers and the number of nodes in each layer can be determined by the hyperparameters of the DNN model 108. These hyperparameters can be set before or simultaneously with training the DNN model 108 on the training dataset 112.

[0018] Each node of the DNN model 108 can correspond to a mathematical function (e.g., a sigmoid function or a rectified linear unit) with a parameter set that is adjustable during network training. The parameter set may include, for example, weight parameters, regularization parameters, etc. Each node can use the mathematical function to compute an output based on one or more inputs from nodes in one or more other layers (e.g., preceding layers). All or some of the nodes in the DNN model 108 can correspond to the same or different mathematical functions.

[0019] In training a DNN model 108, one or more parameters of each node of the DNN model 108 can be updated based on whether the output of the last layer for a given input (from the training dataset) matches the correct result based on the loss function used for the DNN model 108. This process can be repeated for the same or different inputs until the minimum of the loss function is achieved and the training error is minimized. Several methods for training are known in the art, such as gradient descent, stochastic gradient descent, batch gradient descent, gradient boosting, metaheuristics, etc.

[0020] In embodiments, DNN model 108 may include electronic data, which may be implemented as a software component of an application executable, for example, on electronic device 102 or server 104. DNN model 108 may rely on libraries, external scripts, or other logic / instructions for execution by a processing device (such as electronic device 102 or server 104). DNN model 108 may include computer-executable code or routines to enable a computing device such as electronic device 102 or server 104 to perform one or more operations to detect image features in an input image. Alternatively or additionally, DNN model 108 may be implemented using hardware including a processor, microprocessor (e.g., to perform or control the execution of one or more operations), field-programmable gate array (FPGA), or application-specific integrated circuit (ASIC). For example, an inference accelerator chip may be included in electronic device 102 (or server 104) to accelerate computation of DNN model 108 for image feature detection tasks. In some embodiments, DNN model 108 may be implemented using a combination of both hardware and software. Examples of DNN models 108 may include, but are not limited to, Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN), Region with CNN (R-CNN), Fast R-CNN, Faster R-CNN, You Only See Once (YOLO) network, Residual Neural Network (Res-Net), Feature Pyramid Network (FPN), Retina-Net, Single Shot Detector (SSD), and / or combinations thereof.

[0021] Image feature detection model 110 may be an image processing algorithm configured to extract image features associated with a first image. Image feature detection model 110 may be defined by its hyperparameters (e.g., the number of image features, edge thresholds, the number of weights, a cost function, input size, the number of layers, etc.). The hyperparameters of image feature detection model 110 may be tuned, and the weights may be updated to move towards the global minimum of the cost function of image feature detection model 110. Image feature detection model 110 may include electronic data, which may be implemented as a software component of an application, for example, executable on electronic device 102 or server 104. Image feature detection model 110 may rely on libraries, external scripts, or other logic / instructions to be executed by a processing device such as electronic device 102 or server 104. Image feature detection model 110 may include code and routines configured to enable a computing device such as electronic device 102 or server 104 to perform one or more operations, such as extracting a set of image features associated with the first image. Alternatively, the image feature detection model 110 may be implemented using hardware including a processor, a microprocessor (e.g., to perform or control the execution of one or more operations), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). Alternatively, in some embodiments, the image feature detection model 110 may be implemented using a combination of hardware and software. Examples of the image feature detection model 110 may include, but are not limited to, models based on scale-invariant feature transform (SIFT), models based on accelerated robust features (SURF), models based on oriented FAST and rotated BRIEF (ORB), or models based on the fast approximate nearest neighbor library (FLANN).

[0022] Server 104 may include suitable logic, circuitry, interfaces, and / or code that can be configured to store DNN model 108 and image feature detection model 110. Server 104 can use DNN model 108 to generate a first feature vector associated with a first image and can use image feature detection model 110 to generate a second feature vector associated with the first image. Server 104 may also store machine learning models different from DNN model 108 and image feature detection model 110. The stored machine learning models can be configured to determine a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector. In an exemplary embodiment, server 104 may be implemented as a cloud server and can perform operations via web applications, cloud applications, HTTP requests, repository operations, file transfers, etc. Other example implementations of server 104 may include, but are not limited to, database servers, file servers, web servers, application servers, mainframe servers, or cloud computing servers.

[0023] In at least one embodiment, server 104 can be implemented as multiple distributed cloud-based resources using several techniques well known to those skilled in the art. Those skilled in the art will understand that the scope of this disclosure is not limited to implementing server 104 and electronic device 102 as two separate entities. In some embodiments, the functionality of server 104 may be wholly or at least partially integrated into electronic device 102 without departing from the scope of this disclosure.

[0024] Database 106 may include suitable logic, interfaces, and / or code that can be configured to store a training dataset 112 for DNN model 108. Training dataset 112 may include a pre-stored set of training images and a predetermined label assigned to each training image in the pre-stored set of training images. The predetermined label assigned to a training image may include a label corresponding to image features that can be pre-determined for a particular training image. DNN model 108 may be pre-trained on training dataset 112 for an image feature detection task. In embodiments, database 106 may also be configured to store a pre-stored set of second images. Database 106 may be a relational database or a non-relational database. Moreover, in some cases, database 106 may be stored on a server (such as server 104) such as a cloud server, or may be cached and stored on electronic device 102. Furthermore, or alternatively, database 106 may be implemented using hardware including a processor, a microprocessor (e.g., to perform or control the execution of one or more operations), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). In some other cases, database 106 can be implemented using a combination of hardware and software.

[0025] Communication network 114 may include a communication medium through which electronic devices 102, server 104, and database 106 can communicate with each other. Examples of communication network 114 may include, but are not limited to, the Internet, cloud networks, Long Term Evolution (LTE) networks, Wireless Local Area Networks (WLANs), Local Area Networks (LANs), Point-of-Sale (POTS) telephone lines, and / or Metropolitan Area Networks (MANs). Various devices in network environment 100 may be configured to connect to communication network 114 according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Li-Fi, 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access points (APs), device-to-device communication, cellular communication protocols, or Bluetooth (BT) communication protocols, or at least one or a combination thereof.

[0026] In operation, electronic device 102 can initiate a reverse image search query. In an embodiment, the reverse image search can be based on a display device ( Figure 2 The reverse image search is initiated by receiving user input (as shown in the diagram). After initiating the reverse image search, the electronic device 102 can be configured to receive a first image as an image search query. For example, the first image can be obtained by connecting the first image to the user input via the I / O device of the electronic device 102 (as shown in the diagram). Figure 2 The image uploaded corresponds to the image shown in the diagram. The first image can be associated with a static image with a fixed foreground or background object or an image extracted from a video. Electronic device 102 can be configured to extract a first set of image features associated with the received first image using DNN model 108. Electronic device 102 can be configured to extract a second set of image features associated with the received first image using image feature detection model 110. For example, in Figure 3 Details of the first and second image feature sets are provided. Examples of the image feature detection model 110 may include, but are not limited to, models based on scale-invariant feature transform (SIFT), models based on accelerated robust features (SURF), models based on oriented FAST and rotated BRIEF (ORB), or models based on the fast approximate nearest neighbor library (FLANN).

[0027] Electronic device 102 can also be configured to generate a first feature vector associated with the received first image based on an extracted first image feature set. Electronic device 102 can also be configured to generate a second feature vector associated with the received first image based on an extracted second image feature set. The first feature vector associated with the received first image can be a vector that includes information about the first image feature set, and the second feature vector associated with the received first image can be a vector that includes information about the second image feature set. Electronic device 102 can also be configured to generate a third feature vector associated with the received first image based on a combination of the generated first feature vector and the generated second feature vector. The third feature vector may include, but is not limited to, the generated first feature vector and the generated second feature vector. In embodiments, the generation of the third feature vector can also be based on the application of principal component analysis (PCA) transformation to a combination of the generated first feature vector and the generated second feature vector. Details of the generation of the third feature vector are provided, for example, in... Figure 3 Further details are provided below.

[0028] Electronic device 102 can be configured to determine a similarity measure between a generated third feature vector associated with the received first image and a fourth feature vector of each of a set of pre-stored second images (such as images stored in database 106). Examples of similarity measures may include, but are not limited to, cosine distance similarity or Euclidean distance similarity. Electronic device 102 can be configured to identify a pre-stored third image (such as an image identical or similar to the received first image) from the set of pre-stored second images based on the determined similarity measure. Electronic device 102 can also be configured to control a display device to display information associated with the identified pre-stored third image. Details of the determination of the similarity measure and the identification of the pre-stored third image are provided, for example, in... Figure 3 Further details are provided below.

[0029] Figure 2 This is a block diagram illustrating an exemplary electronic device for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to embodiments of the present disclosure. (Combined with...) Figure 1 To explain the components Figure 2 . refer to Figure 2 The diagram shows a block diagram 200 of an electronic device 102. The electronic device 102 may include a circuit system 202, a memory 204, an input / output (I / O) device 206, and a network interface 208. The I / O device 206 may also include a display device 210. The network interface 208 can connect the electronic device 102 to a server 104 and a database 106 via a communication network 114.

[0030] Circuit system 202 may include suitable logic, circuitry, and / or interfaces configured to execute program instructions associated with different operations to be performed by electronic device 102. Circuit system 202 may include one or more dedicated processing units, which may be implemented as separate processors. In embodiments, one or more dedicated processing units may be implemented as integrated processors or clusters of processors configured to collectively perform the functions of one or more dedicated processing units. Circuit system 202 may be implemented based on a variety of processor technologies known in the art. Examples of implementations of circuit system 202 may include x86-based processors, graphics processing units (GPUs), reduced instruction set computing (RISC) processors, application-specific integrated circuit (ASIC) processors, complex instruction set computing (CISC) processors, microcontrollers, central processing units (CPUs), and / or other control circuitry.

[0031] Memory 204 may include suitable logic, circuitry, interfaces, and / or code, which may be configured to store program instructions to be executed by circuitry 202. In at least one embodiment, memory 204 may be configured to store DNN model 108 and image feature detection model 110. Memory 204 may also be configured to store similarity metrics, machine learning models different from DNN model 108 and image feature detection model 110 (e.g., Figure 3 The machine learning model 316), a first weight associated with the generated first feature vector, and a second weight associated with the generated second feature vector, or one or more of these, but not limited to these. In an embodiment, memory 204 may store a first image and a pre-stored identified third image. Examples of implementations of memory 204 may include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), hard disk drive (HDD), solid-state drive (SSD), CPU cache, and / or secure digital card (SD card).

[0032] I / O device 206 may include suitable logic, circuitry, interfaces, and / or code that can be configured to receive input and provide output based on the received input. I / O device 206 may include various input and output devices that can be configured to communicate with circuitry 202. In one example, electronic device 102 may (via I / O device 206) receive user input including a reverse image search query. The reverse image search query may include a first image. In another example, electronic device 102 may (via I / O device 206) receive user input including a first weight associated with a generated first feature vector and a second weight associated with a generated second feature vector. Electronic device 102 may control I / O device 206 to output a pre-stored, identified third image. Examples of I / O device 206 may include, but are not limited to, a touchscreen, keyboard, mouse, joystick, display device (e.g., display device 210), microphone, or speaker.

[0033] Display device 210 may include suitable logic, circuitry, and interfaces that can be configured to display the output of electronic device 102. Display device 210 can be used to display information associated with a pre-stored, identified third image. In some embodiments, display device 210 may be an externally coupled display device associated with electronic device 102. Display device 210 may be a touchscreen that allows user 116 to provide user input via display device 210. The touchscreen may be at least one of a resistive touchscreen, a capacitive touchscreen, a thermal touchscreen, or any other touchscreen that can provide input to display device 210. Display device 210 may be implemented using several known technologies, such as, but not limited to, at least one of a liquid crystal display (LCD), a light-emitting diode (LED) display, a plasma display, or an organic LED (OLED) display, or other display technologies.

[0034] Network interface 208 may include suitable logic, circuitry, interfaces, and / or code that can be configured to facilitate communication between electronic device 102, server 104, and database 106 via communication network 114. Network interface 208 can be implemented using various known technologies to support wired or wireless communication between electronic device 102 and communication network 114. Network interface 208 may include, but is not limited to, antennas, radio frequency (RF) transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, subscriber identity module (SIM) cards, or local buffer circuitry.

[0035] Network interface 208 can be configured to communicate via wired or wireless communication, or a combination thereof, using networks such as the Internet, intranets, wireless networks, cellular telephone networks, wireless local area networks (LANs), or metropolitan area networks (MANs). Wireless communication can be configured to use one or more of a variety of communication standards, protocols, and technologies, such as Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, or IEEE 802.11n), Voice over Internet Protocol (VoIP), Li-Fi, Wi-MAX, protocols for email, instant messaging, and Short Message Service (SMS).

[0036] For example in Figure 3 and 4 The operation of circuit system 202 is further described below. It can be noted that... Figure 2 The electronic device 102 shown may include various other components or systems. For the sake of brevity, descriptions of other components or systems of the electronic device 102 are omitted in this disclosure.

[0037] Figure 3 This is a diagram illustrating exemplary operation of reverse image search based on a deep neural network (DNN) model and an image feature detection model according to embodiments of the present disclosure. Figure 3 Combination Figure 1 and Figure 2 The elements in the text will be explained. (See reference.) Figure 3 A block diagram 300 illustrates exemplary operations from 302 to 314 for inverse image search based on DNN model 108 and image feature detection model 110. These exemplary operations can be performed by any computing system, e.g., by [unclear text - likely a typo]. Figure 1 Electronic device 102 or by Figure 2 The circuit system 202 is executed.

[0038] At 302, a first image can be received. In an embodiment, circuitry 202 can be configured to receive the first image. For example, a first image 302A can be received. The first image 302A can be received from a data source (e.g., persistent storage device (such as memory 204) on electronic device 102, image capture device, cloud server, or a combination thereof). The first image 302A may include an object of interest for which user 116 can use a reverse image search to request similar or identical image results. Alternatively, the first image 302A may correspond to an image from an image sequence from a first video. Circuitry 202 can be configured to extract the first image 302A from the first video. The first video may correspond to a video that includes an object of interest for which user 116 uses a reverse image search to request similar or identical video results. The first image 302A may represent an image with a fixed foreground or background. As shown, for example, the first image 302A may depict a scene from a movie (e.g., an image of Spider-Man as an object of interest, such as...). Figure 3 (as shown in the image).

[0039] Upon receiving the first image 302A, the circuit system 202 can input the received first image 302A into the DNN model 108 and the image feature detection model 110 for image feature extraction. The circuit system 202 can employ the DNN model 108 to extract a first set of image features associated with the first image 302A, as described in 304, for example. The circuit system 202 can also employ the image feature detection model 110 to extract a second set of image features associated with the first image 302A, as described in 306, for example. Operations 304 and 306 can be performed in any order without departing from the scope of this disclosure.

[0040] At point 304, a first set of image features can be extracted. In an embodiment, circuitry 202 can be configured to extract a first set of image features associated with the received first image 302A via a deep neural network (DNN) model (such as DNN model 108). The extracted first set of image features may correspond to unique features associated with one or more objects in the received first image 302A. DNN model 108 may be pre-trained for an image feature extraction task based on a training dataset 112 of a pre-stored set of training images assigned predetermined labels. The predetermined labels assigned to certain training images may include labels corresponding to image features that can be pre-determined for a particular training image. Circuitry 202 may feed the first image 302A as input to DNN model 108 and may receive the first set of image features (i.e., associated with the first image 302A) as output from DNN model 108 based on an image feature detection task performed by DNN model 108 on the first image 302A.

[0041] In an embodiment, the extracted first set of image features may include information that might be needed to classify each included object into a specific object class. Examples of the extracted first set of image features may include, but are not limited to, shape, texture, color, and other high-level image features. Figure 3 As shown, for example, the extracted first image feature set 304A associated with the first image 302A may include colors depicted as gray shadows on an object of interest (such as Spider-Man's face). For example, if the object of interest is the face of a person (such as Spider-Man or any other person / character) in the first image 302A, then the first image feature set 304A may include the shape of the eyes, the shape of the ears, the shape of the nose, and the shape / texture of other high-level facial details of the person / character. Specific implementations of extracting the first image feature set using the DNN model 108 are known to those skilled in the art; therefore, for the sake of brevity, a detailed description of such extraction of the first image feature set is omitted in this disclosure.

[0042] The circuit system 202 can be configured to generate a first feature vector associated with the received first image 302A based on an extracted first image feature set. This first feature vector can also be referred to as a unique set of first image features. The generated first feature vector can include multiple vector elements, where each vector element can correspond to an image feature from the extracted first image feature set. Each vector element of the first feature vector can store a value that corresponds to a certain first image feature from the first image feature set. For example, if the received first image 302A is a high-resolution image (e.g., a "1024x1024" pixel image), then the first feature vector can be a "1x2048" vector with 2048 vector elements. The i-th element of the first feature vector can represent the value of the i-th first image feature.

[0043] At 306, a second set of image features can be extracted. In an embodiment, circuitry 202 can be configured to extract a second set of image features associated with the received first image 302A using an image feature detection model (such as image feature detection model 110). The extracted second set of image features may also correspond to certain unique features associated with one or more objects included in the received first image 302A. In some embodiments, the second set of image features may be image features that might be misdetected by the DNN model 108 or may remain undetected (e.g., at 304). In an embodiment, the extracted second set of image features may include information that might be needed to optimally classify each object (in the first image 302A) into a particular object class. Examples of the second set of image features may include, but are not limited to, edges, lines, contours, and other low-level image features. Examples of image feature detection model 110 may include, but are not limited to, models based on Scale Invariant Feature Transform (SIFT), models based on Accelerated Robust Features (SURF), models based on Oriented FAST and Rotated BRIEF (ORB), or models based on the Fast Approximate Nearest Neighbor Library (FLANN). Detailed implementations of these example methods are known to those skilled in the art; therefore, for the sake of brevity, detailed descriptions of such methods are omitted in this disclosure. Figure 3As shown, for example, the second image feature set 306A associated with the first image 302A depicts a second image feature set extracted according to a SIFT-based model, and the second image feature set 306B associated with the first image 302A depicts a second image feature set extracted according to a SURF-based model. For example, if the object of interest is the face of a person (such as Spider-Man or any other person / character in the first image 302A), then the second image feature set 306A (or the second image feature set 306B) may include the edges and contours of the eyes, the edges and contours of the ears, the edges and contours of the nose, and other low-level facial details of the person / character.

[0044] The circuit system 202 can also be configured to generate a second feature vector associated with the received first image 302A based on the extracted second image feature set. This second feature vector can also be referred to as a unique set of second image features. The generated second feature vector can include multiple vector elements, where each vector element can correspond to an image feature in the extracted second image feature set. Each vector element of the second feature vector can store a value that corresponds to a certain second image feature in the second image feature set. For example, if the received first image 302A is a high-resolution image (e.g., a "1024x1024" pixel image), then the second feature vector can be a "1x2048" vector with 2048 vector elements. The i-th element of the second feature vector can represent the value of the i-th second image feature.

[0045] At point 308, feature vectors can be combined. In an embodiment, circuitry 202 can be configured to generate a third feature vector associated with the received first image 302A based on a combination of the generated first feature vector and the generated second feature vector. In an embodiment, circuitry 202 can be configured to automatically combine the generated first feature vector and the generated second feature vector. For example, if the received first image 302A is a high-resolution image (e.g., an image with "1024x1024" pixels), then the first feature vector can be a "1x2048" vector with 2048 vector elements, and the second feature vector can also be a "1x2048" vector with 2048 vector elements. In this case, the third feature vector can be a "1x4096" vector with 4096 vector elements.

[0046] In an embodiment, circuit system 202 can be configured to determine a first weight associated with a generated first feature vector and a second weight associated with a generated second feature vector using machine learning model 316 (i.e., different from DNN model 108 and image feature detection model 110). Machine learning model 316 can be a regression model trained on a set of feature vectors of similar sizes, where each vector can be labeled with a user-defined weight value. Machine learning model 316 can be trained on a feature vector weight assignment task, where it can receive two sets of feature vectors of similar sizes as input and output weights for each feature vector in the two sets of feature vectors of similar sizes. Machine learning model 316 can be defined by its hyperparameters (e.g., the number of weights, cost function, input size, number of layers, etc.). The hyperparameters of machine learning model 316 can be tuned, and the weights can be updated to move toward the global minimum of the cost function of machine learning model 316. After training on feature information in a training dataset for several epochs, machine learning model 316 can be trained to output weight values ​​for the set of inputs. The output can indicate the weight value of each input in the set of inputs (e.g., the first feature vector and the second feature vector).

[0047] Machine learning model 316 may include electronic data, which may be implemented as a software component of an application, for example, executable on electronic device 102. Machine learning model 316 may rely on libraries, external scripts, or other logic / instructions for execution by a computing device (such as circuit system 202) including a processor. Machine learning model 316 may include code and routines configured to enable the computing device (such as circuit system 202) including a processor to perform one or more operations for determining a first weight associated with a first feature vector and a second weight associated with a second feature vector. Alternatively or additionally, machine learning model 316 may be implemented using hardware including a processor, a microprocessor (e.g., to perform or control the execution of one or more operations), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). Alternatively, in some embodiments, machine learning model 316 may be implemented using a combination of hardware and software.

[0048] Each of the first weight associated with the generated first feature vector and the second weight associated with the generated second feature vector can indicate the reliability of the corresponding feature vector in identifying an object of interest in an image and thereby identifying similar images that may include that object of interest. The first and second weights can specify a confidence value (based on a probability value between 0 and 1) for reliability. Therefore, a more reliable feature vector can have a higher weight value. For example, if the received first image 302A is a high-resolution image and the extracted first image feature set is more detailed than the extracted second image feature set, then the generated first feature vector can be more reliable than the generated second feature vector. In this example, the first weight associated with the generated first feature vector can have a higher weight value (such as 0.6) compared to the second weight associated with the generated second feature vector (such as a value of 0.4). In contrast, if the received first image 302A is a low-resolution image and the extracted second image feature set is more detailed than the extracted first image feature set, then the generated second feature vector can be more reliable than the generated first feature vector. In this example, the first weight associated with the generated first feature vector can have a smaller weight value (such as 0.4) compared to the second weight (such as a value of 0.6) associated with the generated second feature vector. Additionally, if the received first image 302A is a medium-resolution image (e.g., a standard-resolution image), then the first weight associated with the generated first feature vector can have an equal weight value (such as 0.5) compared to the second weight (such as a value of 0.5) associated with the generated second feature vector. Based on the determined first weight and the determined second weight, the circuit system 202 can also be configured to combine the generated first feature vector and the generated second feature vector to further generate a third feature vector based on this combination.

[0049] In an embodiment, the circuit system 202 can be configured to receive user input, which includes a first weight associated with a generated first feature vector and a second weight associated with a generated second feature vector. In one example, the received user input might indicate the first weight associated with the generated first feature vector as "0.4", and then the circuit system 202 could be configured to determine the second weight associated with the generated second feature vector as "0.6". In another example, the received user input might indicate the first weight associated with the generated first feature vector as "0.5" and the second weight associated with the generated second feature vector as "0.5". Based on the received user input, the circuit system 202 can also be configured to combine the generated first feature vector and the generated second feature vector to further generate a third feature vector based on the combination.

[0050] In an embodiment, circuitry 202 may be configured to classify a received first image 302A into a first image tag from a set of image tags associated with DNN model 108 via DNN model 108. The first image tag may specify the image tag to which the received first image 302A may belong. For example, the received first image 302A may have an image tag such as the Spider-Man character. Circuitry 202 may also be configured to determine a first count of images associated with the first image tag in a training dataset (such as training dataset 112) associated with DNN model 108. For example, circuitry 202 may compare the first image tag with image tags of pre-stored training images from a set of pre-stored training images in training dataset 112. If an image tag of a pre-stored training image in training dataset 112 matches the first image tag, then circuitry 202 may increment the first count of images associated with the first image tag by one. Similarly, based on a comparison of the image tag of each training image in the set of pre-stored training images in training dataset 112 with the first image tag, circuitry 202 may determine a first count of images. The circuit system 202 can also be configured to determine a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on a first count of images associated with the first image label. For example, if the first count of images in training dataset 112 is higher than a certain threshold (e.g., a certain threshold count or percentage of the total number of images in training dataset 112), then the first weight associated with the generated first feature vector may have a higher weight value than the second weight associated with the generated second feature vector. Conversely, if the first count of images in training dataset 112 is less than a threshold or nominal value (e.g., a very negligible value, such as a few hundred images in a training dataset 112 of one million images), then the first weight associated with the generated first feature vector may have a smaller weight value than the second weight associated with the generated second feature vector. Based on the determined first weight and the determined second weight, the circuit system 202 can be configured to combine the generated first feature vector and the generated second feature vector to generate a third feature vector, thereby further generating a third feature vector based on this combination.

[0051] In an embodiment, circuitry 202 may be configured to determine an image quality score associated with the received first image 302A based on at least one of an extracted first image feature set or an extracted second image feature set. The image quality score may indicate a qualitative value associated with the fidelity level of the received first image 302A. A higher image quality score may indicate a higher fidelity level of the received first image 302A. The image quality score may correspond to, but is not limited to, sharpness, noise, dynamic range, tone reproduction, contrast, color saturation, distortion, vignetting, exposure accuracy, chromatic aberration, lens flare, color moiré, or artifacts associated with the received first image 302A. Sharpness may correspond to the detail associated with the image features associated with the received first image 302A. For example, if the pixel count or focus of the received first image 302A is high, then the sharpness of the received first image 302A may be high. Noise may correspond to disturbances in the received first image 302A, such as unwanted pixel-level variations in the received first image 302A. Dynamic range corresponds to the amount of tonal difference between the brightest and darkest shadows of light that can be captured in the received first image 302A. Tone reproduction corresponds to the correlation between the amount of light that can be captured in the received first image 302A and the amount of light that the received first image 302A can be exposed to. Contrast corresponds to the amount of color change in the received first image 302A. Color saturation corresponds to the color intensity in the received first image 302A. Distortion corresponds to unwanted pixel changes in the received first image 302A. Vignetting corresponds to the darkening, reduced sharpness, or reduced saturation of the received first image 302A from the corners compared to the center. Exposure accuracy corresponds to capturing the received first image 302A at optimal brightness. Chromatic aberration corresponds to color distortion in the received first image 302A. Lens flare corresponds to the response of the image capture device to bright light. Color moiré corresponds to repeating colored stripes that may appear in the received first image 302A. The artifacts associated with the received first image 302A can correspond to any virtual object that can exist in the received first image 302A.

[0052] The circuit system 202 can also be configured to determine a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector based on the determined image quality score. For example, if the determined image quality score is high, the first weight associated with the generated first feature vector can be assigned a higher weight value than the second weight associated with the generated second feature vector. Conversely, if the determined image quality score is low or nominal, the first weight associated with the generated first feature vector can be assigned a lower weight value than the second weight associated with the generated second feature vector. In an embodiment, the determined first weight can be higher than the determined second weight when the image quality score is higher than a threshold. The threshold can include, for example, image quality scores such as "0.4", "0.6", "0.8", etc. In an embodiment, the circuit system 202 can be configured to receive user input to set a threshold for the image quality score. In another embodiment, the circuit system 202 can be configured to automatically set the threshold for the image quality score. Based on the determined first weight and the determined second weight, the circuit system 202 can be configured to combine the generated first feature vector and the generated second feature vector. Subsequently, the circuit system 202 can also be configured to generate a third feature vector based on a combination of the generated first feature vector and the generated second feature vector.

[0053] At 310, dimensionality can be reduced. In an embodiment, circuitry 202 can be configured to reduce the dimensionality of the generated third feature vector. In some embodiments, circuitry 202 can adjust the size of the generated third feature vector (or compress the generated third feature vector) to match the size of the input layer of the feature extractor, and then pass the sized generated third feature vector to the input layer of the feature extractor. This can reduce unwanted or redundant information from the generated third feature vector. The generation of the third feature vector can also be based on the application of principal component analysis (PCA) transformation to a combination of the generated first feature vector and the generated second feature vector. For example, if the generated third vector is a “1x4096” vector with 4096 vector elements, then after applying the PCA transformation, the generated third vector can be reduced to a “1x256” vector with 256 vector elements. Detailed implementations of the PCA transformation will be known to those skilled in the art, and therefore, for the sake of brevity, a detailed description of such transformation is omitted in this disclosure.

[0054] At point 312, a similarity metric can be determined. In an embodiment, circuitry 202 can be configured to determine a similarity metric between a generated third feature vector associated with the received first image 302A and a fourth feature vector of each second image in a pre-stored set of second images. The pre-stored set of second images can be stored in database 106. In an embodiment, circuitry 202 can be configured to generate a fourth feature vector for each second image in the pre-stored set of second images. For example, circuitry 202 can apply a DNN model 108, an image feature detection model 110, or a combination of both to each second image in the pre-stored set of second images to generate a fourth feature vector for the corresponding pre-stored second image. In another example, the fourth feature vector of each corresponding pre-stored second image can be predetermined and pre-stored in database 106 along with the corresponding pre-stored second image. The similarity metric can correspond to the similarity measurement to determine images similar to the received first image 302A from the pre-stored set of second images. In this scenario, based on a determined similarity metric, the generated third feature vector associated with the received first image 302A can be compared with the fourth feature vector of each second image in a pre-stored set of second images to identify similar images. Examples of similarity metrics may include, but are not limited to, cosine distance similarity or Euclidean distance similarity. In cosine distance similarity, the cosine distance between the generated third feature vector associated with the received first image 302A and the fourth feature vector of each second image in a pre-stored set of second images can be determined. For example, if the fourth feature vector of a certain pre-stored second image has a small cosine distance relative to the generated third vector, then that particular pre-stored second image can be identified as one of the images similar to the received first image 302A.

[0055] At point 314, similar images can be identified. In an embodiment, circuitry 202 can be configured to identify a pre-stored third image as a similar image from a set of pre-stored second images based on a determined similarity metric. For example, circuitry 202 can compare a generated third feature vector associated with the received first image 302A with a fourth feature vector of each second image in the set of pre-stored second images based on the similarity metric. Based on the similarity metric, if it is determined that the fourth feature vector of a certain pre-stored second image matches the generated third feature vector associated with the received first image 302A, then circuitry 202 can identify that particular pre-stored second image as a pre-stored third image (i.e., a similar image) from the set of pre-stored second images.

[0056] The circuit system 202 can also be configured to control a display device (such as display device 210) to display information associated with the identified pre-stored third image. The information associated with the identified pre-stored third image may include, but is not limited to, information such as the pre-stored third image itself, metadata associated with the pre-stored third image, feature mapping between a third feature vector and a fourth feature vector, the file size of the pre-stored third image, the storage location associated with the pre-stored third image, or the file download path associated with the pre-stored third image. In an embodiment, the identified pre-stored third image may correspond to a pre-stored second video. The pre-stored second video may be associated with a first video. For example, the pre-stored third image may be one of the image frames in a set of image frames in the pre-stored second video. In an embodiment, a first image 302A may be extracted from the first video. Since the pre-stored third image may be associated with or similar to the received first image 302A, the pre-stored second video may be associated with or similar to the first video.

[0057] As Figure 3 The example shown illustrates information 314A associated with the identified pre-stored third image, which can be identified based on DNN model 108 and image feature detection model 110 (such as a SIFT-based model). Information 314A may include a feature mapping between a generated third feature vector associated with the received first image 302A and a fourth feature vector associated with the identified pre-stored third image. Figure 3 The example shown also illustrates information 314B associated with the identified pre-stored third image, which can be identified based on DNN model 108 and image feature detection model 110 (such as a SURF-based model). Information 314B may also include a feature mapping between a generated third feature vector associated with the received first image 302A and a fourth feature vector associated with the identified pre-stored third image.

[0058] As discussed above, the disclosed electronic device 102 can automatically generate a third feature vector associated with the received first image 302A based on a combination of the generated first feature vector and the generated second feature vector. The third feature vector can thus include a first set of image features that can be determined by the DNN model 108 and a second set of image features that can be determined by the image feature detection model 110. The first set of image features can include higher-level image features (e.g., facial features such as eyes, nose, ears, and hair), and the second set of image features can include lower-level image features associated with the received first image 302A (e.g., an image of a face) (e.g., points, edges, lines, contours, or basic objects and shapes of the face). The inclusion of both higher-level and lower-level image features in the third feature vector can complement each other for the recognition of similar images. In some scenarios, the first set of image features may not be able to detect and extract all features that may be present in the received first image 302A. In this document, some features may be misdetected or not detected by the DNN model 108. For example, if the received first image 302A is an image that is not adequately represented in the training dataset 112 of the DNN model 108, then the first image feature set may be insufficient to identify images similar to the received first image from the pre-stored set of second images. However, since the second image feature set (i.e., determined by the image feature detection model 110) can include lower-level image features associated with the received first image 302A, including the second image feature set in the third feature vector can further improve the accuracy of identifying images similar to the received first image 302A from the pre-stored set of second images. For example, in cases of poor image quality (e.g., in the case of low-resolution and blurry images), the first image feature set (i.e., higher-level image features) may be insufficient to identify similar images. In such cases, the second image feature set (i.e., lower-level image features) may be more useful and accurate for identifying similar images.

[0059] Figure 4 This is a flowchart illustrating an exemplary method for reverse image search based on a deep neural network (DNN) model and an image feature detection model, according to embodiments of the present disclosure. Figure 4 Combined with from Figure 1 , 2 Let's interpret the elements with 3. (See reference.) Figure 4 A flowchart 400 is shown. The method shown in flowchart 400 can be executed by any computing system, such as electronic device 102 or circuit system 202. The method can begin at 402 and proceed to 404.

[0060] At 404, a first image (such as first image 302A) can be received. In one or more embodiments, circuit system 202 can be configured to receive first image 302A. The reception of first image 302A is, for example, in... Figure 3 Further description (at 302).

[0061] At 406, a first set of image features (such as first image feature set 304A) associated with the received first image (e.g., first image 302A) can be extracted using a deep neural network (DNN) model (e.g., DNN model 108). In one or more embodiments, circuit system 202 can be configured to extract the first image feature set 304A associated with the received first image 302A using DNN model 108. The extraction of the first image feature set 304A is, for example, in... Figure 3 Further description (at 304).

[0062] At 408, a first feature vector associated with the received first image 302A can be generated based on the extracted first image feature set 304A. In one or more embodiments, the circuit system 202 can be configured to generate the first feature vector associated with the received first image 302A based on the extracted first image feature set 304A. The generation of the first feature vector is, for example, in... Figure 3 Further description (at 304).

[0063] At 410, a second set of image features (such as a second set of image features 306A) associated with the received first image 302A can be extracted by an image feature detection model (e.g., image feature detection model 110). In one or more embodiments, the circuit system 202 can be configured to extract the second set of image features 306A associated with the received first image 302A by the image feature detection model 110. In examples, the image feature detection model 110 may include at least one of a model based on scale-invariant feature transform (SIFT), a model based on accelerated robust features (SURF), a model based on oriented FAST and rotated BRIEF (ORB), or a model based on a fast approximate nearest neighbor library (FLANN). The extraction of the second set of image features 306A is, for example, in... Figure 3 Further description (at 306).

[0064] At 412, a second feature vector associated with the received first image 302A can be generated based on the extracted second image feature set 306A. In one or more embodiments, the circuit system 202 can be configured to generate a second feature vector associated with the received first image 302A based on the extracted second image feature set 306A. The generation of the second feature vector is, for example, in... Figure 3 Further description (at 306).

[0065] At 414, a third feature vector associated with the received first image 302A can be generated based on a combination of the generated first feature vector and the generated second feature vector. In one or more embodiments, the circuit system 202 can be configured to generate the third feature vector associated with the received first image 302A based on a combination of the generated first feature vector and the generated second feature vector. In the example, the generation of the third feature vector can also be based on the application of principal component analysis (PCA) transform to the combination of the generated first feature vector and the generated second feature vector. The generation of the third feature vector is, for example, in... Figure 3 Further description (at 308).

[0066] At 416, a similarity measure can be determined between the generated third feature vector associated with the received first image 302A and the fourth feature vector of each second image in the pre-stored set of second images. In one or more embodiments, the circuit system 202 can be configured to determine a similarity measure between the generated third feature vector associated with the received first image 302A and the fourth feature vector of each second image in the pre-stored set of second images. In examples, the similarity measure may include, but is not limited to, at least one of cosine distance similarity or Euclidean distance similarity. The determination of the similarity measure is, for example, in... Figure 3 Further description (at 312).

[0067] At 418, a pre-stored third image (such as a pre-stored third image) can be identified from the set of pre-stored second images based on a determined similarity metric. In one or more embodiments, the circuit system 202 can be configured to identify a pre-stored third image from the set of pre-stored second images based on a determined similarity metric. The identification of the pre-stored third image, for example, is in... Figure 3 Further description (at 314).

[0068] At 420, a display device (such as display device 210) can be controlled to display information associated with the identified pre-stored third image. In one or more embodiments, circuitry 202 can be configured to control display device 210 to display information associated with the identified pre-stored third image. Control of display device 210, for example, is... Figure 3 Further details are provided (at 314). Control can be passed to the end.

[0069] Although flowchart 400 is shown as discrete operations, such as 404, 406, 408, 410, 412, 416, 418, and 420, this disclosure is not limited thereto. Thus, in some embodiments, without departing from the essence of the disclosed embodiments, these discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the specific implementation.

[0070] Various embodiments of this disclosure may provide a non-transitory computer-readable medium and / or storage medium storing instructions executable by a machine and / or computer (such as electronic device 102). The instructions may cause the machine and / or computer to perform operations including receiving a first image (such as first image 302A). The operations may further include extracting a first set of image features (e.g., first image feature set 304A) associated with the received first image 302A using a deep neural network (DNN) model (e.g., DNN model 108). The operations may further include generating a first feature vector associated with the received first image 302A based on the extracted first image feature set 304A. The operations may further include extracting a second set of image features (e.g., second image feature set 306A) associated with the received first image 302A using an image feature detection model (such as image feature detection model 110). The operations may further include generating a second feature vector associated with the received first image 302A based on the extracted second image feature set 304A. The operations may further include generating a third feature vector associated with the received first image 302A based on a combination of the generated first feature vector and the generated second feature vector. The operation may further include determining a similarity measure between a generated third feature vector associated with the received first image and a fourth feature vector of each of the pre-stored second images in a set of pre-stored second images. The operation may further include identifying a pre-stored third image from the set of pre-stored second images based on the determined similarity measure. The operation may further include controlling a display device (such as display device 210) to display information associated with the identified pre-stored third image.

[0071] Exemplary aspects of this disclosure may provide electronic devices (such as circuit systems 202) including circuit systems (such as circuit system 202). Figure 1The electronic device 102. The circuit system 202 can be configured to receive a first image 302A. The circuit system 202 can be configured to extract a first set of image features (such as a first image feature set 304A) associated with the received first image 302A using a deep neural network (DNN) model 108. The circuit system 202 can be configured to generate a first feature vector associated with the received first image 302A based on the extracted first image feature set 304A. The circuit system 202 can be configured to extract a second set of image features (such as a second image feature set 306A) associated with the received first image 302A using an image feature detection model 110. The circuit system 202 can be configured to generate a second feature vector associated with the received first image 302A based on the extracted second image feature set 306A. The circuit system 202 can be configured to generate a third feature vector associated with the received first image 302A based on a combination of the generated first feature vector and the generated second feature vector. Circuit system 202 can be configured to determine a similarity measure between the generated third feature vector associated with the received first image 302A and the fourth feature vector of each second image in a pre-stored set of second images. Circuit system 202 can be configured to identify a pre-stored third image from the pre-stored set of second images based on the determined similarity measure. Circuit system 202 can be configured to control display device 210 to display information associated with the identified pre-stored third image.

[0072] According to an embodiment, the image feature detection model 110 may include at least one of the following: a model based on scale-invariant feature transform (SIFT), a model based on accelerated robust features (SURF), a model based on oriented FAST and rotated BRIEF (ORB), or a model based on fast approximate nearest neighbor library (FLANN), but is not limited to these.

[0073] According to an embodiment, the generation of the third eigenvector can also be based on the application of principal component analysis (PCA) transformation to a combination of the generated first eigenvector and the generated second eigenvector. According to an embodiment, the similarity measure can include, but is not limited to, at least one of cosine distance similarity or Euclidean distance similarity.

[0074] According to an embodiment, circuit system 202 can also be configured to determine a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector using a machine learning model (such as machine learning model 316) different from DNN model 108 and image feature detection model 110. Circuit system 202 can also be configured to combine the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight. Circuit system 202 can be configured to generate a third feature vector based on the combination of the generated first feature vector and the generated second feature vector.

[0075] According to an embodiment, the circuit system 202 can also be configured to receive user input, the user input including a first weight associated with a generated first feature vector and a second weight associated with a generated second feature vector. The circuit system 202 can also be configured to combine the generated first feature vector and the generated second feature vector based on the received user input. The circuit system 202 can be configured to generate a third feature vector based on the combination of the generated first feature vector and the generated second feature vector.

[0076] According to an embodiment, the circuit system 202 may also be configured to classify the received first image 302A into a first image tag from a set of image tags associated with the DNN model 108 via the DNN model 108. The circuit system 202 may be configured to determine a first count of images associated with the first image tag in a training dataset (such as training dataset 112) associated with the DNN model 108. The circuit system 202 may also be configured to determine a first weight associated with a generated first feature vector and a second weight associated with a generated second feature vector based on the determined first count of images associated with the first image tag. The circuit system 202 may also be configured to combine the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight. The circuit system 202 may be configured to generate a third feature vector based on the combination of the generated first feature vector and the generated second feature vector.

[0077] According to an embodiment, circuit system 202 can be configured to determine an image quality score associated with the received first image 302A based on at least one of an extracted first image feature set 304A or an extracted second image feature set 306A. Circuit system 202 can also be configured to determine a first weight associated with a generated first feature vector and a second weight associated with a generated second feature vector based on the determined image quality score. Circuit system 202 can also be configured to combine the generated first feature vector and the generated second feature vector based on the determined first weight and the determined second weight. Circuit system 202 can be configured to generate a third feature vector based on the combination of the generated first feature vector and the generated second feature vector. According to an embodiment, the image quality score can correspond to at least one of, but is not limited to, sharpness, noise, dynamic range, tone reproduction, contrast, color saturation, distortion, vignetting, exposure accuracy, chromatic aberration, lens flare, color moiré, or artifacts associated with the received first image 302A.

[0078] According to an embodiment, the circuit system 202 may also be configured to extract a first image 302A from the first video and to extract a pre-stored third image that can be identified and corresponds to a pre-stored second video. The pre-stored second video may be associated with the first video.

[0079] This disclosure can be implemented in hardware or a combination of hardware and software. It can be implemented in a centralized manner, in at least one computer system, or in a distributed manner, wherein different elements may be distributed across several interconnected computer systems. A computer system or other apparatus suitable for performing the methods described herein may be appropriate. The combination of hardware and software can be a general-purpose computer system having a computer program that, when loaded and executed, can control the computer system to perform the methods described herein. This disclosure can be implemented in hardware that includes a portion of an integrated circuit that also performs other functions.

[0080] This disclosure can also be embedded in a computer program product that includes all features enabling the implementation of the methods described herein and, when loaded into a computer system, is capable of executing those methods. In this context, a computer program means any expression of a set of instructions represented in any language, code, or notation, which is intended to cause a system capable of information processing to directly perform a particular function, or to perform a particular function after one or both of: a) being translated into another language, code, or notation; or b) being copied in a different material form.

[0081] While this disclosure has been described with reference to certain embodiments, those skilled in the art will understand that various changes can be made and equivalents can be substituted without departing from the scope of this disclosure. Furthermore, many modifications can be made to suit particular situations or materials to the teachings of this disclosure without departing from the scope of this disclosure. Therefore, it is intended that this disclosure be limited to the specific embodiments disclosed, but rather that it encompass all embodiments falling within the scope of the appended claims.

Claims

1. An electronic device, comprising: The circuit system is configured as follows: Receive the first image; The first image feature set associated with the received first image is extracted using a deep neural network (DNN) model; A first feature vector associated with the received first image is generated based on the extracted first image feature set; The second image feature set associated with the received first image is extracted using an image feature detection model; A second feature vector associated with the received first image is generated based on the extracted second image feature set; Based on the resolution of the received first image, determine each of the first weight associated with the generated first feature vector and the second weight associated with the generated second feature vector; The first feature vector and the second feature vector are generated by combining the determined first weight and the determined second weight; A third feature vector associated with the received first image is generated based on a combination of the generated first feature vector and the generated second feature vector. Determine a similarity measure between the generated third feature vector associated with the received first image and the fourth feature vector of each second image in the pre-stored set of second images; Identify a pre-stored third image from a pre-stored set of second images based on a defined similarity metric; as well as The control display device displays information associated with the identified pre-stored third image.

2. The electronic device according to claim 1, wherein the image feature detection model includes at least one of the following: a model based on scale-invariant feature transform SIFT, a model based on accelerated robust feature transform SURF, a model based on oriented FAST and rotated BRIEF ORB, or a model based on fast approximate nearest neighbor library FLANN.

3. The electronic device according to claim 1, wherein the generation of the third feature vector is further based on the application of principal component analysis (PCA) transformation to the combination of the generated first feature vector and the generated second feature vector.

4. The electronic device of claim 1, wherein the similarity measure includes at least one of cosine distance similarity or Euclidean distance similarity.

5. The electronic device of claim 1, wherein the circuit system is further configured as follows: A first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector are determined using a machine learning model that differs from DNN models and image feature detection models.

6. The electronic device of claim 1, wherein the circuit system is further configured to: Receive user input, the user input including a first weight associated with a generated first feature vector and a second weight associated with a generated second feature vector; The first feature vector and the second feature vector are combined based on the received user input; and The third feature vector is generated based on the combination of the first and second feature vectors.

7. The electronic device of claim 1, wherein the circuit system is further configured as follows: The received first image is classified into the first image label in the set of image labels associated with the DNN model using the DNN model; Determine the first count of images associated with the first image label in the training dataset associated with the DNN model; Based on the first count of images associated with the first image tag, a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector are determined; A first feature vector and a second feature vector are generated by combining based on a determined first weight and a determined second weight; and The third feature vector is generated based on the combination of the first and second feature vectors.

8. The electronic device of claim 1, wherein the circuit system is further configured to: An image quality score associated with the received first image is determined based on at least one of the extracted first image feature set or the extracted second image feature set; A first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector are determined based on the determined image quality score. A first feature vector and a second feature vector are generated by combining based on a determined first weight and a determined second weight; and The third feature vector is generated based on the combination of the first and second feature vectors.

9. The electronic device of claim 8, wherein the image quality score corresponds to at least one of sharpness, noise, dynamic range, tone reproduction, contrast, color saturation, distortion, vignetting, exposure accuracy, chromatic aberration, lens flare, color moiré, or artifact associated with the received first image.

10. The electronic device of claim 1, wherein the circuit system is further configured to extract a first image from the first video and extract a pre-stored third image identified corresponding to a pre-stored second video, and wherein the pre-stored second video is associated with the first video.

11. A method for reverse image search, comprising: In electronic devices: Receive the first image; The first image feature set associated with the received first image is extracted using a deep neural network (DNN) model; A first feature vector associated with the received first image is generated based on the extracted first image feature set; The second image feature set associated with the received first image is extracted using an image feature detection model; A second feature vector associated with the received first image is generated based on the extracted second image feature set; Based on the resolution of the received first image, determine each of the first weight associated with the generated first feature vector and the second weight associated with the generated second feature vector; The first feature vector and the second feature vector are generated by combining the determined first weight and the determined second weight; A third feature vector associated with the received first image is generated based on a combination of the generated first feature vector and the generated second feature vector. Determine a similarity measure between the generated third feature vector associated with the received first image and the fourth feature vector of each second image in the pre-stored set of second images; Identify a pre-stored third image from a pre-stored set of second images based on a defined similarity metric; as well as The control display device displays information associated with the identified pre-stored third image.

12. The method of claim 11, wherein the image feature detection model comprises at least one of the following: a model based on scale-invariant feature transform SIFT, a model based on accelerated robust feature transform SURF, a model based on oriented FAST and rotated BRIEF ORB, or a model based on fast approximate nearest neighbor library FLANN.

13. The method according to claim 11, wherein the generation of the third eigenvector is further based on the application of principal component analysis (PCA) transformation to a combination of the generated first eigenvector and the generated second eigenvector.

14. The method of claim 11, wherein the similarity measure includes at least one of cosine distance similarity or Euclidean distance similarity.

15. The method of claim 11, further comprising: A first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector are determined using a machine learning model that differs from DNN models and image feature detection models.

16. The method of claim 11, further comprising: Receive user input, the user input including a first weight associated with a generated first feature vector and a second weight associated with a generated second feature vector; The first feature vector and the second feature vector are combined based on the received user input; and The third feature vector is generated based on the combination of the first and second feature vectors.

17. The method of claim 11, further comprising: The received first image is classified into the first image label in the set of image labels associated with the DNN model using the DNN model; Determine the first count of images associated with the first image label in the training dataset associated with the DNN model; Based on the first count of images associated with the first image tag, a first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector are determined; A first feature vector and a second feature vector are generated by combining based on a determined first weight and a determined second weight; and The third feature vector is generated based on the combination of the first and second feature vectors.

18. The method of claim 11, further comprising: An image quality score associated with the received first image is determined based on at least one of the extracted first image feature set or the extracted second image feature set; A first weight associated with the generated first feature vector and a second weight associated with the generated second feature vector are determined based on the determined image quality score. A first feature vector and a second feature vector are generated by combining based on a determined first weight and a determined second weight; and The third feature vector is generated based on the combination of the first and second feature vectors.

19. The method of claim 18, wherein the image quality score corresponds to at least one of sharpness, noise, dynamic range, tone reproduction, contrast, color saturation, distortion, vignetting, exposure accuracy, chromatic aberration, lens flare, color moiré, or artifacts associated with the received first image.

20. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by an electronic device, cause the electronic device to perform operations, the operations including: Receive the first image; The first image feature set associated with the received first image is extracted using a deep neural network (DNN) model; A first feature vector associated with the received first image is generated based on the extracted first image feature set; The second image feature set associated with the received first image is extracted using an image feature detection model; A second feature vector associated with the received first image is generated based on the extracted second image feature set; Based on the resolution of the received first image, determine each of the first weight associated with the generated first feature vector and the second weight associated with the generated second feature vector; The first feature vector and the second feature vector are generated by combining the determined first weight and the determined second weight; A third feature vector associated with the received first image is generated based on a combination of the generated first feature vector and the generated second feature vector. Determine a similarity measure between the generated third feature vector associated with the received first image and the fourth feature vector of each second image in the pre-stored set of second images; Identify a pre-stored third image from a pre-stored set of second images based on a defined similarity metric; as well as The control display device displays information associated with the identified pre-stored third image.

Images