Multi-modal approach to search query input

Abstract

Search queries containing multiple modes of query input are used to identify responsive results. The search queries can be composed of combinations of keyword or text input, image input, video input, audio input, or other modes of input. The multiple modes of query input can be present in an initial search request, or an initial request containing a single type of query input can be supplemented with a second type of input. In addition to providing responsive results, in some embodiments additional query refinements or suggestions can be made based on the content of the query or the initially responsive results.

Description

Background technique [0001] Various methods for searching and retrieving information are known in the art, such as through search engines over wide area networks. Such methods typically utilize text-based searches. A text-based search utilizes a search query that includes one or more textual elements such as words or phrases. The text elements are compared to an index or other data structure to identify documents, such as web pages, that include matching or semantically similar text content, metadata, filenames, or other text representations. [0002] Known methods of text-based searching work relatively well for text-based documents, however they are difficult to apply to image files and data. In order to search for an image file via a text-based query, the image file must be associated with one or more textual elements such as title, filename, or other metadata or tags. Search engines and algorithms for text-based searches cannot search image files based on the content of...

AI Technical Summary

Problems solved by technology

However, such an approach may be limited for identifying text-based documents associated with the input of an image search

Images