61 results about "Multimedia information retrieval" patented technology

A three dimensional user interface allows browsing of a database displayed as a three dimensional information space. The data is organized along three axes. A current plane or layer of data is summarized on an information landscape, with different planes being selectable using a tower that is located at the intersection of the three axes. A control wall with incorporated tools is used to formulate database queries. A preview wall previews selected data. The preview wall also provides transition between previewing the searches in the 3D space and actual viewing of the programming in full screen 2D display. Application to TV programming data is shown.

The invention belongs to the field of multimedia information retrieval and relates to a method for realizing thesaurus-based query expansion and sort in image retrieval. The method comprises a WordNet-based English word semantic similarity metric algorithm, a HowNet-based Chinese word semantic similarity metric algorithm, an expansion rule-based query expansion word selection and optimization algorithm and a retrieval result evaluation and optimization algorithm. In the method, an image search engine is improved by the relevant text processing method and the relevant semantic network dictionary; and the retrieval result is sorted through semantic expansion, user interaction and improved similarity measurement. Compared with the traditional method, the method has the advantages of high accuracy rate, high integrality and low space-time cost. The method has very important significance for performing high-efficiency image retrieval according to image high-layer semantic information and on the basis of a large-scale image data set, and has wide application value in the field of cross-linguistic and cross-media retrieval.

Multimedia information retrieval is performed using meta-descriptors in addition to descriptors. A “descriptor” is a representation of a feature, a “feature” being a distinctive characteristic of multimedia information, while a “meta-descriptor” is information about the descriptor. Meta-descriptors are generated for multimedia information in a repository (10, 12, 14, 16, 18, 20, 22, 24) by extracting the descriptors from the multimedia information (111), clustering the multimedia information based on the descriptors (112), assigning meta-descriptors to each cluster (113), and attaching the meta-descriptors to the multimedia information in the repository (114). The multimedia repository is queried by formulating a query using query-by-example (131), acquiring the descriptor/s and meta-descriptor/s for a repository multimedia item (132), generating a query descriptor/s if none of the same type has been previously generated (133, 134), comparing the descriptors of the repository multimedia item and the query multimedia item (135), and ranking and displaying the results (136, 137).

The invention belongs to the technical field of multimedia information retrieval, and specifically discloses a deep learning-based freehand sketch image retrieval method. According to the method, and edge contour detection technology and a non-maximum value suppression technology are utilized to realize the conversion from colored images to similar sketch images, a deep learning technology is utilized to construct distinguishing feature expressions for querying deep features of sketches and similar sketches, the deep features fuse the high-level semantic features and low-level visual features of images, and the deep features are more distinguishing in sketch retrieval. Through deeply mining visual information of a first retrieval result, uncorrelated images placed at the front in the retrieval result are rejected and a more correlated result is returned to the users. The method is high in correctness and strong in adaptability. On the basis of large-scale image data, the method is significant in carrying out efficient image retrieval which considers semantic information of sketches, so that the influences of fuzziness of freehand sketches can be decreased, the retrieval correlation can be improved and the user experience can be enhanced; and the method has an extensive application value in the field of multimedia image retrieval.

A multimedia information retrieval system includes a storage and an electronic processing device. The latter is configured to perform a process including: computing values of a pairwise similarity measure quantifying pairwise similarity of documents of a multimedia reference repository; storing the computed values in the storage; performing an initial information retrieval process respective to the multimedia reference repository to return a set of initial repository documents; and identifying a set of top ranked documents of the multimedia reference repository based at least on the stored computed values pertaining to the set of initial repository documents.

A system and method for information retrieval are disclosed. The method includes querying a multimedia collection with a first component of a multimedia query (e.g., a text-based part of the query) to generate first comparison measures between the first component of the query and respective objects in the collection for a first media type (e.g., text). The multimedia collection is queried with a second component of the multimedia query (e.g., an image-based part of the query) to generate second comparison measures between the second component of the query and respective objects in the collection for a second media type (e.g., visual). An aggregated score for each of a set of objects in the collection is computed, based on the first comparison measure and the second comparison measure for the object. This includes applying an aggregating function to the first and second comparison measures in which a first confidence weighting is applied to the first comparison measure and a second confidence weighting is applied to the second comparison measure. The first confidence weighting is independent of the second comparison measure. The second confidence weighting is dependent on the first comparison measure. Information based on the aggregated scores is output.

The invention provides an image segmentation method based on an annotated image learn. The method comprises two processes of: 1, learning an annotated training sample, namely segmenting the training image, performing scene classification on the training image, and establishing connection between the annotated words and the segmentation region on a special scene; and 2, determining the annotated words of the region to be segmented according to a model parameter acquired by learning in the process 1, performing information fusion according to the annotated information of the region and finishing segmentation. According to the method, the image segmentation and the identification process are fused by learning the annotated image; the annotated words serve as connecting link of the image segmentation and object identification; connection is established between low-grade visual stimulation and the annotated words representing high-grade semantic information to guide the image segmentation process, so that the cognitive ability of the image segmentation result is improved. The method can be directly applied to the actual application fields such as automatic image annotation, computer-aided diagnosis of a medical image, segmentation and classification of remote sensing images, multimedia information retrieval and the like.

A video information retrieval system comprising a client system having a request issuer for issuing a search request in respect of desired video material; and a video accessor for accessing video material on the basis of a uniform resource locator (URL) and a SMPTE unique material identifier (UMID). The retrieval system also comprising a server system having access to one or more databases containing metadata information relating to a plurality of video material items, a UMID associated with each video material item and at least one URL associated with each UMID. A receiver is provided for receiving a search request from the client system and detecting one or more video material items for which metadata information stored in at least one of the database(s) substantially corresponds to the search request. An information supplier supplies the metadata information, the URL and the UMID relating to the one or more detected video material items to the client system. The server system has at least one video repository having: a video storage arrangement storing video material and associated UMID data. The metadata, the URL and the UMID are communicated between the server and the client using a markup language having descriptors for data content.

A record management system retrieves relevance information through an information retrieval model that models relevance between users, queries, and records based on user interaction data with records. Relevance information between different elements of the record management system are determined through a set of learned transformations in the information retrieval model. The record management system can quickly retrieve relevance information between different elements of the record management system given the set of learned transformations in the information retrieval model, without the need to construct separate systems for different types of relevance information. Moreover, even without access to contents of records, the record management system can determine relevant records for a given query based on user interaction data and the determined relationships between users, queries, and records learned through the information retrieval model.

A search system (200) for a database (224) including records having a multiple disparate types of media is provided. The search system supports queries, that include different types of search criteria, including content based retrieval search criteria. A fuzzy logic method (400) is provided for effectively combining the results of different types of search criteria. The fuzzy logic method also allows confidence levels entered by the user for search criteria to be considered in combining results. Retrieval relevance values for documents for at least some search criteria are used in the fuzzy logic method. For content based image retrieval searches, the retrieval relevance values are computed by mapping a distance between quantitative characterizations of a search basis image, and other images into a finite range.

Disclosed is a method and device for selecting documents, such as Web pages or sites, for presentation to a user, in response to a user expression of interest, during the course of presentation to the user of a document, such as a video or audio selection, whose content varies with time. The method takes advantage of information retrieval techniques to select documents related to the portion of the temporal document in which the user has expressed interest. The method generates the search query to use to select documents by reference to text associated with the portion of the temporal document in which the user has expressed interest, as by using the closed caption test associated with the video, or by using speech recognition techniques. The method further uses a weighting function to weigh the terms used in the search query, depending on their temporal relationship to the user expression of interest.

The invention discloses a multimedia information searching method and system. The method includes the steps of extracting feature data of current multimedia information, obtaining a feature bit vector of the current multimedia information according to the extracted feature data, segmenting the feature bit vector of the current multimedia information to obtain k sub-vectors of the current multimedia information, determining a candidate set corresponding to each sub-vector according to each sub-vector of the current multimedia information, finding out a feature bit vector, corresponding to each vector identification in each obtained candidate set, in a multimedia feature database, calculating hamming distances between the feature bit vector of the current multimedia information and the found feature bit vectors, and outputting the multimedia information, corresponding to the feature bit vectors with the hamming distances conforming to a preset condition, as a search result. Through the method and due to the fact that a segmentation index structure is set up to search the feature bit vectors, the searching speed of the multimedia information and the searching efficiency of the multimedia information can be greatly improved.

The invention relates to a method and system for adsorbing multimedia information through track points, and belongs to the technical field of computer geographical information systems. In an existing geographical information system, generally speaking, track point information can only show the movement attribute of a target. The method includes the following steps: 1, responding to a map operation to obtain query conditions; 2, querying all track points of the target according to the query conditions, and drawing a track on a map; 3, querying all the related multimedia information according to the coordinates and geographical radii of the track points; 4, classifying and aggregating the related multimedia information; 5, hooking the multimedia event information to a time axis according to the sequence of occurrence time. By adopting the method and system, the related information such as videos, voices, pictures and texts around the track points of the target can be conveniently and rapidly shown on the electronic map, and then information of the track points of the target can be closely and sequentially related to the multimedia information of events occurring around the track points of the target.

The invention discloses a deep learning-based sketch retrieval method, and relates to the technical field of multimedia information retrieval. The method comprises the steps of firstly calculating a conventional picture edge probability graph and obtaining a feature descriptor of the edge probability graph to realize conversion from a color conventional graph to a similar freehand image; establishing a feature library required for freehand image retrieval through a convolutional neural network; and extracting depth features of edge graphs of different levels of the similar freehand image and adrawn sketch by utilizing a deep learning technology to perform similarity matching. By providing new feature extraction and matching methods, the sketch drawn by a user can be understood more accurately; the method has high accuracy and high adaptability; the influence of fuzziness of the freehand sketch can be reduced; the retrieval correlation is improved; the user experience is enhanced; andthe method has wide application values in the field of multimedia image retrieval.

The invention discloses a recommendation method and device for multimedia information and belongs to the technical field of Internet. The recommendation method comprises: determining first multimedia classes interested by a first user according to a historical click action of the multimedia information; determining second multimedia classes which are potentially interested and are not historically clicked by the first user according to each first multimedia class of the first user and a plurality of historical click actions on the multimedia information by the second user; determining a target multimedia class according to the first multimedia classes interested by the first user and the second multimedia classes which are potentially interested by the first user; and recommending the multimedia information of the target multimedia class. When the information is recommended, the determined target multimedia class is more accurate by realizing the multimedia classes of the multimedia information historically clicked by the first user and realizing the multimedia classes which are potentially interested by the first user, so that the clicking rate of the recommended multimedia information can be improved and resources and financial resources can be saved.

A method and apparatus for encoding knowledge using a multimedia network. A multimedia network represents semantic concepts and their relations using multimedia content. A multimedia network associates words and multimedia content with the semantic concepts in order to illustrate and exemplify the semantic concepts as well as describe lexical, semantic, and perceptual relations. The multimedia network can be searched, browsed, or summarized for purposes of information discovery. The multimedia network can also be used for personalizing multimedia content or for querying a multimedia information repository by expanding a query to include related concepts encoded in a Multimedia network.

The present invention discloses a multimedia classification recommendation method, apparatus and system. The method comprises: receiving state information uploaded by a client; matching the state information with a preset state tag, and determining one or more state tags matched with the state information; performing a multimedia information retrieval according to the one or more state tags, so as to obtain multimedia information corresponding to the one or more state tags; and pushing the multimedia information corresponding to the one or more state tags to the client. A current behavior state and a mood state such as a metal condition of a user are analyzed by means of the state information, and a video that is suitable for being watched under the current state is pushed, so as to make video recommendation more personalized and more coincide with the behavior and internal feelings of the user.

The invention discloses an interactive type image retrieval method of combining user evaluation and labels, and belongs to the field of multimedia information retrieval. The method utilizes a comprehensive retrieval method based on the combination of physical characteristics of images and text. In the process of retrieval, a user is allowed to carry out text information description on query images or select keywords provided by a system. By carrying out relevant evaluation of 'satisfied' or 'unsatisfied' on retrieval results, an image retrieval system automatically carry out text marks on relevant satisfied images which are marked by the user to form high-level semantic information. Along with constant use of the user, the system can generate a rich semantic information database. Difference of different users to the text marks of the same image and difference of the same user to the text marks of the same image in different times are considered, and reliability of the users is combined in the process of generating the semantic information database. In retrieval, the comprehensive retrieval method based on the combination of the characteristics and the texts is utilized to carry out retrieval on the query images with semantic information, so that accuracy of the retrieval results is improved. The interactive type image retrieval method of combining the user evaluation and the labels has the advantages of being high in efficiency, high in accuracy and friendly in interactive mode.

The invention discloses an image retrieval method, and belongs to the field of intelligent information processing such as multimedia information retrieval, mode identification and the like. A correctly-matched correlated image is obtained by using geometric verification after initial retrieval, weight adjustment is performed on document vectors of the correlated image and an inquiry image to construct a new inquiry vector so as to obtain extended inquiry, and new retrieval is performed to obtain a retrieval result. According to the method, weights of implicit visual words existing in the correlated image are added in the inquiry vector, so that the weights of the same visual words in the inquiry image and the correlated matched image are increased, and the retrieval efficiency is increased greatly.

The invention discloses a method for reordering image or video search, which relates to the field of multimedia-oriented information retrieval. The method comprises the following steps of grading an image sample set into a grade A, a grade B and a grade C according to the degrees of inquiry topic relevance; constructing a relevance diagram, an irrelevance diagram and a global diagram; acquiring a relevance divergence, an irrelevance divergence and a global divergence; constructing a target function according to the relevance divergence, the irrelevance divergence and the global divergence, and acquiring a novel characteristic vector of an image sample; inputting the novel characteristic vector of a marked image sample serving as a training set into a training model to obtain a trained ordering model; and ordering the image sample through the trained ordering model and outputting an ordering result. The invention discloses a dimensionality reduction method which belongs to the field relevant to multimedia retrieval and ordering. According to the method, specific properties of data are utilized fully on the premise of limited monitoring information, the ordering performance can be improved by effectively utilizing a small number of marks, and searching precision is increased.

The invention discloses an image-text cross-modal retrieval method, system and device and a storage medium, and the method comprises the following steps: obtaining to-be-retrieved character/picture data, and obtaining similarity information by combining the character/picture data and a retrieval model trained by adopting a combined loss function; and obtaining corresponding picture/text data according to the similarity information. According to the invention, the to-be-retrieved character/picture data is input into the retrieval model to calculate the similarity matrix, corresponding picture/text data is obtained according to the similarity matrix; due to the fact that the retrieval model is trained through the combined loss function, the closer distance between the relevant pictures and the relevant texts can be kept, the retrieval model is far away from irrelevant data, the retrieval accuracy between the pictures and the relevant texts is greatly improved, and the method can be widely applied to the technical field of multimedia information retrieval.