248 results about "Text query" patented technology

The invention provides a cross-modal retrieval method for semantic and vector fusion in a data space, which belongs to the field of cross-modal information retrieval, and comprises the following steps: firstly, collecting and preprocessing multi-modal data; generating modal embedding and storing by utilizing the pre-training model; a shared semantic space is constructed, cross-modal vector alignment is optimized through comparative learning, and a modal mapping network is designed to enhance the embedding projection effect; storing the aligned embedding by using a Milvus database, and constructing an HNSW index; user text or image query is processed, text query analyzes limiting conditions to generate enhanced embedding, and image query extracts characters through OCR and fuses the characters with image features to generate embedding; in a database, through condition screening and semantic similarity calculation, a Top-K candidate item is retrieved; performing multi-modal correlation sorting on the candidate results and returning the results; according to the method, the shared semantic space is constructed, the alignment effect of different modal embedding is optimized, efficient storage and index management of multi-modal embedding are carried out, and real-time retrieval of large-scale cross-modal data is achieved.

The invention provides a multi-modal geographic positioning method based on a knowledge graph. The method comprises the following steps: graph construction: constructing a geographic knowledge graph; image positioning: calculating the entity similarity between the to-be-queried image and each sub-image so as to select an image positioning candidate sub-image, calculating the image matching similarity between the to-be-queried image and each image positioning candidate sub-image so as to determine an image positioning target sub-image, and taking the latitude and longitude coordinates of each image positioning target sub-image as a positioning result; text positioning: converting a to-be-queried text into map representation, calculating the map similarity between a text query sub-graph and each sub-graph to select a text positioning candidate sub-graph, and calculating the semantic similarity between the to-be-queried text and each text positioning candidate sub-graph, and calculating a text matching similarity corresponding to each text positioning candidate sub-graph based on the graph similarity and the semantic similarity corresponding to each text positioning candidate sub-graph so as to determine a text positioning target sub-graph, and taking latitude and longitude coordinates of each text positioning target sub-graph as a positioning result.

The invention discloses a graph-driven intelligent medical image retrieval method and system, and relates to the field of image retrieval, and the method comprises the steps: firstly, coding a text query input by a user through constructing a medical knowledge graph, querying to generate a graph-driven enhanced embedded vector containing an advanced medical concept, and querying a related graph concept set; then, performing preliminary vector similarity retrieval by using the enhanced vector, and quickly screening out Top-K preliminary candidate images; and finally, on the basis of querying a related map concept set, carrying out map concept secondary sorting on the preliminary candidate images, and accurately matching deep medical semantics of the images by calculating a concept coincidence degree. The mechanism of two-stage retrieval and map deep fusion ensures that the retrieval result is not only similar in vision, but also highly related in medical concept, so that the accuracy and intelligent level of medical image retrieval are remarkably improved.

An exemplary method for detecting deepfake images and providing customized analysis comprises: receiving, from a user, a textual user inquiry regarding an image; inputting the textual inquiry and the image into a deepfake detection model, wherein the deepfake detection model comprises: an image encoder for generating a plurality of image embeddings based on the image; a text encoder for generating a plurality of textual embeddings based on the textual inquiry; one or more layers for generating a plurality of answer embeddings; and a language model for generating a textual analysis based on the plurality of answer embeddings; and outputting the textual analysis, wherein the textual analysis includes a classification result of whether the image is fake and further includes one or more visual features in the image and one or more attributes of the one or more visual features that contribute to the classification result.

The disclosure describes utilizing an image query system to improve response accuracy and reduce computational steps resources in responding to natural language queries of input images. In various implementations, the image query system utilizes grounding information from one or more sources to determine accurate information for an input image. For example, the image query system uses a single comprehensive image prompt to obtain extensive visual image grounding information for the input image from a visual-based large generative model. Additionally, or in alternative implementations, the image query system obtains reverse image search grounding information for the input image. The image query system then cleverly utilizes the grounding information with a large generative language model to generate text query responses to image-based queries of the input image more accurately and efficiently.

A system and method for querying data sources for cybersecurity analysis is presented. The system and method include: receiving security logs from at least one data source, wherein the security logs lack pre-defined schema; generating a schema of the security logs based on at least a type of data of the security logs, wherein the generated schema includes fields of the security logs and values of the fields; embedding field vectors, wherein each field vector is a vector representation of a value of each respective field; embedding value vectors, wherein each value vector is a vector representation of a natural language description of each value in each respective field; and generating a query in a cyber language query, using an AI system, for execution on at least one target data source based, in part, on the generated schema, the embedded field vectors, and the embedded value vectors.

Systems and methods receive a selection of and filter documents for an electronic document search. A semantic vector analysis module is applied to text of the filtered documents to build respective floating point vectors for each of the documents, and the respective floating point vectors are stored to a vector database. A textual query is received and vectorized to produce a query vector representing search criteria. The vector database is searched in accordance with the query vector to identify similar documents that satisfy a similarity measure. A ranked list of vectors that include highest similarity scores relative the query vector is generated and similar documents are identified. A neural network is used to process document data of the similar documents to generate a textual response to the textual query, and control signal(s) are transmitted to a user device to initiate displaying the generated textual response and question answer events.

The invention relates to a weak supervision online video moment positioning method and system based on memory perception, and belongs to the technical field of artificial intelligence, and the method comprises the steps: carrying out the multi-modal feature fusion of a given video and a text query thereof, and obtaining the unified frame level representation at each stage; inputting the fused features into an offline module and an online module in an integral and frame-by-frame manner by using an offline guide online model architecture; in the off-line module, generating a Gaussian mask to reconstruct query of a covered part of words, and obtaining a proposal of an action starting moment; in the on-line module, the long-term historical memory in the window is used for enhancing the score, the attention weight of the score in the window is dynamically generated, and the score of the current frame is calculated in a weighted mode; taking the proposal obtained by the offline module as a pseudo tag, and providing supervision information for the score sequence of the online module; and high-performance weak supervision on-line moment positioning can be completed only by independently deducing the on-line module. The expansion capability and the application value of the model are remarkably improved.

The invention discloses a landslide disaster early warning method and device based on deep learning, a medium and a product, and relates to the field of geological disaster monitoring. Firstly, a remote sensing landslide data set is obtained and preprocessed, and a multi-modal data set is constructed; each sample in the multi-modal data set comprises a remote sensing image, a text query, a non-text query and a corresponding mask; constructing a deep learning model, wherein the deep learning model comprises an encoder, a DenseASPP module, a decoder and a prediction head; the encoder comprises an image encoder, a text encoder and a visual sampler; dividing the multi-modal data set into a training set, a verification set and a test set which are respectively used for training, verification and testing of a deep learning model, and taking the trained deep learning model as a landslide extraction model; the landslide extraction model is adopted to identify the landslide area in the remote sensing image, and the landslide disaster early warning is carried out according to the landslide area change, so that the efficiency and precision of landslide area identification and landslide disaster early warning can be remarkably improved.

The invention relates to the technical field of speech recognition and natural language processing, and particularly discloses a law enforcement recorder audio semantic retrieval method and system based on retrieval enhancement, and the system comprises a data acquisition module which is used for acquiring audio data and text query; the voice adapter module is used for projecting the audio data to a text embedding space to obtain voice representation; the cross-modal retriever is used for carrying out cross-modal retrieval on the voice representation and the text query to obtain voice Tokens; the voice language model is used for obtaining character hypotheses; the query generation module is used for extracting query fragments possibly containing entity names; the entity retrieval module is used for retrieving an entity database according to the query fragment to obtain a related entity name; the context construction module is used for constructing context information; and the large language model is used for obtaining a semantic retrieval result. According to the method, the identification and retrieval accuracy of the entity name and key information in the law enforcement recorder audio is improved.

Systems and methods for multimodal query suggestion can include obtaining image data, determining text strings associated with the image data, and providing the text strings as selectable options for performing a multimodal query with the image data. The text strings can be determined based on performing object detection and classification on the image data. The object classifications can then be leveraged for determining potential text queries a user may select for obtaining additional information about the classified object.

A computer-implemented method for performing searches in a document database is disclosed. The method comprises automatically detecting a line of business associated with a user, receiving a text query from the user, and generating a query embedding from the text query. The method further comprises scoring entries in a reverse index using a hybrid scoring function. The reverse index comprises titles, title embeddings, sentences, sentence embeddings, and entity tags corresponding to documents in the document database. The hybrid scoring function is used to generate a score based both on a keyword match score between the text query and the reverse index and on a cosine similarity score calculated from embeddings in the text query and in the reverse index. The method also comprises ranking scores for sentences in the document database, and displaying a sentence associated with a top score to the user.

Embodiments of the present invention provide systems, methods, and computer storage media for a visual and text search interface used to navigate a video transcript. In an example embodiment, a freeform text query triggers a visual search for frames of a loaded video that match the freeform text query (e.g., frame embeddings that match a corresponding embedding of the freeform query), and triggers a text search for matching words from a corresponding transcript or from tags of detected features from the loaded video. Visual search results are displayed (e.g., in a row of tiles that can be scrolled to the left and right), and textual search results are displayed (e.g., in a row of tiles that can be scrolled up and down). Selecting (e.g., clicking or tapping on) a search result tile navigates a transcript interface to a corresponding portion of the transcript.

Proposed is an electronic device for at least one of video moment retrieval and highlight detection which includes a storage unit and a processor, wherein the processor obtains a plurality of first video features from a video, obtains a text query feature from a text query, obtains a plurality of weights from the plurality of first video features and the text query feature, obtains a plurality of second video features from the plurality of weights and the plurality of first video features, obtains a plurality of third video features from the plurality of second video features by using an encoder, obtains a plurality of fourth video features from the plurality of third video features and a time query by using a decoder, and selects at least one of time points or time periods of the video by using the plurality of fourth video features.

A method for technical issue resolution. The method includes: receiving, from a user, a text query concerning a technical issue; obtaining query-related context relevant to the text query; and processing, through a large language model (LLM), the text query and the query-related context to produce a multimodal query response used by the user to address the technical issue. More specifically, embodiments described herein utilize text topic and zero shot classification models to translate multimodal technical documentation (e.g., including text and images) into topic relevant metadata; and process queries, pertaining to technical issues, using a multimodal LLM provided with query-related text and image context derived from said topic relevant metadata.

The invention belongs to the technical field of visual positioning, and relates to a visual positioning method based on semantic comprehension and attribute distinguishing enhancement. The framework mainly comprises a feature coding module, a semantic sensitive data enhancement module, a fine-grained attribute guiding module and a multi-stage cross-modal decoder module. Specifically, the feature coding module performs feature coding on an input graph. The semantic sensitive data enhancement module generates a plurality of queries which are consistent with long text semantics by keeping the consistency of spatial relation words in combination with a large language model, so that a training data set for the long text is expanded. The fine-grained attribute guiding module extracts attribute prior information from a text query in combination with a text graph model and an image encoder, constructs a visual feature representation with higher discrimination by using the information guiding model, and generates a target query with attribute difference at the same time.

The invention discloses a target-level three-dimensional point cloud cross-modal semantic retrieval method and device and electronic equipment. The target-level three-dimensional point cloud cross-modal semantic retrieval method comprises the steps that independent three-dimensional target point clouds are segmented from three-dimensional scene point clouds, and unique identifiers are given to the independent three-dimensional target point clouds; projecting each target point cloud to three orthogonal two-dimensional observation planes to generate a multi-view composite image; a pre-trained vision-language basic model is adopted to code and fuse the synthesized image, and a unified multi-modal feature vector is generated; a vector database in which the feature vectors are associated with their identifiers is constructed. And encoding a natural language query text into a text query vector by using the model, calculating the semantic similarity between the text query vector and the feature vector in the database, and positioning and returning the corresponding three-dimensional target point cloud. According to the method, accurate and efficient cross-modal retrieval from a natural language to a three-dimensional point cloud target is realized, the problem that a traditional method is difficult to support semantic fine-grained retrieval of the three-dimensional target point cloud is solved, and a key technical support is provided for training data management in the fields of intelligence and the like.

A three-dimensional scene local editing method based on text control comprises the steps that a three-dimensional Gaussian semantic field is constructed based on text query, and accurate three-dimensional object masks are directly generated by embedding semantic features into anisotropic Gaussian primitives and combining multi-view semantic supervision and cosine similarity calculation of a CLIP model; screening an effective view angle by taking a target object as a center, determining an effective view angle range through azimuth clustering analysis and kernel density estimation, and generating a continuous dense surround-view camera track; an image grid technology is adopted to organize and edit visual angles, a submerged space alignment module GLAM fusing global-local attention is utilized to synchronously process multiple frames of images, style benchmarks are unified and geometric consistency is enhanced through a cross-visual-angle attention mechanism, a multi-visual-angle consistent editing result is generated, and only Gaussian parameters corresponding to a target object are finely adjusted, so that the target object can be edited more accurately. And the background area is kept unchanged. According to the method, the boundary definition of local editing is remarkably improved, and efficient and accurate three-dimensional content modification of a multi-target complex scene is achieved.

A set of locations in a data object to be annotated is identified as corresponding to metadata of the data object. A natural language text query is generated using the metadata of a data object. A set of scores is generated for the set of locations using a generative neural network, and the set of scores indicate whether individual candidate locations satisfy the natural language text query. Based on the set of scores, a location in the data object is annotated to generate an annotated location as corresponding to the metadata.

The invention discloses a Texturesql method and device based on a large model and a storage medium. A query problem is obtained, and a text query condition and an SQL query condition are generated based on the query problem; data knowledge enhancement is conducted on the text query condition and the SQL query condition through vector knowledge bases, data knowledge enhancement results are obtained, and the vector knowledge bases comprise a text vector knowledge base and an SQL vector knowledge base; inputting the data knowledge enhancement result into an LLM inference model to generate an SQL task; and executing the SQL task. According to the method, the context information represented by the query problem is enhanced by building the text knowledge base and the SQL knowledge base, the problem data represented by the query problem is accurately positioned, the more accurate SQL task is generated through reasoning of the SQL large model, and the technical effect of improving the accuracy of generating the SQL query statement is achieved.

The system receives a voice query at an audio interface and converts the voice query to text. The system can determine pronunciation information during conversion and generate metadata that indicates a pronunciation of one or more words of the query, include phonetic information in the text query, or both. A query includes one or more entities that may be more accurately identified based on pronunciation. The system searches for information, content, or both among one or more databases based on the generated text query, pronunciation information, user profile information, search histories or trends, and optionally other information. The system identifies one or more entities or content items that match the text query, and retrieves the identified information to provide to the user.

Recent developments in machine learning (commonly coined “artificial intelligence” or “AI”) have vastly expanded applications for this technology, such as myriad “chat” agents adept at understanding natural human language. While state of the art generative models can parse text queries from a user and provide comprehensive, accurate responses (including generating images depicting desired content), current implementations struggle with understanding all information present in images of documents, especially images of business documents. In particular, generative models fail to understand structured and semi-structured information, e.g., as indicated by graphical information such as lines, geometric relationships (e.g., indicated by tables, graphs, figures, etc.), formatting, and other contextual information that human readers easily and implicitly understand. The disclosed inventive concepts transform structured and semi-structured information along with textual content into a textual representation that allows generative models to better understand textual content and non-textual structured information present in document images.

Systems and methods for orchestrating interaction with an artificial intelligence (AI) application in a contact center environment receive, via an AI agent, a voice message from a user; convert the message from voice to text; generate an initial computational inference process based on the text message; determine whether or not all information required to execute the initial computational inference process is available to the processor; when a determination is made that all information required is available: execute the initial computational inference process; generate a text reply based on the initial computational inference process; convert the text reply to a voice reply; and send the voice reply to the user via the AI agent; when a determination is made that information is unavailable: generate a text query requesting the information; convert the text query to a voice query; and send the voice query to the user via the AI agent.

The invention relates to the technical field of image texts, in particular to an image text alignment method based on a multi-modal large language model, which comprises the following steps of: segmenting an image into local areas and encoding the local areas into visual feature vectors by adopting a pre-trained visual encoder; visual and text features are mapped to the same dimension space through a learning projection layer, an InfoNCE loss function is adopted to maximize the similarity of a positive sample pair, dynamic interaction between text query and an image area is achieved through a cross attention mechanism, language module parameters are frozen to pre-train a visual encoder, an overall module is combined to be fine-tuned, and text query is completed. An error case is analyzed, and a verified and improved alignment module is obtained through a data enhancement optimization module. The method solves the problems that a traditional image text alignment method is incomplete in manual feature extraction and low in precision, an existing deep learning model is limited in cross-modal semantic understanding ability, and the generalization ability is poor when facing long-tail data.

The invention discloses a field generalization semantic segmentation enhancement method based on an efficient multi-scale and simple attention module, and belongs to the field of computer vision. According to the invention, the model is based on a TQDM (Text Query-Driven Mask Transfer) framework, and an EMA (Efficient Multi-Scale Attention Module) and a SimAM (Simple Parameter-free Attention Module) are fused, so that the adaptability and the robustness of a semantic segmentation task on a plurality of domains are improved. According to the invention, by introducing the EMA module, multi-scale feature aggregation and cross-space information interaction are realized, so that the understanding ability of the model to a complex scene is enhanced; meanwhile, in combination with a SimAM module, feature expression is optimized under the condition of not increasing extra parameters, and the precision of small target segmentation and target boundary detection is improved. The method can be widely applied to computer vision tasks related to cross-domain semantic segmentation, such as automatic driving, intelligent monitoring and medical image analysis.

To-be-queried information is received by a server device. The to-be-queried information is transmitted from a terminal device. A multimodal information search is performed by the server device based on the to-be-queried information to obtain multimodal search results. A content digest extraction of a target text in the multimodal search results is performed to obtain one or more content digest fragments. Based on the one or more content digest fragments, a text query result is generated. When a first preset number of search results in the multimodal search results lack a rich media query result, key information extraction on the text query result is performed to obtain key description information. A supplemental rich media query result is acquired according to the key description information. The text query result and the supplemental rich media query result are fused into a target query result that is transmitted to the terminal device.

The present disclosure describes techniques for generating video descriptions using a machine learning model. A plurality of sets of visual tokens corresponding to a plurality of frames of a video is generated. A first type of tokens is generated by implementing temporal pooling on the plurality of sets of visual tokens corresponding to the plurality of frames. A second type of tokens is generated by compressing each of the plurality of sets of visual tokens corresponding to each of the plurality of frames. A third type of tokens is generated by applying cross-attention between each of the plurality of sets of visual tokens and a fourth type of tokens including text tokens generated based on an input text query. A text description of the video is generated based on the first type of tokens, the second type of tokens, the third type of tokens, and the fourth type of tokens.

The invention discloses a materialized view construction method, a data processing system, a medium and a program product, and relates to the field of electrical digital data processing.The method comprises the steps that text query information is received, the execution sequence and the execution type of query operation in the text query information are determined, and a logic plan is generated; extracting a scanning operation, a projection operation, a connection operation and an aggregation operation from the logic plan to obtain a query sub-plan representing query semantics; according to the operation type of the query sub-plan, converting the query sub-plan into an initial plan slice; normalizing the initial plan film to generate a standard plan film in a preset format; according to a preset aggregation strategy, performing deduplication merging processing on the standard plan slices to obtain a target plan slice; and generating a table establishment statement of the materialized view based on the target plan film to construct the materialized view. By implementing the method, generation of the materialized view can be optimized, and the redundancy of the materialized view is reduced.

The invention discloses an unsupervised cross-modal Hash retrieval method based on multistage interaction, which belongs to the technical field of cross-modal Hash retrieval, and mainly comprises the following steps of: preprocessing cross-modal data; an unsupervised cross-modal Hash retrieval network based on multistage interaction is constructed; network training: inputting the image-text training samples into the constructed network in batches for network training; and modal retrieval: inputting the image-text query sample set and the retrieval sample set into the trained multi-level interaction-based unsupervised cross-modal Hash retrieval network, respectively generating corresponding Hash codes, and obtaining a query result by calculating the Hamming distance between the Hash codes of the query sample and the retrieval sample. The minimum Hamming distance is the final query result. According to the method, high-dimensional multi-modal features can be compressed into compact binary codes, the cross-modal retrieval efficiency is remarkably improved, and the method can be used for real-time image search and cross-modal recommendation systems.

A concept for behavior planning of a vehicle in a traffic scene is proposed that extends the planning capabilities of a rule-based planner so that the rule-based planner can also be used for behavior planning in more complex scenarios. A rule-based planner (50) is used which, on the basis of a scene representation (2) of the traffic scene (1), carries out a behavioral planning (6) which simulates at least one planning scenario from a planner-specific set of predetermined planning scenarios, According to the invention, a text-based neural network (30) is used to select the at least one planning scenario. For this purpose, at least one text query (3) describing the traffic scene is generated for the neural network (30) based on the scene representation (2), with which a behavioral recommendation (4) is requested that is supported by the planner-specific set of predefined planning scenarios. Based on the text query (3), the neural network (30) generates a text-based behavioral recommendation (4) that is assigned to at least one of the predefined planning scenarios. The behavioral planning (6) of the rule-based planner (50) is then based on the planning scenario thus selected.