46results about "Multimedia data indexing" patented technology

This application discloses a method and apparatus for diagnosing skin diseases based on cross-modal hash retrieval and RAG. The method includes: inputting skin disease data into a diagnostic model; when the input is unimodal image data, optimizing the hash table according to lesion features and retrieval performance, and obtaining a first retrieval result based on the hash table; when the input is multimodal image-text pair data, mapping the data of each modality to a hash code of each modality through a trained second hash function, performing cross-modal hash retrieval based on the hash code of each modality to obtain cross-modal retrieval results, and fusing multiple cross-modal retrieval results to obtain a second retrieval result; generating a diagnostic report based on prompts corresponding to the first or second retrieval result using a large language model. This method enhances feature extraction capabilities, improves the accuracy of boundary and small target recognition, supports efficient retrieval in dynamic data environments, adapts to continuously added data, and integrates multimodal information for retrieval, improving the comprehensiveness and accuracy of diagnosis.

The application discloses a multi-source education data search method and device, equipment and a storage medium, and the method comprises the steps of: obtaining education information from a plurality of heterogeneous data sources through a hybrid search strategy, and performing recursive data association on the education information to generate an extended information set; performing data screening on the extended information set based on a pre-trained model to obtain a screened extended information set; receiving a user query request, using a multi-modal hybrid retrieval model to search information in the screened extended information set, and obtaining a search result. Compared with the prior art, the application eliminates information silos and improves the education information search matching accuracy by obtaining education information from a plurality of heterogeneous data sources through a hybrid search strategy, performing recursive data association on the education information to generate an extended information set, receiving a user query request, and using a multi-modal hybrid retrieval model to search information in the screened extended information set.

Some cases involve retrieval technology field, information retrieval method, device, electronic equipment, storage medium and program product are disclosed, the method comprises: obtaining first information;Obtain the first feature vector of the first information and the first discrete code word of the first information;Contrast the first discrete code word and the second discrete code word, obtain the first contrast result, the second discrete code word corresponds to the second information;Filter the second information by using the first contrast result, obtain the third information;Contrast the first feature vector and the second feature vector, obtain the second contrast result, the second feature vector corresponds to the third information;Filter the third information by using the second contrast result, obtain the fourth information, the fourth information is used to represent the retrieval result of the first information. Through the implementation of the information retrieval method, the retrieval accuracy and the overall retrieval efficiency can be improved.

Example methods and systems for indexing fingerprints are described. Fingerprints may be made up of sub-fingerprints, each of which corresponds to a frame of the media, which is a smaller unit of time than the fingerprint. In some example embodiments, multiple passes are performed. For example, a first pass may be performed that compares the sub-fingerprints of the query fingerprint with every thirty-second sub-fingerprint of the reference material to identify likely matches. In this example, a second pass is performed that compares the sub-fingerprints of the query fingerprint with every fourth sub-fingerprint of the likely matches to provide a greater degree of confidence. A third pass may be performed that uses every sub-fingerprint of the most likely matches, to help distinguish between similar references or to identify with greater precision the timing of the match. Each of these passes is amenable to parallelization.

The application discloses an interactive system and method based on digital audio and video family tree and family memory simulation, and aims to solve the problems of weak interaction of traditional family tree, rigid digital clone model and split memory personality. The system constructs an interactive family tree knowledge base containing member nodes, blood relationship and related data, and forms a structured family knowledge graph in multiple dimensions. The model graph co-evolution is proposed: multi-modal data is collected through daily interaction, incremental learning and knowledge graph updating are adopted in parallel, parameter efficient fine-tuning is used for lightweight learning of the digital clone, only a small amount of parameters are updated to overcome forgetting, and new event relationships are extracted to update the graph. Through bidirectional indexing, the personality and memory semantics are consistent, the multi-modal interaction is relationship-aware and dynamically evolved. The application synchronously updates the personality and memory, solves the problem that the digital clone cannot continuously evolve with the family, and realizes stereoscopic inheritance of family memory.

Various embodiments provide for systems, methods, and computer-readable storage media for annotating a collection of media items, such as digital images. According to some embodiments, an annotation system automatically determines one or more annotations for a plurality of media content items, and generates a collection of media content items that associates the determined annotations with the plurality of media content items. Depending on the embodiment, annotations that may be determined for the plurality of media content (and associated with the collection for the media content items) can include, without limitation, a caption, a geographic location, a category, a novelty measurement, an event, and a highlight media content item representing the collection.

The present disclosure relates to a multimedia processing method and device, electronic equipment and storage medium. The method comprises: in response to an associated information display operation for target multimedia information published by a current account, displaying an associated information adding page; the associated information adding page displays associated information matched with the target multimedia information; the associated information is obtained according to the content of the target multimedia information and audience account information matched with the target multimedia information; in response to a trigger operation for the displayed associated information, determining target associated information; and in response to an associated information adding operation for the target multimedia information, displaying the target multimedia information added with the target associated information. Compared with the traditional technology which requires the user to independently judge and select the associated information related to the multimedia, the associated information of the present disclosure can be determined according to the content of the multimedia and the audience, thereby improving the association degree of the associated information and the multimedia.

The invention relates to the technical field of multi-modal corpus construction, in particular to a multi-modal corpus construction method and system. Comprising a multi-source heterogeneous data acquisition module, a multi-modal preprocessing and analysis module, a cross-modal automatic alignment and association module, a translation feature labeling and management module, a structured storage and index module, an AI intelligent retrieval and teaching application module, a semantic optimization multi-modal adaptation module and a user display terminal. The multi-source heterogeneous data acquisition module comprises a data source interface unit and a metadata extraction unit; a multi-modal corpus system deeply integrated with a translation theory is constructed to directly serve the core demand of a translation subject, and a triple screen array mode is adopted to cooperate with display, so that display content is reasonably distributed, a user can check and operate in multiple dimensions, the use experience and the working efficiency are greatly improved, and the user experience and the working efficiency are improved. And various requirements of translation teaching and the like are met.

This application provides a method for generating and identifying metadata, as well as an electronic device. The method includes: first, acquiring media content; then, generating metadata for the media content, the metadata including an AIGC (Artificial Intelligence Generated Content) identifier, which indicates whether the media content is AIGC. Subsequently, the AIGC identifier in the media content's metadata can be used to identify whether the media content is AIGC; this can help users determine the authenticity of media content to a certain extent, and increase users' vigilance when using (e.g., forwarding) media content.

The application relates to the technical field of search, and discloses a keyword recommendation method, device and equipment and a storage medium, the method comprising the following steps: extracting a search keyword from a search instruction input by a user when the search instruction is received; obtaining a multimedia information keyword candidate set corresponding to the user when the search keyword does not satisfy a preset condition; determining a user interest word set according to a preset fine-grained recognition strategy and the multimedia information keyword candidate set; performing recall on the multimedia information keyword candidate set and the user interest word set respectively according to a preset recall strategy to obtain a plurality of keyword update candidate sets; performing coarse sorting on the keyword update candidate sets; and performing keyword recommendation according to a sorting result. In the application, when a search keyword input by a user does not satisfy a preset condition (i.e. is not standardized or is semantically confused and cannot be recognized), the multimedia information keyword is recommended by fusing a user behavior (i.e. a multimedia information keyword candidate set corresponding to the user) of comprehensive search.

The present disclosure provides a network model training method, a data processing method, and an apparatus. The network model training method comprises: acquiring target sample data, wherein the target sample data comprises text sample data and image sample data; inputting the target sample data into a network model to be trained to obtain a sample recognition result; and adjusting a parameter of a text encoder on the basis of a text recognition result and first supervision data corresponding to the text recognition result, adjusting a parameter of an image encoder on the basis of an image recognition result and second supervision data corresponding to the image recognition result, and a hybrid image-text recognition result and third supervision data corresponding to the hybrid image-text recognition result, and adjusting a parameter of a hybrid encoder on the basis of the hybrid image-text recognition result and the third supervision data corresponding to the hybrid image-text recognition result to obtain the trained network model formed by the text encoder, the image encoder, and the hybrid encoder.

A method for interpreting messages, user-defined alert conditions, voice commands and performing an action in response is described. A method for annotating media content is described. A method for presenting additional content associated with media content identified based on a fingerprint is described. A method for identifying that an advertisement portion of media content is being played based on a fingerprint derived from the media content is described. A method of one media device recording particular media content automatically in response to another media device recording the particular media content is described. A method of concurrently playing media content on multiple devices is described. A method of publishing information associated with recording of media content is described. A method of deriving fingerprints by media devices that meet an idleness criteria is described. A method of loading, modifying, and displaying a high definition frame from a frame buffer is described.

The application discloses a knowledge base file analysis and retrieval system based on big data analysis, comprising a data access module for generating a uniform feature sequence, a manifold embedding construction module for generating a manifold embedding vector set and a geodesic distance matrix, a path distribution reparameterization alignment module for generating a path probability distribution function, a soft alignment matrix and a path energy value, a joint energy construction module for generating a joint energy result, an energy optimization inference module for outputting a file alignment result, a structure index generation module for generating a structure index, and a retrieval sorting module for outputting a retrieval sorting result. The application realizes structure alignment optimization retrieval.

A computing system can obtain a multimodal multi-index data structure. The data structure can include a first and second plurality of data entries having first or second data types, and a plurality of index data structures. Each index data structure can include a plurality of index data entries each correlating a respective data entry of the first or second plurality of data entries with a corresponding indexing value. The computing system can receive a multimodal input comprising a query directed to a machine-learned agent. The computing system can retrieve, based at least in part on the query and the plurality of index data structures, a first data entry of the first or second plurality of data entries. The computing system can provide, to the machine-learned agent, the first data entry. The machine-learned agent can generate, based at least in part on the first data entry, an inference output.

The present application relates to the technical field of information retrieval and credit leasing, and particularly relates to an intelligent retrieval system for high-concurrency credit leasing business data. The present application acquires multi-source data through a data acquisition unit, and after encapsulation by a data processing unit, an asynchronous queue dispatching vector generation unit calls an AI model to generate a feature vector. During retrieval, a retrieval recognition unit analyzes a retrieval request and determines a mixed retrieval weight, and performs a query by fusing semantic and keyword strategies. A management unit adaptively switches index types, adjusts retrieval nodes and index parameters based on the number of retrieval requests and concurrent pressure within a preset time. A calculation unit determines a retrieval representation value based on the response time of a standard query, an analysis unit determines a retrieval state according to the retrieval representation value, and dynamically adjusts the number of retrieval nodes based on the retrieval state, optimizes the preset number of retrieval requests and related parameters, and forms a closed loop of data-driven and autonomous optimization. The present application improves retrieval efficiency.

A client application sends an application programming interface call to an interface of a content management system. The call specifies one or more content item search criteria. Sending the call causes the content management system to perform a lookup in a content item index to identify at least one content item that satisfies the one or more content item search criteria. Based on sending the call, the client application receives from the content management system a suggestion to attach the at least one content item to a text being displayed by at the computing system.

This application discloses an information recognition method, apparatus, storage medium, and electronic device based on attention modules. Embodiments of this application can be applied to various scenarios such as vehicle-mounted systems, cloud technology, artificial intelligence, intelligent transportation, and assisted driving, for example, in voice scenarios based on parallel computing. The method includes: acquiring target media resource features of a target media resource, inputting the target media resource features into a target information recognition model, processing the target media resource features through N layers of attention modules to obtain a target representation vector, wherein the i-th layer attention module is used to determine the i-th layer representation vector output by the i-th layer attention module based on a set of shared parameters and an i-th set of non-shared parameters, and determining the target information recognition result based on the target representation vector, wherein the target information recognition result represents the target information recognized from the target media resource. This application solves the technical problem in related technologies where attention recognition models suffer significant performance loss due to the effort to accelerate the computation process.

This application provides a method and system for constructing a multimodal large-scale model for safety management in the construction process. The method includes: acquiring structured and unstructured data related to safety management in the construction process, constructing a safety element ontology, and assigning global object identifiers and relational constraints to objects in the structured and unstructured data; processing the structured and unstructured data, generating evidence objects based on global object identifiers and unified time stamps; constructing six categories of libraries based on the evidence objects, using the evidence objects and entries associated with object identifiers in the six categories as training samples, performing multimodal representation alignment training and consistency verification training based on relational constraints to obtain a multimodal large-scale model for safety management in the construction process, and deploying the multimodal large-scale model as a reasoning model for risk classification, hazard closure-loop judgment, and emergency resource allocation based on evidence object references. This application can improve the accuracy of risk identification, accelerate hazard closure-loop handling, and improve emergency response efficiency.

A client application sends an application programming interface call to an interface of a content management system. The call specifies one or more content item search criteria. Sending the call causes the content management system to perform a lookup in a content item index to identify at least one content item that satisfies the one or more content item search criteria. Based on sending the call, the client application receives from the content management system a suggestion to attach the at least one content item to a text being displayed by at the computing system.

The application provides a media information violation detection method and device, electronic equipment and storage medium, and relates to the technical field of violation detection. The method comprises the following steps: acquiring media information to be detected; determining key data in the media information, the key data being data used for evaluating whether a violation exists; reorganizing the key data based on a reorganization rule corresponding to an information type of the media information to obtain reorganized data; and matching the reorganized data with data in a preconfigured standard database to determine whether there is a violation in the media information; wherein the data recorded in the standard database is data representing that a violation exists or data representing that no violation exists. The violation detection method, device, electronic equipment and storage medium provided by the application can conveniently and quickly detect violations in various types of media information, and can meet the detection requirements of massive media information.

This application discloses a file processing method, an electronic device, and a storage medium. The file processing method includes: determining a first identifier vector for a file based on its position in a file set; generating a first storage location for storing the first identifier vector based on a hash function and the file's feature vector; determining a second identifier vector based on the hash table corresponding to the hash function and the first identifier vector; and storing the second identifier vector in a second storage location obtained by performing a pseudo-random permutation on the first storage location. The second storage location is used to locate the first storage location corresponding to the feature vector of the file to be searched when searching for a file. Thus, based on the second storage location and the position of the file to be searched in the file set, the corresponding file can be found without requiring the user to provide precise keywords for the search, thereby improving file search efficiency.

The application discloses a kind of based on edge computing's chemical industry park real-time safety inspection system, including data acquisition module: for collecting equipment operating condition data and environmental safety data;Risk analysis module: for executing analysis processing to collected data, generates risk state vector;Space mapping module: for mapping risk state vector to risk state space and determining position coordinates;State kernel construction module: for executing topological division to risk state space and constructing segmented affine state transition subkernel;State update module: for generating system state update result under state transition subkernel constraint;Parameter update module: for executing spectral structure splitting to complex parameter space and completing parameter update;Result output module: for outputting safety inspection state based on updated complex parameter and system state update result.This application realizes risk evolution controlled modeling and parameter stable update.

This invention relates to the field of educational management technology, and discloses a method and system for managing intelligent teaching and research activities that supports multimodal input. The method includes: collecting audio streams, video streams, and comment texts of teaching and research activities; writing the comment texts into pointer fields pointing to speech segments and behavior tag indices to obtain chained annotations; establishing and encapsulating mutual pointers between the audio stream, video stream, and chained annotations based on time identifiers to obtain a teaching and research dataset; constructing a composite teaching activity curve with the speech text time period as the horizontal axis and the composite value of behavior frequency and speech transition number as the vertical axis; using the peaks, troughs, and slope change points of the curve as rhythm feature points, and matching them with a behavioral semantic database to obtain a rhythm analysis graph; using the graph as the time axis skeleton, embedding the chained annotations into the target time interval according to the pointer direction to obtain a comprehensive analysis report; this invention can improve the efficiency of intelligent teaching and research activity management that supports multimodal input.

This document describes techniques and systems that enable privacy controls for sharing embeddings for searching and indexing media content. A set of images of a user's face are obtained and a machine-learned model is applied to the set of images to generate a user-specific dataset of face embeddings for the user. Media content stored in a media storage is indexed by applying the machine-learned model to the media content to provide indexed media information identifying one or more faces shown in the media content. Access to the indexed media information by another user querying the media content for images or videos depicting the user is controlled based on a digital key shared by the user with the other user, where the digital key is associated with the user-specific dataset and the user-specific dataset is usable to identify the images or videos depicting the user.

The embodiment of the application provides a vehicle behavior recognition method and device based on multi-source perception data fusion, realizes effective integration of information through feature checking and standardization conversion, constructs an analysis mechanism, combines space-time clustering and trajectory analysis, establishes a reliable recognition strategy, introduces evidence construction, ensures comprehensiveness of analysis through multi-source association and data combination, and effectively solves the deficiencies of traditional technologies in data processing, group identification and evidence construction, and the like, thereby providing technical support for vehicle behavior recognition.

The technical problem of reducing the amount of processing involved when searching for customizable media content items that are suitable for incorporating input text is addressed by providing a hybrid search system. In some examples, the hybrid search system executes a rough search first, to determine whether a line of text can be incorporated into a media content item, based on character count conditions associated with the media content item. A more thorough evaluation of the input text with respect to the media content item is executed subsequent to the rough search if the rough search produces a result indicating uncertainty with respect to whether the combination of specific characters included in the input text can or cannot be incorporated into the media content item.

A media data decryption method includes: obtaining a media data encryption package, wherein the media data encryption package is obtained by performing pixel rearrangement encryption on original media data based on position index data, and the position index data correspond to one or more structural parameters of a display screen; decrypting the media data encryption package to obtain the position rearrangement index data and media data to be decrypted; configuring a programmable logic circuit of a control system of the display screen in a media data play terminal based on the position rearrangement index data; and decrypting the media data to be decrypted based on the configured programmable logic circuit to obtain decrypted media data.

Methods, apparatus, systems and articles of manufacture are disclosed for efficient media indexing. An example method disclosed herein includes means for initiating a list of hash seeds, the list of hash seeds including at least a first hash seed value and a second hash seed value among other hash seed values, means for generating to generate a first bucket distribution based on the first hash seed value and a first hash function and generate a second bucket distribution based on the second hash seed value used in combination with the first hash seed value, means for determining to determine a first entropy value of the first bucket distribution, wherein data associated with the first bucket distribution is stored in a first hash table and determine a second entropy value of the second bucket distribution.