1085results about "Metadata multimedia retrieval" patented technology

The invention discloses a cross-modal information fusion and retrieval method and system driven by semantic comprehension, and the method comprises the steps: obtaining text, image and audio original data, and extracting an initial feature set of each modal through a deep neural network; dynamically distributing each modal weight coefficient based on an attention mechanism, and performing weighted fusion on the initial feature set to obtain cross-modal fusion feature representation; through a cross-modal semantic association analysis model, high-dimensional semantic association features are extracted from the fusion feature representation, and semantic enhancement feature vectors are generated; constructing a cross-modal semantic graph network based on the vector, complementing missing modal features, and generating an optimized multi-modal feature set; and inputting the optimized feature set and the query sample into a contrast learning model, calculating a semantic similarity score, and generating a cross-modal retrieval result sorting list according to the score.

The invention provides an intelligent retrieval method and system for unstructured asset content based on a large model, and relates to the technical field of data retrieval, and the method comprises the steps: carrying out semantic expansion on retrieval keywords through a knowledge graph enhancement module, obtaining asset feature vectors through a multi-modal feature extraction model, and constructing a hierarchical asset feature index database; the comprehensive similarity is calculated through a weighted fusion algorithm, and retrieval results are optimized and reordered through semantic comprehension and correlation analysis capacity of the pre-trained large model. According to the method, the retrieval precision and efficiency of the unstructured asset content are improved, and intelligent association analysis and semantic understanding of cross-modal assets are realized.

The invention discloses a collaborative personalized learning system and method based on a large model, and relates to the technical field of collaborative learning, and the system collects and fuses multi-modal data, generates a student state vector, plans a personalized learning path based on the state and a knowledge graph, and generates multi-modal explanation content according with the student style. The method comprises the following steps: extracting a student reasoning path, identifying thinking deviation, analyzing task performance, generating a path and content adjustment suggestion, identifying a motivation state, triggering an intervention strategy, receiving a teacher strategy, and realizing path and style co-construction. The multi-modal content of the matched style is generated, adaptive intervention is achieved in combination with inference analysis and emotion adjustment, and the learning efficiency and the teaching response intelligent level are improved.

The invention provides a wisdom education method and system based on student behavior sequence recommendation, and relates to the technical field of wisdom education. According to the method and the system, through an AI autonomous identification knowledge graph construction and innovative knowledge fusion method, a course knowledge system is structurally and hierarchically displayed, and a knowledge framework is clearly presented, so that knowledge graph framework construction of all-platform courses is facilitated, pre-repaired courses and subsequent courses are efficiently associated, and the construction efficiency is improved. And a clear and complete course learning path is provided for students. By adopting a recommendation algorithm based on a student behavior sequence, combining SASRec and introducing a self-attention mechanism of a sparse attention module, behavior data of students can be deeply analyzed, a learning path is adjusted according to real-time learning data, and personalized learning suggestions are provided for the students. According to the method and the system, multi-dimensional functions such as knowledge graph construction, a student behavior sequence recommendation algorithm and intelligent learning path optimization are integrated, and the method and the system have wide coverage and strong comprehensiveness.

The invention discloses a visual scene method, system and equipment based on digital twinning and a medium, and relates to the technical field of digital twinning and visual modeling, and the method comprises the steps: collecting multi-format source data, carrying out the field standard mapping, and carrying out the consistency verification of the multi-format source data after the field standard mapping; writing the multi-format source data after consistency verification into a to-be-fused buffer area, selecting model component data in the to-be-fused buffer area to execute coordinate reference conversion, and performing association binding on the converted model component data and the structured graphic and text information by constructing an identification field; and synchronously loading the associated and bound model component data and the structured graphic and text information to a predefined template container, generating a scene configuration file according to a container structure, and calling a release engine to register to a multi-terminal rendering service channel. According to the method, accurate alignment of multi-format model components in a unified space coordinate system is realized by constructing an affine transformation coordinate reference conversion scheme.

The invention discloses a multi-modal enhanced retrieval method and system, and the method comprises the steps: obtaining heterogeneous data, and executing data processing to obtain a meta-information set; encoding the meta-information set into a high-dimensional meta-information vector, and constructing a vector index; receiving a query statement of a user, calculating the similarity between the query statement and the meta-information vector, and obtaining a meta-information query result; and calling a large language model based on the obtained meta-information to generate a summary result, and merging and outputting the summary result and heterogeneous data corresponding to the meta-information. According to the method, multi-modal data is processed into unified semantic vector representation, efficient vectorization indexing and semantic generation and expansion capacity of a large-scale language model are combined, high-value information can be rapidly extracted from multi-source heterogeneous data, and real-time, cross-modal and high-precision retrieval requirements are supported. A reordering and traceability mechanism introduced in the scheme further enhances the correlation and reliability of the information, and provides comprehensive and accurate knowledge support for the generation of query results.

The invention belongs to but is not limited to the technical field of artificial intelligence, and particularly relates to a controllable content large model generation method and system based on multi-modal thinking chain reasoning, and the method comprises the steps: S1, improving the capability of multi-agent planning task dependence and execution sequence through employing a thinking chain method, enabling a complex task to be subdivided into a plurality of subtasks through the method, and enabling the subtasks to be subdivided into a plurality of subtasks; a resource transmission process between the sub-tasks is precisely planned and is executed in a chain mode; s2, complex tools are combined into sub-agents with different modal characteristics, so that the number of selectable sub-tasks during task planning is reduced, and the planning capability is improved; and S3, storing scene information of a previous dialogue by means of an external database, and when a new request is encountered, improving correlation and accuracy of multi-agent response by quoting historical interaction information and historical multi-modal resources which have context similarity with the current request, and realizing a more intelligent reaction mechanism.

Disclosed are a method for designing lightning shielding by using diverter strips on an aircraft, a system, a device, and a medium. The method comprises: determining boundary conditions of a simulation space; according to the boundary conditions of the simulation space, performing numerical calculation on an electric field in the simulation space; according to the calculation result of the electric field and in combination with fractal theory, determining a probability of the direction of lightning leader development, and, according to the probability of the direction of lightning leader development, randomly selecting a subsequent development position of a lightning downward-moving stepped leader; determining, on the basis of testing and simulation results of a corona discharge characteristic at the tip of a metal electrode having a small radius of curvature, whether an initial corona at an end portion of a diverter strip has been initiated, and determining whether an initial corona at an end portion of a diverter strip has been initiated; determining whether the lightning downward-moving stepped leader can, after attaching to an insulating plate, run through a gap along the surface and reach an adjacent diverter strip; repeating the process above, and performing multiple simulations to solve the probability of lightning protection of diverter strips and establish a relationship between the probability of lightning protection of diverter strips and the distance between the diverter strips at different thicknesses of and using different materials for the insulating plate.

The embodiment of the invention provides a metadata generation method, a metadata identification method and electronic equipment. The method comprises the following steps: firstly, acquiring media content; next, metadata of the media content is generated, the metadata of the media content comprises an AIGC (Artificial Intelligence Generation Content) identifier, and the AIGC identifier indicates whether the media content is the AIGC or not; further, subsequently, whether the media content is the AIGC or not can be identified according to the AIGC identifier in the metadata of the media content; therefore, the user can be helped to judge the authenticity of the media content to a certain extent, and the vigilance of the user in using (such as forwarding) the media content is improved to a certain extent.

The invention discloses a digital collection full-life-cycle traceability tracking system based on a block chain, and the system comprises a digital identification module, a verification module, a packaging transfer module, a verification module, an integration module, and a system construction module, and achieves the full-life-cycle traceability tracking of a digital collection through the technologies of multi-dimensional feature extraction, watermark embedding, distributed storage, and zero-knowledge proof. And tracking and verification of the whole process from creation to transaction of the digital collections are realized. According to the system, the transaction authorization model is encapsulated by adopting the intelligent contract, so that safe transfer of ownership is ensured; a tamper-proof verification mechanism is constructed through multi-level integrity verification; different platform data are integrated by using a cross-chain technology to form a comprehensive traceability graph, so that key problems of authenticity verification, copyright protection, value evaluation and the like faced by a digital collection market are effectively solved. In addition, the system also establishes a scientific authenticity evaluation and value dynamic evaluation system, provides an objective basis for market pricing, and significantly improves the credibility and safety of the digital collections.

The invention discloses a multi-modal information identifier generation method and system based on semantic hash. The method comprises the following steps: carrying out data preprocessing and multi-modal feature extraction on multi-modal data to obtain a unified representation containing rich semantic information; mapping the features to a shared semantic space by adopting an independent alignment projection network of each mode, and introducing cross-mode contrast learning and label supervision to realize semantic alignment among different modes; compressing the high-dimensional features after multi-modal data alignment to generate a Hash code with a fixed length; generating a unified semantic hash code containing semantic information of at least two modals; searching the number of times that the Hash code appears in a database through the unified semantic Hash code, generating a redundant code, and splicing the redundant code with the unified semantic Hash code to form an identifier of the sample; on the basis of the unified semantic hash codes, the hash codes related to the to-be-queried category in the test set are queried in the training set, and unified identification and efficient retrieval of different modes are achieved.

The invention discloses an active real-time interaction system based on LLM. The active real-time interaction system comprises a self-cognition module, a real-time perception module, an active planning module, a situation decision module, a dynamic execution module, an emotion engine module, an adaptive interaction module and a safety management and control module. The system adopts a multi-dimensional vector to represent role attributes, and constructs a hierarchical memory storage structure to record interaction experience; generating an environment state vector through a multi-modal data processing technology; executing the target task decomposition algorithm to generate a multi-path execution plan; making a real-time decision based on the multi-dimensional decision factor; mapping the abstract decision into an instruction sequence and monitoring an execution process; simulating a system emotional state and influencing decision expression; dynamically adjusting an interaction strategy according to the user characteristics; evaluating decision rationality and starting a corresponding intervention mechanism. All the modules form a closed-loop workflow through a standardized data interface, the problems that a traditional AI system is passive in response, lacks self-cognition, is limited in environmental perception, is single in planning capability, lacks flexibility in decision making and the like are solved, and active service, self-evolution and safety controllability of the system are achieved.

The invention discloses a media file query and retrieval method and system based on an AI dialogue scene, the method combines a mixed mode of vector retrieval and keyword retrieval, realizes accurate retrieval of cross-modal data, and can process various media files including texts, pictures, videos, PPT and the like. According to the method, a double-round scoring mechanism of a large language model (LLM) is combined, firstly, the relevancy of file content is scored, then secondary scoring is carried out according to the matching degree of a file title and user query, and it is ensured that the file most relevant to query is called. In addition, the system can further utilize a user feedback mechanism to continuously optimize the retrieval precision, and the user experience is improved. Compared with the prior art, the method has the advantages that the accuracy and the efficiency of multi-modal data retrieval are remarkably improved, the limitation of a traditional AI dialogue system during multimedia file processing is solved, and the method is suitable for intelligent dialogue application scenes needing to quickly and accurately obtain various types of media files.

Methods, systems, and media for processing queries relating to presented media content are provided. In some implementations, a method comprises: receiving a request to associate with a media playback device that is presenting media content to a user of the mobile device, wherein a mobile application executing on the mobile device and a media application executing on the media playback device exchange media playback information; activating a microphone associated with the mobile device to receive ambient sounds in response to associating with the media playback device; converting the received ambient sounds to one or more text inputs; determining whether the text inputs include a trigger term that corresponds to a request to initiate a query relating to the presented media content and the query; in response to determining that the trigger term has been included in the text inputs, determining the media playback information from the media application that includes timing information corresponding to when during the presentation of the media content the query was received and media content identification information; causing a search to be performed that includes the query, the timing information, and the media content identification information; obtaining a search result that is responsive to the query; and presenting at least a portion of the search result to the query on a mobile display associated with the mobile device.

The invention discloses a cultural heritage digital display platform, which is mainly applied to the fields of museums, cultural heritage protection institutions, education institutions and the like, and specifically comprises a central control module, and a user interaction module, a multimedia display module, a virtual reality rendering module, a database management module and an intelligent recommendation module which are connected with the central control module. The cultural heritage digital display platform has remarkable beneficial effects in the aspects of improving cultural heritage protection and inheritance efficiency, enhancing user experience and interactivity, realizing personalized recommendation and accurate service, promoting cultural education and propagation, optimizing data management and analysis capability and the like, and is beneficial to protection and inheritance of the cultural heritage; and the cultural literacy and aesthetic level of the public can be improved, and a positive contribution is made to the development of cultural heritage.

The invention provides an integrated sharing method and system for digital media content resources, and is applied to the technical field of data processing. The method comprises the following steps: preprocessing digital media content resources, cloud platform basic data and user demand data to generate a standardized media content sequence, cloud platform features, user demand features and resource classification features; processing the resource association relationship data to generate resource association features; processing a strategy in the shared strategy updating mechanism information based on the resource value evaluation system, and generating updated shared strategy data; processing the standardized media content sequence, the cloud platform feature and the resource association feature based on a target integrated sharing model to generate a target integrated sharing feature; and processing the target integration sharing feature based on the target integration sharing model and the updated sharing strategy data to generate a digital media content resource integration sharing result.

The invention provides an interaction method and device, a storage medium and a program product, and relates to the technical field of computers. The interaction method comprises the following steps: generating a first content in response to an instruction of generating the first content input by a user in a dialogue area; an editing area matched with the body of the first content is displayed, the first content is displayed in the editing area, and the editing area and the dialogue area are located on the same display interface; in response to an editing operation triggered by the user in the dialogue area or the editing area, determining an editing instruction for the first content based on the first content and the style of the first content; and based on the editing instruction, editing the first content in the editing area.

The invention discloses a personalized teaching content generation method and system based on a digital portrait, and the method comprises the steps: collecting the multi-dimensional learning data of a student in real time, analyzing the data keyword of the student, constructing the portrait of the student, generating a teaching target based on the portrait of the student, and enabling the teaching target to comprise a teaching theme, exercises related to the theme, and knowledge points related to the theme. Converting the teaching target into an executable instruction of a large language model through a structured prompt engineering technology; the large language model generates initial teaching content according to the teaching instruction; verifying and optimizing the generated initial teaching content to ensure that the generated teaching content conforms to a teaching target and can adapt to the current cognitive level and learning requirements of students; and outputting the verified and optimized teaching content to the students and automatically adapting to the content presentation form according to the learning style preference of the students.

An electronic device associated with a media-providing service stores, in a vector space, a plurality of respective vector representations for respective media content items. The electronic device receives a user input, including a text string. The electronic device generates, using a neural network, a structured query based on the text string. The electronic device determines, based on the structured query, whether to generate a vector representation of a portion of the text string. When the electronic device determines to generate the vector representation of the portion of the text string, it generates the vector representation of the portion of the text string, wherein the vector representation is embedded in the vector space, and identifies a set of media items using the vector representation of the portion of the text string. And the electronic device provides one or more select media items from the set of media items to a user.

The invention discloses a cross-modal image-text retrieval method based on pulse fusion, and belongs to the technical field of image-text retrieval. The method comprises the following steps: inputting a target image set into a target detection network to obtain an image floating point code, and inputting a target text set into a word segmentation device to obtain a word floating point code; inputting the image floating point code and the word floating point code into a pulse encoder to obtain a first image pulse code and a first text pulse code; inputting the first image pulse code and the first text pulse code into a pulse cross attention fusion module to obtain a second image pulse code and a second text pulse code; respectively carrying out weighted accumulation and average pooling on the second image pulse code and the second text pulse code to obtain an image floating point feature vector set and a text floating point feature vector set, and calculating cosine similarity to obtain an image-text alignment result; and the text retrieval result of each image in the target image set is obtained based on the image-text alignment result, so that the accuracy and efficiency of image-text retrieval are improved.

The invention discloses a conference content intelligent generation processing method and system based on a multi-modal large model, and the method comprises the steps: collecting the original data of a conference, and completing the standardization preprocessing; inputting a multi-modal large model, extracting multi-modal features and carrying out semantic alignment; executing cross-modal hash coding, generating binary codes and establishing an index database; performing hash retrieval on the related fragments, and constructing a conference content directed graph; based on a conference content directed graph structure, searching an optimized path by adopting a Monte Carlo tree; and generating structured conference content, and outputting a summary, an abstract and an action item. According to the method, efficient extraction, accurate retrieval and structured intelligent generation of the conference content are realized by fusing a multi-modal large model, cross-modal Hash coding and Monte Carlo tree search.

Techniques for suggesting media assets, the technique including: requesting a set of candidate media assets for a set of user media items based on a knowledge graph metadata network describing the set of user media items; receiving metadata for the set of candidate media assets; determining one or more sets of ranked media assets based on the received metadata; and outputting the determined one or more sets of ranked media assets.

The invention provides an interaction method and device, a storage medium and a program product, and relates to the field of terminals. The interaction method comprises the following steps: displaying a session interface between a first user and a virtual object; receiving an input message on the session interface; and displaying image-text content related to the input message in a message body on the session interface, the image-text content comprising at least one image group and at least one text block, and the image-text content being a part of content in a media work published by a second user corresponding to the virtual object.

The invention discloses a model training method, electronic equipment and a computer readable storage medium, and relates to the technical field of data processing and large models. The method comprises the steps of obtaining a training data set, wherein the training data set comprises image annotation data of various types of concerned objects and description text annotation data associated with the image annotation data; training the initial multi-modal retrieval model by adopting the training data set to generate a learnable prompt; carrying out image coding training on the initial multi-modal retrieval model by adopting the training data set and the learnable prompt to generate an intermediate multi-modal retrieval model; the training data set is adopted to conduct text and image alignment training on the middle multi-modal retrieval model, a target multi-modal retrieval model is generated, and the target multi-modal retrieval model is used for conducting multi-modal retrieval on the target retrieval request to obtain a target retrieval result. According to the method and the device, the technical problems that a multi-modal retrieval model in related technologies cannot adapt to variability and large-scale data requirements, and the model performance is relatively poor are solved.

The invention discloses an AIGC-based multimedia teaching courseware generation method, relates to the technical field of data processing, and mainly solves the problems of low efficiency, single courseware content and low courseware quality. The AIGC-based multimedia teaching courseware generation method comprises the implementation steps of collecting demand data of teaching courseware; processing and analyzing the received demand data; creating a detailed teaching courseware outline and a teaching courseware content structure; generating multimedia elements and courseware contents in the teaching courseware; integrating and editing the content generated by the AIGC tool; multimedia teaching courseware is previewed and detected and exported and iteratively managed, and a teaching target and student data are analyzed through a dynamic analysis algorithm, so that the courseware generation efficiency is improved; by adopting the multimedia material content, the diversification of the courseware content is improved; and the generated media material contents are fused through an information fusion algorithm, so that the courseware quality is improved.

The invention provides a medical visual question answering method based on multi-scale feature fusion and question enhancement. The method comprises the steps of obtaining a to-be-recognized medical image and a to-be-answered text; inputting the to-be-recognized medical image and the to-be-answered text into a pre-trained medical question and answer model to obtain a detection result about the to-be-answered text; the detection result is a text result; wherein the pre-trained medical question and answer model is provided with a visual extraction model, a text extraction model and a classification model; the visual extraction model is a visual model based on a multi-scale visual feature extraction method, a multi-scale visual feature enhancement method and a multi-scale visual feature fusion method; and a multi-head cross attention feature interaction mechanism is arranged between the visual extraction model and the text extraction model. By combining the visual extraction model, the text extraction model and the classification model in the pre-trained medical question and answer model, the understanding ability of the medical image is improved, and finally the recognition precision and the answer quality of the medical visual question and answer task are integrally improved.