643results about "Multimedia data clustering/classification" patented technology

The invention relates to the technical field of artificial intelligence, in particular to an automatic construction method of an end-to-end agent based on graph structure semantic fusion. The method comprises the following steps: receiving business demand data input by a user; business target and demand constraint condition analysis is carried out on the business demand data, and a core workflow framework of the intelligent agent is generated; performing end-to-end execution path analysis on the core workflow framework of the intelligent agent to obtain an end-to-end workflow; constructing a dynamic evolution semantic map; and constructing an end-to-end call chain execution strategy based on the end-to-end workflow, and performing agent instance packaging and agent instance reinforcement learning enhancement processing according to the dynamic evolution semantic map, thereby automatically constructing an end-to-end agent. According to the invention, by fusing the graph structure knowledge and the generation capability of the large language model, an efficient, accurate and extensible agent automatic construction scheme is provided for various complex business scenes.

An ESG-specific multimodal AI foundation model is disclosed, featuring a Transformer-based architecture with approximately 30 billion parameters, designed explicitly for environmental, social, and governance (ESG) domain applications. This model uniquely supports extremely long context windows (up to 128,000 tokens), critical for comprehensive ESG analyses of lengthy documents such as sustainability reports and policies. It integrates textual and visual data through gated cross-attention and a Mixture-of-Experts (MoE) architecture, achieving precise multimodal context comprehension. The invention employs Group Relative Policy Optimization (GRPO) reinforcement learning strategy, refining model outputs based on group-relative advantages computed from multiple candidate generations, thus significantly enhancing ESG-specific reasoning and output quality.

A method of training a text model using a plurality of text passage combinations, each text passage combination comprising a respective first text passage and a respective second text passage describing a same matter as the respective first text passage but being differently worded than the respective first text passage. The training comprises minimizing a measure of statistical difference between a respective value of a first text embedding and the corresponding value of a second text embedding over the plurality of text passage combinations. The method then comprises jointly training the text model and an image model based on plurality of image-text combinations, each comprising a respective image and a respective textual report describing the respective image. The joint training comprises minimizing a measure of statistical difference between the value of an image embedding and the corresponding value of a third text embedding over the plurality of image-text combinations.

The invention discloses a visual scene method, system and equipment based on digital twinning and a medium, and relates to the technical field of digital twinning and visual modeling, and the method comprises the steps: collecting multi-format source data, carrying out the field standard mapping, and carrying out the consistency verification of the multi-format source data after the field standard mapping; writing the multi-format source data after consistency verification into a to-be-fused buffer area, selecting model component data in the to-be-fused buffer area to execute coordinate reference conversion, and performing association binding on the converted model component data and the structured graphic and text information by constructing an identification field; and synchronously loading the associated and bound model component data and the structured graphic and text information to a predefined template container, generating a scene configuration file according to a container structure, and calling a release engine to register to a multi-terminal rendering service channel. According to the method, accurate alignment of multi-format model components in a unified space coordinate system is realized by constructing an affine transformation coordinate reference conversion scheme.

The invention discloses a multi-modal enhanced retrieval method and system, and the method comprises the steps: obtaining heterogeneous data, and executing data processing to obtain a meta-information set; encoding the meta-information set into a high-dimensional meta-information vector, and constructing a vector index; receiving a query statement of a user, calculating the similarity between the query statement and the meta-information vector, and obtaining a meta-information query result; and calling a large language model based on the obtained meta-information to generate a summary result, and merging and outputting the summary result and heterogeneous data corresponding to the meta-information. According to the method, multi-modal data is processed into unified semantic vector representation, efficient vectorization indexing and semantic generation and expansion capacity of a large-scale language model are combined, high-value information can be rapidly extracted from multi-source heterogeneous data, and real-time, cross-modal and high-precision retrieval requirements are supported. A reordering and traceability mechanism introduced in the scheme further enhances the correlation and reliability of the information, and provides comprehensive and accurate knowledge support for the generation of query results.

This disclosure relates to a content cluster system that provides a framework for leveraging large language models (LLMs) to tag content items (e.g., interactive multimedia content items, such as gaming content items) with attribute tags and, based on attribute tags for related content items, generating cluster descriptions of a cluster of content items. Features of the content cluster system involve tagging content items, determining multi-dimensional embeddings for the content items, and clustering the content items based on proximity of values contained within the multi-dimensional embeddings. The content cluster system may further utilize the LLM to generate one or more cluster descriptions based on the associated tags to create a more creative and dynamic representation of related groupings of titles (e.g., gaming titles). By utilizing LLMs and machine learning resources, the content cluster system provides a scalable approach to conventional approaches in determining and presenting groupings of titles.

The invention relates to the technical field of information retrieval and generation, in particular to a retrieval enhancement method based on multi-modal data fusion and modal perception. According to the method, firstly, a dual-channel architecture is adopted to perform feature extraction and coding on a text and an image respectively, and mutually independent embedded representation spaces are constructed, so that high-quality collaboration and matching of cross-modal representation are realized; and a pseudo-pairing generation mechanism is introduced to effectively mine and reconstruct the existing non-paired data in the knowledge base. And designing a query modal perception and dynamic weighting mechanism for accurately controlling the fusion proportion of the image-text bimodal information in the retrieval stage so as to match the modal demand difference of different query contents. And further executing aggregation retrieval and reordering of the cross-modal information by using dynamic weighted fusion retrieval to generate a candidate set of multi-modal responses. According to the method, accurate matching and dynamic weight adjustment of the image-text content are realized, and the accuracy and expression integrity of the generated content are improved.

The invention provides an AI-based multi-user data storage docking and secure transmission method, which comprises the following steps of: acquiring a propagation path of a hot topic according to a multi-modal content association graph, performing fragmentation storage on the propagation path by adopting a clustering algorithm, and generating a fragmentation storage index table; when the relevance of different modal contents is higher than a preset threshold value, determining that the sensitivity grading result is consistent with the multi-modal relevance, performing dynamic rule adjustment by adopting a deep learning model, generating a dynamic auditing rule set, and obtaining boundary judgment parameters of the rule set; and distributing the dynamic auditing rule set and the decision transparency index to each service line by adopting a cross-platform data synchronization protocol according to the auditing decision log, generating a cross-platform consistency auditing standard, and obtaining a synchronization log of standard execution.

The invention belongs to the field of multi-mode sensing fusion, and relates to an intelligent multi-mode patrol sensing method, system and equipment and a storage medium, which are used for carrying out automatic patrol, real-time sensing and abnormity warning in a complex environment. Comprising the steps of natural language instruction analysis, task decomposition and distribution, intelligent agent scheduling and execution, multi-modal information collection and fusion, anomaly recognition and event response, result feedback and task closed loop, and through the integration of an OWL framework and a DeepSeek-R1 large model, a multi-modal data fusion technology and an NL-SLAM autonomous navigation algorithm, the multi-modal data fusion algorithm and the multi-modal data fusion technology are integrated. The problems that in the prior art, dynamic task understanding based on natural languages cannot be achieved, a sensing system lacks efficient multi-modal data fusion, so that the anomaly recognition precision is low, and complex tasks or emergencies are difficult to deal with are solved, various abnormal conditions in a complex environment are effectively dealt with, and the method is suitable for popularization and application. The stability and expansibility of task execution are improved, and the navigation success rate and the task completion rate of the system in an unstructured environment are enhanced.

The invention provides a knowledge question and answer rapid processing method and system based on artificial intelligence, and relates to the field of artificial intelligence. A multi-modal knowledge graph is constructed, collected multi-source teaching data is fused through a mixed retrieval strategy, and the mixed retrieval strategy comprises semantic retrieval, vector retrieval and metadata retrieval; multi-level question and answer processing is executed based on an RAG enhancement framework, a multi-modal input intention is analyzed, cross-library joint retrieval is performed, and an optimization answer is generated in combination with a teaching scene; distilling the global model to a lightweight TinyBERT architecture, dynamically optimizing question and answer quality through a cognitive reinforcement learning framework, positioning a key document from a comprehensive retrieval list, evaluating an optimized answer, and reconstructing an answer with a key document verification score; according to the invention, the professional skill level of teachers and students in the fields of artificial intelligence and large model application can be improved, and the personalized requirements of teachers and students in teaching, scientific research and innovation courses can be met.

Systems and methods for reducing the search space by processing media content to refine search parameters. A computing device may obtain the media content in response to receiving a request for inclusion of the media content in a media content knowledge repository, extract an audio component, a video component, and a text component of the media content, and determine attributes within the extracted components. The computing device may determine segment attributes based on a result of correlating the determined audio, video, and text attributes, integrate the segment attributes into the media content knowledge repository, and / or perform any of a variety of responsive actions.

The invention provides a legal service method and system based on a large language model and related equipment, and the method comprises the steps: obtaining user request information, and recognizing the class information and field information of the user request information; according to the category information and the domain information, a preset agent meeting the category information and the domain information is selected from a preset agent library, the preset agent meeting the category information and the domain information serves as a target agent, and the target agent comprises a preset large language model meeting the domain information and a preset function matrix meeting the category information; inputting the user request information into the target agent to obtain an initial result; on the basis of the user request information, enhanced information is inquired in a preset vector knowledge base through an RAG retrieval technology; and generating a target result for the user request information based on the enhanced information and the initial result, and displaying the target result through a visualization technology. And different legal problems can be flexibly handled according to the diversity and individuation requirements of legal services.

The invention discloses a digital collection full-life-cycle traceability tracking system based on a block chain, and the system comprises a digital identification module, a verification module, a packaging transfer module, a verification module, an integration module, and a system construction module, and achieves the full-life-cycle traceability tracking of a digital collection through the technologies of multi-dimensional feature extraction, watermark embedding, distributed storage, and zero-knowledge proof. And tracking and verification of the whole process from creation to transaction of the digital collections are realized. According to the system, the transaction authorization model is encapsulated by adopting the intelligent contract, so that safe transfer of ownership is ensured; a tamper-proof verification mechanism is constructed through multi-level integrity verification; different platform data are integrated by using a cross-chain technology to form a comprehensive traceability graph, so that key problems of authenticity verification, copyright protection, value evaluation and the like faced by a digital collection market are effectively solved. In addition, the system also establishes a scientific authenticity evaluation and value dynamic evaluation system, provides an objective basis for market pricing, and significantly improves the credibility and safety of the digital collections.

An electronic device includes a memory storing one or more instructions; a communication circuit; and a processor operatively coupled to the memory and the communication circuit, in which the one or more instructions, when executed by the processor, cause the electronic device to: identify a use pattern of specified media items from a plurality of media items stored in the memory, determine, based on the use pattern, a score of each of the specified media items, extract, using a main AI model stored in the memory, a feature corresponding to a characteristic of each of the specified media items, acquire a first AI model trained based on the score and the feature, based on the first AI model, determine a first preference of each of first media items from the plurality of media items, and based on the first preference, perform a function related to the first media items.

A system or process may generate a summarization of multimedia content by determining one or more salient moments therefrom. Multimedia content may be received and a plurality of frames and audio, visual, and metadata elements associated therewith are extracted from the multimedia content. A plurality of importance sub-scores may be generated for each frame of the multimedia content, each of the plurality of sub-scores being associated with a particular analytical modality. For each frame, the plurality of importance sub-scores associated therewith may be aggregated into an importance score. The frames may be ranked by importance and a plurality of top-ranked frames are identified and determined to satisfy an importance threshold. The plurality of top-ranked frames are sequentially arranged and merged into a plurality of moment candidates that are ranked for importance. A subset of top-ranked moment candidates are merged into a final summarization of the multimedia content.

The invention discloses a multi-modal information identifier generation method and system based on semantic hash. The method comprises the following steps: carrying out data preprocessing and multi-modal feature extraction on multi-modal data to obtain a unified representation containing rich semantic information; mapping the features to a shared semantic space by adopting an independent alignment projection network of each mode, and introducing cross-mode contrast learning and label supervision to realize semantic alignment among different modes; compressing the high-dimensional features after multi-modal data alignment to generate a Hash code with a fixed length; generating a unified semantic hash code containing semantic information of at least two modals; searching the number of times that the Hash code appears in a database through the unified semantic Hash code, generating a redundant code, and splicing the redundant code with the unified semantic Hash code to form an identifier of the sample; on the basis of the unified semantic hash codes, the hash codes related to the to-be-queried category in the test set are queried in the training set, and unified identification and efficient retrieval of different modes are achieved.

The invention discloses an active real-time interaction system based on LLM. The active real-time interaction system comprises a self-cognition module, a real-time perception module, an active planning module, a situation decision module, a dynamic execution module, an emotion engine module, an adaptive interaction module and a safety management and control module. The system adopts a multi-dimensional vector to represent role attributes, and constructs a hierarchical memory storage structure to record interaction experience; generating an environment state vector through a multi-modal data processing technology; executing the target task decomposition algorithm to generate a multi-path execution plan; making a real-time decision based on the multi-dimensional decision factor; mapping the abstract decision into an instruction sequence and monitoring an execution process; simulating a system emotional state and influencing decision expression; dynamically adjusting an interaction strategy according to the user characteristics; evaluating decision rationality and starting a corresponding intervention mechanism. All the modules form a closed-loop workflow through a standardized data interface, the problems that a traditional AI system is passive in response, lacks self-cognition, is limited in environmental perception, is single in planning capability, lacks flexibility in decision making and the like are solved, and active service, self-evolution and safety controllability of the system are achieved.

Systems and methods for reducing the search space by processing media content to refine search parameters. A computing device may obtain the media content in response to receiving a request for inclusion of the media content in a media content knowledge repository, extract an audio component, a video component, and a text component of the media content, and determine attributes within the extracted components. The computing device may determine segment attributes based on a result of correlating the determined audio, video, and text attributes, integrate the segment attributes into the media content knowledge repository, and / or perform any of a variety of responsive actions.

The invention discloses a media file query and retrieval method and system based on an AI dialogue scene, the method combines a mixed mode of vector retrieval and keyword retrieval, realizes accurate retrieval of cross-modal data, and can process various media files including texts, pictures, videos, PPT and the like. According to the method, a double-round scoring mechanism of a large language model (LLM) is combined, firstly, the relevancy of file content is scored, then secondary scoring is carried out according to the matching degree of a file title and user query, and it is ensured that the file most relevant to query is called. In addition, the system can further utilize a user feedback mechanism to continuously optimize the retrieval precision, and the user experience is improved. Compared with the prior art, the method has the advantages that the accuracy and the efficiency of multi-modal data retrieval are remarkably improved, the limitation of a traditional AI dialogue system during multimedia file processing is solved, and the method is suitable for intelligent dialogue application scenes needing to quickly and accurately obtain various types of media files.

The invention discloses a teaching corpus construction method and system based on deep learning, and a medium, and belongs to the cross technical field of medical education, artificial intelligence and multi-modal data processing. The method comprises the following steps: a data acquisition and annotation stage: acquiring medical inquiry video data of different scenes; in the model training stage, a multi-modal deep learning model architecture is used for carrying out cross-modal alignment on multi-modal data; then semantic understanding and question classification are carried out based on the pre-training language model after fine tuning; in the corpus construction and optimization stage, multi-modal data are integrated, a corpus management system is built, dynamic updating and self-adaptive optimization are carried out on a corpus, meanwhile, a typical non-language posture library is constructed for a regional culture background, and teaching is carried out in combination with the corpus. According to the invention, in combination with medical professional knowledge, an efficient and intelligent medical inquiry video teaching corpus is constructed through deep fusion of natural language processing, computer vision, speech recognition and multi-modal data fusion technologies.

The invention discloses a cultural heritage digital display platform, which is mainly applied to the fields of museums, cultural heritage protection institutions, education institutions and the like, and specifically comprises a central control module, and a user interaction module, a multimedia display module, a virtual reality rendering module, a database management module and an intelligent recommendation module which are connected with the central control module. The cultural heritage digital display platform has remarkable beneficial effects in the aspects of improving cultural heritage protection and inheritance efficiency, enhancing user experience and interactivity, realizing personalized recommendation and accurate service, promoting cultural education and propagation, optimizing data management and analysis capability and the like, and is beneficial to protection and inheritance of the cultural heritage; and the cultural literacy and aesthetic level of the public can be improved, and a positive contribution is made to the development of cultural heritage.

The invention provides an integrated sharing method and system for digital media content resources, and is applied to the technical field of data processing. The method comprises the following steps: preprocessing digital media content resources, cloud platform basic data and user demand data to generate a standardized media content sequence, cloud platform features, user demand features and resource classification features; processing the resource association relationship data to generate resource association features; processing a strategy in the shared strategy updating mechanism information based on the resource value evaluation system, and generating updated shared strategy data; processing the standardized media content sequence, the cloud platform feature and the resource association feature based on a target integrated sharing model to generate a target integrated sharing feature; and processing the target integration sharing feature based on the target integration sharing model and the updated sharing strategy data to generate a digital media content resource integration sharing result.

The invention discloses a personalized teaching content generation method and system based on a digital portrait, and the method comprises the steps: collecting the multi-dimensional learning data of a student in real time, analyzing the data keyword of the student, constructing the portrait of the student, generating a teaching target based on the portrait of the student, and enabling the teaching target to comprise a teaching theme, exercises related to the theme, and knowledge points related to the theme. Converting the teaching target into an executable instruction of a large language model through a structured prompt engineering technology; the large language model generates initial teaching content according to the teaching instruction; verifying and optimizing the generated initial teaching content to ensure that the generated teaching content conforms to a teaching target and can adapt to the current cognitive level and learning requirements of students; and outputting the verified and optimized teaching content to the students and automatically adapting to the content presentation form according to the learning style preference of the students.

The embodiment of the invention discloses an intelligent decision-making method, computer equipment and a computer readable storage medium. The method comprises the following steps that when an intelligent agent obtains request content, the intelligent agent performs analysis operation on the request content so as to split at least one thinking path; taking each thinking path as input of a local solving model, and solving the thinking paths according to the local solving model to obtain corresponding candidate answers; summarizing candidate answers corresponding to all thinking paths to obtain a candidate answer set; and taking the candidate answer set as the input of a global meta-learner, and performing fusion processing operation on the candidate answer set according to the global meta-learner to obtain a final answer and outputting the final answer. Therefore, the request can be converted into various thinking paths by utilizing the intelligent agent, and various candidate answer sets are obtained by parallel solution; and carrying out global learning on the candidate answer set by a global meta-learner to obtain a final answer which is optimal in performance and best in conformity with expectation, and outputting the final answer.

The invention discloses a conference content intelligent generation processing method and system based on a multi-modal large model, and the method comprises the steps: collecting the original data of a conference, and completing the standardization preprocessing; inputting a multi-modal large model, extracting multi-modal features and carrying out semantic alignment; executing cross-modal hash coding, generating binary codes and establishing an index database; performing hash retrieval on the related fragments, and constructing a conference content directed graph; based on a conference content directed graph structure, searching an optimized path by adopting a Monte Carlo tree; and generating structured conference content, and outputting a summary, an abstract and an action item. According to the method, efficient extraction, accurate retrieval and structured intelligent generation of the conference content are realized by fusing a multi-modal large model, cross-modal Hash coding and Monte Carlo tree search.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a commercial real estate question and answer content generation system based on a retrieval enhancement technology. The system comprises an original data processing module, a text content slicing module, a knowledge base construction module, a query rewriting module, a problem planning and disassembling module, an intention routing module, a mixed retrieval module and a content generation module. The method has the characteristics that the efficiency is higher, the accuracy is higher, and meanwhile, the use experience of a user is also greatly improved.

The invention provides an intelligent decision-making method and system for emergency allocation of first-aid materials. According to the method, by integrating a graph neural network, deep learning, reinforcement learning, Bayesian optimization and a genetic algorithm, the demand information of multiple emergency response points is efficiently processed, and by receiving the demand information including the number, the type and the severity of injuries and patients, the demand urgency degree is dynamically calculated through an intelligent evaluation system, and a preliminary deployment plan is generated. And a traffic prediction model is combined with real-time and historical data to pre-estimate the road traffic capacity, various logistics distribution strategies are simulated to optimize the preliminary scheme, and an optimal path plan is determined. And then, the material distribution effect is verified in different scenes by means of an AI simulation drilling system, the configuration scheme is adjusted, and finally a comprehensive allocation instruction is formed to guide actual operation. According to the technical scheme of the invention, intelligent decision support for the first-aid materials can be realized, and the efficiency and accuracy of first-aid material allocation are remarkably improved.

The invention provides an ambiguity detection method and device for multi-modal and multi-language information, and the method comprises the steps: employing a ViT image encoder and an XLM-R text encoder to extract image and multi-language text features, achieving the depth interaction between modals through a cross attention mechanism, generating multi-modal fusion features, and carrying out the detection of the multi-modal fusion features. Comprehensively capturing fine-grained association between the image and the text; an image and a text encoder of a pre-training model are frozen by adopting a LoRA fine tuning technology, and only low-rank parameter optimization is performed on a classifier, so that the computing resource demand is remarkably reduced; a dynamic memory mechanism is introduced, the recognition capability of the model on complex discrimination signals is enhanced by retrieving historical sample information and dynamically fusing current features, and newly input data is accurately predicted. According to the method, the language limitation of the existing method is broken through, the accurate detection of the discrimination signal can be realized under the multi-language and multi-culture background, and the blank of the multi-modal discrimination detection in the multi-language environment is filled.

The invention discloses a material labeling method, a model training method, a multimedia content labeling method and a related product. The method comprises the following steps: acquiring a target material and n preset labels; the preset label comprises category information of the material, and n is an integer greater than 1; searching the n preset tags by utilizing target material embedding of the target material to obtain m candidate tags matched with the target material embedding; m is a positive integer smaller than n; according to the m candidate tags and the target material, target cue words are generated, and the target cue words are used for guiding a language model to determine tags with category information matched with the target material from the m candidate tags; and under the guidance of the target cue word, determining a label with category information matched with the target material in the m candidate labels by utilizing a language model, and taking the label as a target label.

The invention relates to a teaching content display data determination method and device and computer equipment. The method comprises the following steps: acquiring teacher teaching state information and student learning state information of a target classroom; according to the teacher teaching state information and the student learning state information, adjusting the initial teaching content of the target classroom to obtain optimized teaching content; according to the teacher teaching state information and the student learning state information, the initial content display mode of the target classroom is adjusted, and optimized content display data is obtained; integrating the optimized teaching content into the optimized content display data to obtain content integration matching information; and performing matching optimization on the content fusion matching information to obtain target teaching content display data. By adopting the method, a better teaching effect of teachers in classroom teaching can be achieved.