Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

1931 results about "Text graph" patented technology

In natural language processing (NLP), a text graph is a graph representation of a text item (document, passage or sentence). It is typically created as a preprocessing step to support NLP tasks such as text condensation term disambiguation (topic-based) text summarization, relation extraction and textual entailment.

Decision-making method and model for offline reinforcement learning and continuous online fine tuning

The invention discloses a decision-making method and model for off-line reinforcement learning and continuous on-line fine tuning, and the method comprises the steps: obtaining original text-image pair data and historical track data, and obtaining an enhanced state sequence through feature matching verification and standardization processing; performing sequence prediction by using a multi-level attention mechanism, and obtaining pre-training model parameters through knowledge distillation and task adaptability enhancement; on the basis of stability evaluation and plasticity control, online knowledge migration is carried out on the model, and optimized model parameters are obtained by combining experience playback optimization; and finally, performing multi-dimensional performance evaluation and resource optimization. According to the method, through multi-modal feature fusion, dynamic knowledge migration and an adaptive optimization mechanism, the problems of insufficient feature representation, disastrous forgetting, low optimization efficiency and the like are effectively solved, and the learning efficiency and generalization ability of the model are improved.
Owner:NANJING ARTIFICIAL INTELLIGENCE CHIPS RES INST OF AUTOMATION CHINESE ACAD OF SCI +1

Text mining data query method and system based on cross-modal similarity

The invention provides a cross-modal similarity-based text mining data query method and system, relates to the technical field of data mining, comprises a cross-modal similarity-based text mining data query method and system, and belongs to the technical field of natural language processing. The method comprises the following steps: constructing a multi-level feature extraction network to respectively obtain text, image and time sequence feature vectors; feature fusion is realized by using an adaptive cross-modal contrast learning network, and a cross-modal similarity matrix is constructed; performing hierarchical screening on the target text by adopting a multi-level screening model to obtain a candidate text set; a text relation reasoning module is constructed based on a graph neural network, and text clustering and relation prediction are completed through a multi-task learning framework. According to the method, the accuracy of cross-modal text data query is improved, the adaptivity of feature fusion is enhanced, and efficient text relationship mining is realized.
Owner:JIANGXI SHUYI TECH CO LTD

Semantic comprehension driven cross-modal information fusion and retrieval method and system

The invention discloses a cross-modal information fusion and retrieval method and system driven by semantic comprehension, and the method comprises the steps: obtaining text, image and audio original data, and extracting an initial feature set of each modal through a deep neural network; dynamically distributing each modal weight coefficient based on an attention mechanism, and performing weighted fusion on the initial feature set to obtain cross-modal fusion feature representation; through a cross-modal semantic association analysis model, high-dimensional semantic association features are extracted from the fusion feature representation, and semantic enhancement feature vectors are generated; constructing a cross-modal semantic graph network based on the vector, complementing missing modal features, and generating an optimized multi-modal feature set; and inputting the optimized feature set and the query sample into a contrast learning model, calculating a semantic similarity score, and generating a cross-modal retrieval result sorting list according to the score.
Owner:SHANGHAI CIVIL AVIATION VOCATIONAL & TECH COLLEGE

Intelligent file classification and retrieval method and system

The invention discloses an intelligent file classification and retrieval method and system, and the method comprises the steps: carrying out the unified format conversion and feature extraction of text, image and audio data according to the original data format of a file through a multi-modal data fusion algorithm, and obtaining a standardized file feature data set; based on a deep learning adaptive classification model, performing classification processing on the standardized archive feature data set to generate multi-level archive classification labels; analyzing the semantic relationship between the archive classification labels through an enhanced semantic network construction technology, generating a semantic association map, and optimizing the semantic association map to obtain an optimized semantic association map; and according to a retrieval request input by a user, performing semantic matching and correlation sorting by utilizing the optimized semantic correlation graph, and outputting archive information most related to the retrieval request. By utilizing the embodiment of the invention, the relevance and accuracy of retrieval can be improved, and modernization and intellectualization of archive management are promoted.
Owner:HANGZHOU YUNJIA TECH CO LTD

Knowledge graph-combined brilliance large-scale model culture knowledge generation and retrieval method

The invention relates to a brilliance large-scale model culture knowledge generation and retrieval method combined with a knowledge graph. The method comprises the steps that entity extraction, relation modeling and dynamic completion processing are carried out according to multi-modal culture data, a culture knowledge graph is constructed in combination with a fuzzy time reasoning algorithm and a cross-modal alignment mechanism, and the multi-modal culture data comprises an unstructured culture text, image data and space-time metadata; the space-time metadata comprises text space-time metadata and image moment metadata; performing semantic alignment processing by combining a large model through a user query statement and a culture knowledge graph to obtain joint representation of cross-modal semantic alignment; and performing multi-modal retrieval-generation joint optimization based on the joint representation to obtain cultural knowledge description and associated multi-modal content. According to the method, through multi-modal data fusion, dynamic knowledge completion and fuzzy time reasoning, culture knowledge description can be accurately generated, associated multi-modal content can be retrieved, and the accuracy and the intelligent level of culture knowledge service are improved.
Owner:JINAN DONGDE CONSTRUCTION LABOR CO LTD

Cross-modal knowledge reasoning method based on multi-modal large model

The invention relates to a cross-modal knowledge reasoning method based on a multi-modal large model. In a cross-modal knowledge reasoning process, an existing model is usually limited by single-modal information extraction and shallow feature fusion, so that deep semantic association among data such as texts, images and videos is difficult to fully capture. In order to solve the problem, the invention provides a model for fusing multi-modal information such as texts, images, videos, documents and the like, and processing of multi-modal data is converted into unified feature extraction, interaction and deep reasoning tasks by fully utilizing a supervision fine tuning strategy, a self-adaptive attention mechanism and a cross-language processing technology. The model adopts a modular design, integrates multi-source data complementary analysis, spatial-temporal feature modeling and emotional semantic analysis, and realizes multi-modal collaborative interaction, dynamic scene understanding, long video key event analysis and man-machine co-emotional response. Through sufficient training, the multi-modal large model shows excellent logical reasoning ability and emotion understanding ability in a complex cognitive task, and a brand new solution is provided for efficient extraction, deep semantic analysis and intelligent response of cross-modal information.
Owner:SHENYANG INST OF COMPUTING TECH CO LTD THE CHINESE ACAD OF SCI

AI-driven multilingual intelligent document abstract and mind map generation system

The invention relates to the technical field of artificial intelligence, in particular to an AI-driven multilingual intelligent document abstract and mind map generation system, which realizes unified representation of data such as multilingual texts, images and tables through a multi-modal semantic feature generation module, and constructs a dynamically optimized causal semantic map; generating a semantic flow path with clear logic based on a dynamic semantic flow modeling method, wherein the semantic flow path is used for multi-language abstract generation; the structure of the mind map is optimized by using the generative adversarial network, and the content of the map is further perfected through user interaction adjustment and logic consistency verification. The system supports a user to efficiently extract document core information, a logic structure is displayed in a visual mode, the accuracy, logicality and efficiency of document generation are remarkably improved, and the system is suitable for generation and decision support of complex multi-modal documents.
Owner:BEIJING SOIN TECH CORP LTD

Multi-modal bill processing method based on dynamic knowledge enhancement

The invention discloses a multi-modal bill processing method based on dynamic knowledge enhancement. The multi-modal bill processing method comprises the following steps: S1, constructing a dynamic knowledge base containing an aging weight; s2, synchronously processing text, image and format features of the bill by adopting a multi-modal feature fusion network to generate a composite feature vector; s3, semantic-level, format-level and timeliness three-level fusion retrieval is carried out based on the composite feature vector, and a three-level fusion retrieval engine comprises dynamic weighted sorting with timeliness attenuation, a difference degree triggered artificial review mechanism and a policy sensitive slope adjustment algorithm; s4, setting a multi-expert cooperative verification system, wherein the multi-expert cooperative verification system comprises cooperative work of a rule engine, a large language model and a logical reasoning module; s5, implementing a dynamic knowledge updating mechanism, and automatically triggering incremental learning of the knowledge base when policy change or format update is detected; and S6, outputting structured data, and synchronously generating an auditing traceability chain containing a decision path. According to the method, the key field identification accuracy can be improved, and auditing traceability and non-perceptual increment updating in the whole process are realized.
Owner:HUAIYIN INSTITUTE OF TECHNOLOGY

PDF extraction method and system based on deep learning and layout analysis

The invention relates to a PDF extraction method and system based on deep learning and layout analysis, and the method comprises the steps: obtaining a PDF file, carrying out the element recognition of a scanning version file and a text version file through an OCR technology and a deep learning model, and enabling the elements to comprise a text, a picture, a table and a formula; dividing a page of the PDF file into a plurality of logic blocks according to the identified elements, and determining a hierarchical relationship and a sequence; deleting elements without substantial help from the identified elements, and extracting the content of the document; performing page layout analysis according to the document content, and reordering and organizing each logic block to obtain a layout; document contents are classified and analyzed, and then classified storage is carried out; and according to user requirements, splicing the classified and stored document contents in a corresponding layout to generate a document in a target format. Compared with the prior art, efficient and accurate PDF extraction is achieved, and meanwhile the important structure and content of an original document are reserved.
Owner:SHANGHAI ARTIFICIAL INTELLIGENCE INNOVATION CENT

Document retrieval method based on multistage index and feature clustering

The invention relates to the technical field of document retrieval and information processing in the data processing technology, in particular to a document retrieval method based on multistage indexing and feature clustering, which comprises the following steps: performing high-dimensional space mapping on multi-modal features such as texts and images through a quantum embedding layer to generate cross-modal joint feature representation; a first-level index of a multi-level index architecture is dynamically initialized based on a meta-clustering algorithm, and semantic blocks of a second-level index are divided in combination with a multi-head self-attention mechanism. And an optimal transmission matrix is generated by using a Sinkhorn algorithm to align cross-node feature distribution. The multi-target mixed retrieval strategy is fused with vector retrieval, keyword retrieval and graph retrieval results, and weight distribution is dynamically adjusted. Through collaborative optimization of quantum calculation, federated learning and causal reasoning, a closed-loop technical architecture from feature analysis to dynamic index construction is formed, the problems of insufficient cross-modal fusion, static clustering deviation and semantic association deficiency are solved, and the precision, efficiency and dynamic adaptability of heterogeneous document retrieval are improved.
Owner:TAIJI COMPUTER CORPORATION LIMITED

Text image generation method and diffusion generative model training method

Provided in the embodiments of the present disclosure are a text image generation method and a diffusion generative model training method. The text image generation method comprises: acquiring an initial image; inputting the initial image into a conditional encoder, so as to obtain an image feature of the initial image; inputting the initial image and the image feature into a diffusion generative model, so as to obtain noise data corresponding to the initial image, wherein the diffusion generative model is obtained by means of performing training on the basis of a sample text image, a sample image feature of the sample text image and a noise sample image, the noise sample image is obtained by means of adding sample noise to the sample text image, and the sample image feature comprises at least one of a sample visual feature and a sample semantic feature; and according to the initial image and the noise data, generating a target text image corresponding to the initial image.
Owner:ALIBABA INNOVATION PRIVATE LIMITED

Text detail map-based method for supervising end-to-end text detection and recognition

A text detail map-based method for supervising end-to-end text detection and recognition, pertaining to the field of text processing. The method comprises the following steps: given an input image containing text in any shape, processing the input image by means of two separate processing branches; designing a text attention head (TAH), and designing a feature pyramid enhancement fusion module (FPEFM); the FPEFM performing feature self-enhancement at different sizes, fusing text image local features and global text position information extracted by a TAH module, and fusing features extracted by the TAH from feature maps of different sizes; stacking a plurality of FPEFMs to continuously enhance the feature representation capability of a model and the depth of the model; and sampling the feature maps to a unified size to obtain a final enhanced feature map.
Owner:CHONGQING UNIV OF TECH

Multi-source heterogeneous data fusion knowledge graph method and system

The invention relates to the technical field of knowledge maps, in particular to a knowledge map method and system for multi-source heterogeneous data fusion. The method comprises the following steps: cleaning multi-source data through an auto-encoder, dynamically weighting and standardizing after low-rank decomposition and PCA denoising, aligning GNN entities and authenticating standard approval serial numbers; combining CNN / GNN to extract texts, images and sensor multi-modal features, fusing time sequence information with BiLSTM, and performing weighted aggregation; a BERT-BiLSTM-SelfAttention-CRF is adopted to identify an entity, a GCN inference relationship is adopted, a TransE is embedded into an entity relationship to a low-dimensional space, and the entity relationship is stored to Neo4j to support real-time query; dynamically updating the atlas by incremental learning; and visually displaying the constructed knowledge graph. According to the method, a complete closed loop from data cleaning, multi-modal fusion and graph construction to dynamic optimization is formed, and comprehensiveness, accuracy and expandability of the knowledge graph in a multi-source heterogeneous scene are ensured.
Owner:NORTHWEST INST OF ECO ENVIRONMENT & RESOURCES CAS

Advertisement copywriting generation method and device based on multi-modal fusion, equipment and medium

The invention discloses an advertisement copywriting generation method and device based on multi-modal fusion, equipment and a medium, and relates to the technical field of advertisement marketing. The method comprises the steps of obtaining multi-modal data of a target video, the multi-modal data comprising visual data, auditory data and related metadata of the video, and extracting associated data from a local knowledge base; and preprocessing the multi-modal data and the local knowledge base data, and converting the multi-modal data and the local knowledge base data into feature forms which can be used for analysis. According to the method, semantic calibration is carried out on multi-modal input by means of a local knowledge base, it is ensured that generated content strictly follows domain knowledge constraints, the problem of deviation caused by the fact that a traditional model depends on implicit knowledge is solved, multi-modal information ambiguity is eliminated, collaborative semantic generation of texts, images and structured data is achieved, content dimensions are enriched, and the method has the advantages of being high in practicability and easy to popularize. And an efficient solution is provided for the landing of the intelligent generation technology in the vertical field.
Owner:SHANGHAI WANGMAI INFORMATION TECH GRP CO LTD

LLM-driven complex report OCR error self-correction method and system

The invention discloses an LLM-driven complex report OCR error self-correction method and system, and the method comprises the following steps: S1, obtaining complex report image data, executing OCR processing, and constructing an original field data set; s2, extracting context information, identifying semantic contradiction fields, and generating a to-be-corrected field set; s3, the pointer generation network generates a plurality of field correction candidates to form a candidate field set; s4, constructing a dobby machine model, selecting an optimal field correction result, and forming a correction field output set; s5, executing format analysis, and extracting a chart title field, a legend field and a data region text; s6, generating a corrected field result of a chart title field according to a chart structure semantic consistency mechanism; and S7, performing field restoration and format reconstruction, and outputting structured report data. According to the method, intelligent error correction and structured reconstruction of fields in a complex report are realized by fusing a large language model, a pointer generation network and a multi-arm machine mechanism.
Owner:ZHEJIANG FULIN TECH CO LTD

Multi-agent dynamic arrangement method based on multi-modal analysis and adaptive retrieval

The invention discloses a multi-agent dynamic arrangement method based on multi-modal analysis and adaptive retrieval, and relates to the technical field of artificial intelligence and information retrieval. Comprising the steps of S1, converting a text, an image, structured data and voice content input by a user into a unified multi-mode semantic representation, S2, converting the unified multi-mode semantic representation into a specific execution process, and S3, automatically scheduling a reasoning agent, a knowledge obtaining agent and an execution agent according to DAG nodes, task elements and available resources, and obtaining the task elements and the execution agent according to the reasoning agent, the knowledge obtaining agent and the execution agent. S4, after task process construction and agent arrangement are completed, dynamic retrieval, evidence convergence and strategy optimization are carried out on information requirements related to a user task, so that a reasoning agent obtains complete knowledge support with consistent context, and S5, knowledge evidence is combined with a task process, so that the task process is completed. The method comprises the following steps: step S6, implementing problem solving, strategy generation and task closed-loop execution through a reasoning agent, step S6, performing actual operation on a target task by an execution agent according to an executable instruction sequence output by the reasoning agent, and outputting a result, and step S7, performing result verification according to an output result returned by the execution agent, and the correctness, integrity and consistency of an output result are examined through rule verification, model evaluation and evidence alignment.
Owner:INSPUR GROUP CO LTD +1

Digital Content Creation

Methods, systems, and apparatus, including computer-readable storage media for generating model-generated digital content based on prompts and other inputs. The prompts may be, for example, inputs for modifying digital content associated with an existing campaign. The prompts may include national language input. The natural language prompts may include, for example, a theme, information related to an upcoming event, the topic of the campaign, or the like. The other inputs may also include information associated with the campaign creator's style. The information associated with the campaign creator's style may be in the form of embeddings. A system implementing the methods described herein can generate the digital content for the modified campaign based on the received inputs received from the campaign. The generated digital content may be in the form of text, image, or the like.
Owner:GOOGLE LLC

Long-tail image recognition method based on multi-modal semantic generation and image-text fusion

The invention discloses a long-tail image recognition method based on multi-modal semantic generation and image-text fusion. The method comprises the following steps: extracting structured semantic description from a tail image; carrying out semantic rewriting and enhancement based on a multi-modal visual language model, and generating an image semantic extension description; the image semantic extension description is optimized based on semantic duplicate judgment and a style alignment mechanism, and an optimized text description set is obtained; inputting the optimized text description set into a text graph model, generating a tail class image sample, performing semantic and visual quality screening, and constructing to obtain an enhanced image set for training; constructing a training data set based on the original long-tail data set and the enhanced image set, and training an image-text fusion classification model; and inputting a to-be-identified image into the trained image-text fusion classification model, and outputting classification results of all categories. According to the method, the discrimination capability in a long-tail distribution scene is enhanced, and the method has a stronger generalization characteristic.
Owner:SOUTH CHINA UNIV OF TECH

Artificial intelligence-powered large-scale content generator

An AI-powered content generation system that creates consistent, coherent, and engaging multi-modal content by integrating multiple specialized AI components. The system analyzes user input, identifies key elements, and maintains continuity throughout the generation process. It incorporates a feedback loop to learn and adapt based on user preferences, enabling personalized content experiences. The modular architecture allows for seamless integration of AI components focusing on text, images, audio, and interactive elements. The system ensures consistency across modalities and over extended periods, while managing rights, licenses, and royalties using blockchain technology. This advanced platform revolutionizes content creation, consumption, and management in the digital age.
Owner:QOMPLX INC

An intelligent prediction system for social public opinion risk

This application relates to the field of home kitchens, and discloses a social public opinion risk intelligent prediction system, including: a data collection module, which is used to collect text, image and video data from multi-source heterogeneous data sources in real time, including social media, news websites, forums, blogs and self-media; a data preprocessing module, which is used to clean, normalize, segment and tag the collected data to form a structured data set; an advanced natural language processing module, which is used to use the pre-trained Transformer model and graph neural network to perform deep semantic analysis on the pre-processed text data, and extract entity, relationship and context semantic features. By integrating advanced natural language processing, deep learning and multimodal data fusion technology, the system can monitor and accurately predict social public opinion in real time, quickly capture emergencies and public opinion changes, and deeply analyze complex semantics and emotional tendencies.
Owner:吉林警察学院

Large model Agent intelligent decision-making method and system fusing multi-modal data

The invention discloses a multi-modal data fused large model Agent intelligent decision-making method and system, belongs to the technical field of artificial intelligence, multi-modal data processing, deep learning, reinforcement learning and intelligent decision-making, and aims to solve the technical problem of how to improve the performance and adaptability of intelligent decision-making in processing complex tasks and dynamic environments. According to the technical scheme, the method comprises the steps of multi-modal data fusion, wherein text, image and audio data from different modals are integrated, and unified feature representation is generated through feature extraction and feature fusion technologies; intelligent decision-making: decision-making reasoning is carried out based on the fused feature representation, and a final decision-making result is generated by adopting a deep learning model and a reinforcement learning algorithm; adaptive learning: monitoring data changes and decision-making effects in real time, and dynamically adjusting deep learning model parameters and strategies; and feedback optimization: further optimizing the performance of the deep learning model by collecting the feedback information of the decision result.
Owner:浪潮智慧城市科技有限公司

Text-to-image system and method

Techniques and systems for training and / or implementing a text-to-image generation model are provided. A pre-trained multimodal model is leveraged for avoiding slower and more labor-intensive methodologies for training a text-to-image generation model. Accordingly, images without associated text (i.e., bare images) are provided to the pre-trained multimodal model so that it can produce generated text-image pairs. The generated text-image pairs are provided to the text-to-image generation model for training and / or implementing the text-to-image generation model.
Owner:ADOBE INC

Multi-modal natural language understanding and generating system and method

The invention discloses a multi-modal natural language understanding and generating system and method. The method comprises the following steps: constructing a cross-modal pre-training module, training a multi-modal encoder, and establishing a cross-modal association mapping space; mixing prompt fine tuning is carried out, and a complete blank filling template is constructed; according to the intention reasoning network, extracting multi-round dialogue intention representation of the user, and retrieving an external knowledge base for fine-grained reasoning; constructing a unified semantic representation framework, embedding the text, the image and the voice into a unified space, and generating a query vector of multi-modal intention perception; and the knowledge query module based on key value memory generates entity-level multi-modal replies and optimizes the semantic comprehension and generation capability of the dialogue model. According to the method, the multi-modal information understanding and generating capacity is improved, deep association and understanding of image and text information are achieved, downstream task adaptability is enhanced, task completion accuracy and efficiency are improved, unified semantic representation of the multi-modal information is achieved, and support is provided for information retrieval and utilization.
Owner:UNIV OF ELECTRONIC SCI & TECH OF CHINA CHENGDU COLLEGE

Multi-modal heterogeneous model retrieval enhancement method and system

The invention provides a multi-modal heterogeneous model retrieval enhancement method and system, and the method comprises the steps: building a knowledge and application example double-corpus based on user multi-modal query, and designing a joint retrieval mechanism to obtain a result set; mapping and scheduling to obtain feature representation through special processing channels for texts, images and audios and a Spiking neural network with a segmented trapezoidal topological structure; constructing a three-stage cascade architecture of a basic model, an advanced model and human experts, and obtaining a decision path and answer candidate set in combination with a recursive and discarding decision mechanism; a Hamiltonian graph network is used for representing a multi-modal relation, and a gradient-free descent method is used for rapidly training and optimizing model parameters; an enhanced retrieval result is obtained through cross-modal semantic alignment and dynamic retrieval window adjustment; and high-quality response is obtained through context-aware sorting and retrieval enhanced reasoning. According to the method, the multi-modal information retrieval processing efficiency and the heterogeneous model reasoning response quality are improved.
Owner:贵州中汇科技发展有限公司

Traffic large model construction and decision-making method and device based on multi-modal two-way map reasoning

The invention discloses a traffic large model construction and decision-making method and device based on multi-modal two-way map reasoning, and the method comprises the steps: constructing a multi-modal data set of a text, an image and a track, generating fusion features through spatial-temporal clustering and cross-modal Transform coding, carrying out the two-way map reasoning in combination with a traffic knowledge map, and carrying out the decision-making of the traffic large model. The method comprises the following steps: generating an embedded representation through a forward graph neural network, reversely mapping a decision scheme generated by a language model to a graph to verify consistency, outputting knowledge to enhance embedding, fusing multi-modal features and knowledge embedding by adopting an LoRA multi-task joint fine tuning technology, adapting to traffic field tasks, deploying a real-time inference engine, and carrying out real-time inference on the traffic field. And processing the dynamic data flow through an aging perception attention mechanism, and outputting traffic event identification, path planning and scene question and answer results in parallel. Compared with the prior art, the method has the advantages that the problems of insufficient multi-source heterogeneous data fusion, low knowledge utilization efficiency and poor real-time decision consistency can be solved, and the semantic understanding and decision accuracy of the traffic large model is effectively improved.
Owner:HUAIYIN INSTITUTE OF TECHNOLOGY

Unmanned aerial vehicle knowledge graph construction method based on multi-modal large model recognition

The invention relates to an unmanned aerial vehicle knowledge graph construction method based on multi-modal large model recognition, and the method comprises the steps: carrying out the data fusion of texts, images, videos and sensor data related to an unmanned aerial vehicle, and constructing an unmanned aerial vehicle multi-modal data set; constructing a multi-modal large model integrated with a Transform architecture and a self-attention mechanism for extracting structured information from input data, and training the multi-modal large model by using a multi-modal data set of the unmanned aerial vehicle; structured information is extracted from unmanned aerial vehicle data input in real time by using the trained multi-modal large model to construct a knowledge graph, knowledge graph data is updated regularly through an online learning mechanism, and the quality and consistency of the knowledge graph are ensured by adopting an automatic verification and error correction mechanism. According to the method, the defects of low accuracy and low efficiency in unmanned aerial vehicle knowledge graph construction in the prior art are overcome, and the construction quality of the knowledge graph is improved.
Owner:CHINA ACAD OF AEROSPACE SCI & TECH INNOVATION

Multi-mode RAG knowledge question-answering method and device applied to vertical field

The invention provides a multi-modal RAG knowledge question answering method and device applied to the vertical field, relates to the technical field of artificial intelligence, supports multi-modal input of to-be-answered texts including texts, pictures, audios and videos, and makes full use of semantic information of multi-modal data. And utilizing a plurality of preset retrieval models to retrieve the knowledge fragments corresponding to the to-be-answered text to obtain a plurality of retrieval results. Various retrieval results are fused through an RRF algorithm, and scores are calculated by using a target reordering model for reordering, so that the recall rate is improved. And based on the retrieval content and the preset knowledge frequency information, fine tuning sorting is further carried out, so that the method is more accurate and meets user expectation. According to the method, the natural language understanding ability of a large language model, a target reward model and related information recalled by the RAG are combined to generate optimal solution data, and meanwhile, a target scoring model and domain expert knowledge are utilized to provide more professional high-quality solution data meeting actual requirements.
Owner:LINKAGE TECHNOLOGY (NANJING) CO LTD

Multi-modal data fusion method and system based on large model agent

The invention discloses a multi-modal data fusion method and system based on a large-model intelligent agent, belongs to the technical field of artificial intelligence, big data processing and intelligent agents, and aims to solve the technical problem of how to effectively integrate data of different modalities such as texts, images and voices by using the large-model intelligent agent. According to the technical scheme, the method comprises the steps that data collection and preprocessing are conducted, specifically, various modal data of texts, images and voice are collected through a web crawler, an API interface, a camera and microphone equipment, the collected data are preprocessed, the preprocessed multi-modal data are obtained, and the data quality is ensured; feature extraction and mapping: extracting corresponding modal features from the preprocessed multi-modal data through CNN and Transform models, mapping the different modal features to the same space, and combining the aligned features to form comprehensive feature representation; carrying out multi-modal fusion processing; and performing intelligent decision and feedback.
Owner:浪潮智慧城市科技有限公司

Embedded reconstructed text-image alignment style migration method

The invention relates to an innovative embedded reconstructed text-image alignment style migration method, which comprises the following steps of: constructing a basic framework based on a diffusion model, and adding noise through a forward process and denoising to learn image style characteristics. And extracting style embedding from the artwork data set by utilizing perceptron attention and a feedforward network. Text embedding is generated through a text encoder, and guidance is provided for style migration. A cross attention mechanism is adopted to fuse style embedding and text embedding, multi-modal embedding is generated, and interaction of text and image information is optimized. And fusing multi-modal embedding and image embedding through linear interpolation, splicing with text embedding to form complete prompt embedding, and integrating the prompt embedding into a diffusion model to generate a style image matched with text description. According to the method, the quality and diversity of artistic image style migration are remarkably improved, the method has wide application potential in the field of artistic creation and design, rich and diversified stylized visual achievements can be provided, and the requirements of multiple creation scenes are met.
Owner:TSINGHUA SHENZHEN INTERNATIONAL GRADUATE SCHOOL

Multi-modal fusion rumor detection method and system based on dynamic graph convolutional neural network

The invention discloses a multi-modal fusion rumor detection method and system based on a dynamic graph convolutional neural network. According to the method, a dynamic feature graph of a language propagation path is constructed, and potential features in the language propagation process are extracted and analyzed by utilizing time sequence changes and key node relations between nodes in a propagation graph. A neural network is adopted to extract and enhance image data, text semantic features are extracted in combination with a text feature modeling network, text feature vectorization expression is achieved based on a BERT model, and rich semantic information is obtained. And a gating mechanism is introduced to dynamically adjust fusion weights of different modal features, and an information fusion strategy is optimized. A collaborative attention mechanism is further adopted for deep fusion, interactive learning of text, image and propagation path features is enhanced, and the relevance of cross-modal and time series data is improved. And finally, inputting the fused feature vectors into a classifier for accurate classification, thereby realizing accurate detection of the social media rumors. According to the method, the multi-modal features are effectively integrated, and the false information identification efficiency is remarkably improved.
Owner:CHONGQING UNIVERSITY OF SCIENCE AND TECHNOLOGY +1