A large model rag retrieval method and system for mixed text and image output

The RAG retrieval method using large-scale models with mixed text and image output solves the accuracy problem of large models when processing multimedia data, achieves efficient mixed retrieval, and improves the effectiveness and application scope of question answering systems.

CN120763309BActive Publication Date: 2026-04-21XIAN COAL SCI TRANSPARENT GEOLOGICAL TECH CO LTD
2 Cites 0 Cited by

Patent Information

Application Number
CN202511142513.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-08-15
Publication Date
2026-04-21
Estimated Expiration
2045-08-15

AI Technical Summary

Technical Problem

Existing large-scale models struggle to generate accurate question-and-answer results when processing data containing text, images, and charts. In particular, large-scale text-based language models cannot output image information, thus limiting their application scenarios.

Method used

The large-scale RAG retrieval method, which uses a hybrid image and text output, generates a corpus tree by preprocessing the original documents, performs retrieval using word vectors and an improved Gaussian mixture clustering algorithm, generates the most relevant corpus, and combines image and table information with text to output the final question-and-answer results.

Benefits of technology

It enables efficient mixed retrieval of text, image, and chart data, improves the accuracy and efficiency of question-and-answer results, expands the application scenarios of large models, breaks the bias that large text language models can only process text information, and adds the ability to retrieve images.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

This invention belongs to the field of large-scale artificial intelligence model technology and discloses a method and system for large-scale RAG retrieval with mixed text and image output. The method preprocesses the original document corpus to generate a corpus tree, and processes the corpus of images, tables, etc., generating key-value pairs which are stored in a database. It generates word vectors for question-and-answer statements and searches them against the corpus tree to find the most relevant corpus. The most relevant corpus is concatenated into a prompt and then sent to the large-scale text-to-speech model. The large-scale question-and-answer model outputs relevant results. Based on the results output by the large-scale text-to-speech model, the key information of images and tables in the results is retrieved and replaced to generate the final question-and-answer results. This invention solves the problem of existing technologies struggling to handle data containing text, images, and charts, and generates highly accurate final question-and-answer results.
Need to check novelty before this filing date? Find Prior Art