PDF intelligent retrieval and generation method and system based on RAG

Abstract

The invention discloses a PDF intelligent retrieval and generation method and system based on RAG, and the method comprises the steps: obtaining input document data, employing a pre-established classification model to analyze the document data, and extracting text content and image content to form a first data set; performing feature extraction on the image content in the first data set by adopting a deep learning model, and performing semantic analysis on the text content in the first data set by applying a natural language processing technology to obtain a multi-modal feature set; according to the multi-modal feature set, performing unified coding processing by using an information integration algorithm to generate a second data set, and if it is detected that the integrity of a fusion feature vector in the second data set is lower than a preset threshold value, supplementing context semantic analysis to fill missing information; and performing clustering processing on the fusion feature vectors in the second data set by adopting a preset index construction mechanism to generate a retrieval index database containing a classification index structure. According to the method, the accuracy and comprehensiveness of document retrieval are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of document processing and intelligent retrieval, and particularly discloses a PDF intelligent retrieval and generation method and system based on RAG. BACKGROUND

[0002] Under the background of rapid development of information digitization, the field of document processing and intelligent retrieval is particularly critical. This field is directly related to how to efficiently extract valuable information from massive unstructured data to meet the growing demand of users for knowledge acquisition, especially when dealing with widely used document formats, its importance is self-evident. However, many current solutions often have low processing efficiency, insufficient extraction accuracy and single output form when facing complex document content, which is difficult to fully meet the diversified needs of users for information acquisition. Specifically, the challenges in this field mainly focus on how to accurately extract information from complex documents and generate user-friendly results.

[0003] The first problem is the diversity and unstructured nature of document content. Different formats and qualities of documents often contain various elements such as text and images, and traditional processing methods are difficult to uniformly process these heterogeneous data, resulting in limited information extraction integrity.

[0004] Due to the incompleteness of information extraction, the subsequent retrieval link is also affected, making it difficult to accurately match the relevant content of user needs, and thus causing the final generated results to deviate greatly from user expectations.

[0005] This chain problem from extraction to retrieval to output makes the efficiency and accuracy of the entire information processing process face severe challenges.

[0006] Therefore, how to efficiently extract and integrate multi-modal information in complex documents, accurately retrieve relevant content based on user needs, and finally generate intuitive and easy-to-understand output forms has become a key problem that needs to be solved. The solution to this problem will directly determine whether the goal of intelligently extracting information from massive documents and meeting the diversified needs of users can be truly achieved. SUMMARY

[0007] The present application provides a PDF intelligent retrieval and generation method and system based on RAG, aiming to solve at least one of the defects in the prior art.

[0008] One aspect of the present application relates to a PDF intelligent retrieval and generation method based on RAG (Retrieval-Augmented Generation), comprising the following steps: The input document data is acquired, a pre-established classification model is used to analyze the document data, and text content and image content are extracted to form a first data set; A deep learning model is used to extract features from the image content in the first data set, and a natural language processing technique is applied to the text content in the first data set to perform semantic analysis, obtaining a multi-modal feature set; According to the multi-modal feature set, an information integration algorithm is applied for unified coding processing to generate a second data set, and if the integrity of the fusion feature vector in the second data set is detected to be lower than a preset threshold, context semantic analysis is supplemented to fill in the missing information; A preset index construction mechanism is used to cluster the fusion feature vector in the second data set to generate a retrieval index library containing a classification index structure; According to the user input query condition, a similarity algorithm is used to match the retrieval index library, and if the relevance of the matching result is lower than a preset threshold, the semantic query range is expanded for re-matching.

[0009] Further, the step of acquiring input document data, using a pre-established classification model to analyze the document data, and extracting text content and image content to form a first data set includes: The original content is acquired from the input document data, and a pre-established classification tool is used to preliminarily analyze the document to extract text content and image content to form an initial set; For the initial set, the content recognition tool is used to determine its integrity, and if the text content is missing, the document scanning tool is used to acquire complete data to obtain a first text set; If the image content resolution is lower than a preset threshold, the image enhancement tool is used for processing to obtain a first image set; For the first text set, a text segmentation tool is used for segmentation processing to acquire segmented text units, and a semantic matching tool is used to determine the business category of the text units based on the text units to obtain a second text set; For the first image set, an image segmentation tool is used to split it into multiple image units, a feature extraction tool is used to acquire visual features of the image units, and if the feature value is lower than a preset threshold, an image optimization tool is used to adjust to obtain a second image set; According to the second text set and the second image set, a data fusion tool is used for association mapping to form a unified business data set, a content verification tool is used to detect the consistency of the data set to determine whether it meets the classification target, and a final first data set is obtained.

[0010] Further, the step of adopting a deep learning model to extract features of image content in the first data set, and simultaneously applying natural language processing technology to the text content in the first data set for semantic analysis to obtain a multi-modal feature set includes: For the image content in the first data set, an image segmentation tool is used to split it into multiple visual element units, and a feature extraction tool is used to process the visual element units to obtain corresponding image features. If the value of the image features is lower than a preset threshold, an image enhancement tool is used to adjust to obtain an adjusted image feature set; According to the adjusted image feature set, a feature mapping tool is used to compare the image features with a preset classification standard. If it does not meet the standard, a feature optimization tool is used for secondary processing to determine a subset of image features that meet the standard; For the text content in the first data set, a text segmentation tool is used to split it into multiple language units, and a semantic analysis tool is used to process the language units to obtain corresponding text semantic features to obtain a text semantic set; Through a data integration tool, the image feature subset and the text semantic set are multi-modal fused, and a content classification tool is used to match the fused data to a business to determine whether it meets a preset target to obtain a final multi-modal feature set.

[0011] Further, according to the multi-modal feature set, an information integration algorithm is applied for unified encoding processing to generate a second data set. If the integrity of the fused feature vector in the second data set is detected to be lower than a preset threshold, the step of supplementing context semantic analysis to fill in missing information includes: According to the multi-modal feature set, an information integration tool is used to uniformly process feature encoding to generate a preliminary fusion data set. The encoding consistency of the preliminary fusion data set is detected. If the consistency is detected to be lower than a preset threshold, a data calibration tool is used for adjustment to obtain a fusion data set with consistent encoding; For the fusion data set with consistent encoding, a feature detection tool is used to evaluate the integrity of the vector. If it is found that the integrity of the fusion data set is lower than a preset threshold, a context analysis tool is used to extract supplementary information to determine a fusion data set with improved integrity; Through an information filling tool, for the missing part in the fusion data set with improved integrity, a semantic supplement tool is used for content filling to obtain a filled fusion data set; A business matching tool is used to classify and compare the filled fusion data set. According to pre-established rules, the fusion data set is judged to obtain a final data set that meets the business target.

[0012] Further, the step of generating a retrieval index library containing a classification index structure by clustering the fusion feature vectors in the second data set using a preset index construction mechanism comprises: For the fusion features in the second data set, a preliminary grouping processing is performed on the feature vectors using a preset mechanism, and a preliminary classification result is obtained through clustering operation; According to the preliminary classification result obtained by clustering operation, for the construction process of classification index, the grouped feature vectors are structured and arranged using an index construction tool to determine the hierarchical distribution of classification index; Through the hierarchical distribution of the classification index, the index structure is optimized and adjusted using a retrieval index generation tool to obtain a retrieval index framework with high query capability; For the retrieval index framework, the feature vectors in the second data set are mapped in batches combined with the data processing module, and if data deviation is detected during the mapping process, the mapping result that meets the standard is corrected through a calibration tool; According to the mapping result that meets the standard, for the construction process of the retrieval library, the optimized index structure and the feature vectors are associated and stored using a storage management tool to obtain the complete retrieval library content; Through the complete retrieval library content, the matching degree of the classification index and the retrieval index is detected, and if the matching degree is lower than the preset threshold, the index structure is locally optimized through an adjustment tool to determine the final retrieval library architecture; According to the final retrieval library architecture, the query response capability of the retrieval library is tested using a verification tool to obtain the verified retrieval library data.

[0013] Further, according to the user input query condition, the retrieval index library is matched using a similarity algorithm, and if the relevance of the matching result is lower than the preset threshold, the semantic query range is expanded to re-match the steps comprising: For the user input query condition, the input content is structured and processed using an analysis tool to obtain the decomposed query elements; According to the decomposed query elements, a cosine similarity algorithm is used to perform a preliminary matching operation on the index structure in the retrieval index library to obtain an initial matching result; For the initial matching result, the degree of fit between the matching result and the query condition is calculated through a relevance evaluation module to determine whether the degree of fit reaches the preset threshold; If the degree of fit is lower than the preset threshold, the query condition is adjusted in range through a semantic expansion tool to obtain the expanded semantic content; According to the expanded semantic content, the retrieval index library is re-matched to determine the updated matching result; For the updated matching result, the integrity of the matching result is detected by the verification module, and if data is missing, the related information is extracted from the index structure by the supplement tool to obtain complete matching data. According to the complete matching data, the matching result is associated with the query condition input by the user and saved by using the storage management module to determine the final query record.

[0014] The application relates to a PDF intelligent retrieval and generation system based on RAG, which is used for realizing the above-mentioned PDF intelligent retrieval and generation method based on RAG. The forming module is used for obtaining input document data, analyzing the document data by using a pre-established classification model, extracting text content and image content to form a first data set; The acquisition module is used for extracting features of the image content in the first data set by using a deep learning model, and simultaneously performing semantic analysis on the text content in the first data set by applying a natural language processing technology to obtain a multi-modal feature set; The first generation module is used for generating a second data set by applying an information integration algorithm to unified coding processing according to the multi-modal feature set, and if the integrity of the fusion feature vector in the second data set is lower than a preset threshold, the missing information is filled by supplementing context semantic analysis; The second generation module is used for clustering the fusion feature vector in the second data set by using a preset index construction mechanism to generate a retrieval index library containing a classification index structure; The matching module is used for matching the retrieval index library by using a similarity algorithm according to the query condition input by the user, and if the relevance of the matching result is lower than a preset threshold, the semantic query range is expanded for re-matching.

[0015] Further, the forming module comprises: The forming unit is used for obtaining original content from the input document data, and performing preliminary analysis on the document by using a pre-established classification tool to extract text content and image content to form an initial set; The first acquisition unit is used for judging the integrity of the initial set by using a content recognition tool, and if the text content is missing, complete data is obtained by re-acquiring the complete data by using a document scanning tool to obtain a first text set; The second acquisition unit is used for processing the image content by using an image enhancement tool if the resolution of the image content is lower than a preset threshold to obtain a first image set; The third acquisition unit is used for performing word segmentation processing on the first text set by using a text segmentation tool to obtain segmented text units, and determining the business category of the text units by classifying the text units by using a semantic matching tool to obtain a second text set. The fourth acquisition unit is configured to, for the first image set, split the image set into a plurality of image units by using an image segmentation tool, acquire visual features of the image units by using a feature extraction tool, and if the feature values are lower than a preset threshold, adjust the image units by using an image optimization tool to obtain a second image set; The fifth acquisition unit is configured to, according to the second text set and the second image set, perform correlation mapping by using a data fusion tool to form a unified business data set, detect consistency of the data set by using a content verification tool, determine whether the data set meets a classification target, and obtain a final first data set.

[0016] Further, the acquisition module comprises: The sixth acquisition unit is configured to, for image content in the first data set, split the image content into a plurality of visual element units by using an image segmentation tool, process the visual element units by using a feature extraction tool to acquire corresponding image features, and if the values of the image features are lower than a preset threshold, adjust the image features by using an image enhancement tool to obtain an adjusted image feature set. The first determination unit is configured to, according to the adjusted image feature set, compare the image features with a preset classification standard by using a feature mapping tool, if the image features do not meet the standard, perform secondary processing on the image features by using a feature optimization tool to determine an image feature subset that meets the standard. The seventh acquisition unit is configured to, for text content in the first data set, split the text content into a plurality of language units by using a text segmentation tool, process the language units by using a semantic analysis tool to acquire corresponding text semantic features, and obtain a text semantic set. The eighth acquisition unit is configured to perform multi-modal fusion of the image feature subset and the text semantic set by using a data integration tool, perform business matching on the fused data by using a content classification tool to determine whether the data meets a preset target, and obtain a final multi-modal feature set.

[0017] Further, the first generation module comprises: The ninth acquisition unit is configured to, according to the multi-modal feature set, uniformly process feature encoding of the multi-modal feature set by using an information integration tool to generate a preliminary fused data set, detect encoding consistency of the preliminary fused data set, if the detected consistency is lower than a preset threshold, adjust the preliminary fused data set by using a data calibration tool to obtain a fused data set with consistent encoding. The second determination unit is configured to, for the fused data set with consistent encoding, evaluate vector integrity of the fused data set by using a feature detection tool, if it is found that the integrity of the fused data set is lower than a preset threshold, extract supplementary information by using a context analysis tool to determine a fused data set with improved integrity. The tenth acquisition unit is configured for filling the missing part in the integrity-improved fusion data set by using an information filling tool, and filling the content by combining a semantic supplement tool to obtain a filled fusion data set. The eleventh acquisition unit is configured for classifying and comparing the filled fusion data set by using a service matching tool, judging the fusion data set according to a pre-established rule, and obtaining a final data set meeting a service target.

[0018] The present application has the following beneficial effects: The present application provides a PDF intelligent retrieval and generation method and system based on RAG. The input document is parsed by a pre-established classification model to extract text and image content to form an initial data set. Deep learning and natural language processing techniques are used to extract features and analyze semantics of images and text respectively to obtain a multi-modal feature set. An information integration algorithm is then applied for unified coding to generate a fusion feature vector, and context semantics are supplemented as necessary to fill in missing information. Finally, a retrieval library containing a classification index is constructed by clustering processing, and similarity matching is performed according to user query conditions, and the semantic query range is expanded as necessary. The present application realizes intelligent analysis, feature extraction, information fusion and efficient retrieval of multi-modal documents, and improves the accuracy and comprehensiveness of document retrieval. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The present application provides a PDF intelligent retrieval and generation method and system based on RAG. The input document is parsed by a pre-established classification model to extract text and image content to form an initial data set. Deep learning and natural language processing techniques are used to extract features and analyze semantics of images and text respectively to obtain a multi-modal feature set. An information integration algorithm is then applied for unified coding to generate a fusion feature vector, and context semantics are supplemented as necessary to fill in missing information. Finally, a retrieval library containing a classification index is constructed by clustering processing, and similarity matching is performed according to user query conditions, and the semantic query range is expanded as necessary. The present application realizes intelligent analysis, feature extraction, information fusion and efficient retrieval of multi-modal documents, and improves the accuracy and comprehensiveness of document retrieval. DETAILED DESCRIPTION

[0020] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the drawings in the specification and specific embodiments.

[0021] As shown in the drawings, Figure 1 The present application provides a PDF intelligent retrieval and generation method and system based on RAG. The input document is parsed by a pre-established classification model to extract text and image content to form an initial data set. Deep learning and natural language processing techniques are used to extract features and analyze semantics of images and text respectively to obtain a multi-modal feature set. An information integration algorithm is then applied for unified coding to generate a fusion feature vector, and context semantics are supplemented as necessary to fill in missing information. Finally, a retrieval library containing a classification index is constructed by clustering processing, and similarity matching is performed according to user query conditions, and the semantic query range is expanded as necessary. The present application realizes intelligent analysis, feature extraction, information fusion and efficient retrieval of multi-modal documents, and improves the accuracy and comprehensiveness of document retrieval. Step S100, acquire the input document data, parse the document data by using a pre-established classification model, and extract text content and image content to form a first data set.

[0022] Document data refers to a digital process of transmitting information such as text, charts and symbols contained in a document to a computer system for processing through an input device (such as a keyboard or a scanner) or a file system. Its essence is to convert structured or unstructured content in external storage media (such as electronic documents and paper files) into a data stream recognizable by a computer. The document data can be a PDF document.

[0023] Text content is an information carrier based on character sequences, containing a collection of human-readable symbols such as letters, numbers, and symbols, which are converted into binary data that can be processed by computers through encoding (such as ASCII, UTF-8, etc.).

[0024] Image content is a visual information carrier composed of pixel matrices as basic units, expressing features such as shape, texture, and color through color channels (such as RGB, CMYK) and spatial distribution, and relying on specific encoding formats (such as JPEG, PNG) for digital storage and transmission.

[0025] The first data set is an initial data set used for data processing or analysis in a specific scenario, which is essentially a collection of structured or semi-structured data elements organized according to established rules.

[0026] Step S200, using a deep learning model to extract features from image content in the first data set, and applying natural language processing technology to text content in the first data set for semantic analysis, obtaining a multi-modal feature set.

[0027] Deep learning model is a multi-layer nonlinear computing model based on artificial neural network (ANN), which realizes complex pattern recognition and prediction through hierarchical feature extraction and nonlinear transformation of large amounts of data.

[0028] Natural language processing (NLP) is an interdisciplinary field of computer science and linguistics, aiming to enable computers to understand, generate, and manipulate human natural language through algorithms and models. Its core is to analyze the grammatical structure, semantic relationship, and pragmatic scene of text, convert language into machine computable form (such as vector representation or probability distribution), and complete classification, translation, generation and other tasks. Typical technologies include morphological analysis, syntactic parsing, semantic understanding, and dialogue systems.

[0029] Feature extraction is a process of identifying, transforming, and extracting key information related to the target task from raw data, aiming to convert high-dimensional, redundant, or unstructured data into low-dimensional, high-discriminative feature representation to improve the efficiency and performance of machine learning models. Its essence is to reveal the essential structure of data through mathematical or statistical methods, remove noise and irrelevant interference, and form numerical features that can be directly processed by algorithms. In this embodiment, feature extraction refers to extracting features of image content.

[0030] Semantic Analysis is the process of interpreting the underlying meaning of language expressions (text or code) to understand their logic, intent, or sentiment. In this embodiment, semantic analysis refers to the interpretation of the content of the text.

[0031] Multimodal Feature Set refers to the combination of feature vectors extracted from different modalities (such as images, text, speech, etc.), aiming to form a more comprehensive and robust data representation by integrating complementary information from multiple sources. Its core goal is to preserve the unique information of each modality and eliminate redundancy, providing high-discriminative input features for subsequent tasks (such as classification, detection, generation).

[0032] Step S300, according to the multimodal feature set, apply information integration algorithm for unified encoding processing to generate the second data set, if the integrity of the fusion feature vector in the second data set is detected to be lower than the preset threshold, supplement the context semantic analysis to fill in the missing information.

[0033] Information integration algorithm is a set of algorithms that extract, correlate, and fuse multi-source heterogeneous data (such as text, images, speech, etc.) to generate unified and semantically consistent high-level representations.

[0034] Encoding Processing is the process of converting raw data (such as text, images, speech, etc.) into structured representations that can be recognized, stored, or transmitted by computers. Its purpose is to map complex, heterogeneous input data into standardized numerical or symbolic sequences suitable for specific tasks (such as machine learning modeling, data transmission) through specific rules or algorithms.

[0035] Second data set refers to the supplementary data set used in conjunction with the main data set in multimodal research, its core role is to introduce additional modalities or scene information to enhance the diversity and joint modeling ability of the main data set.

[0036] Fusion feature vector is a unified and high semantic correlation vector representation generated by integrating multi-source, multi-modal or heterogeneous data feature representation. Its core goal is to combine the complementary information of different data sources (such as text, images, sensor data, etc.), eliminate redundancy and noise, and form a more comprehensive and robust representation of the target object, thereby improving the performance of machine learning models in downstream tasks (such as classification, retrieval, generation).

[0037] Contextual Semantic Analysis is a technique that deconstructs and reasons about the deep meaning of text or data by combining language context, background knowledge, and multi-modal information. Its core is to break through the limitations of isolated semantics and achieve dynamic and scenario-based understanding.

[0038] Filling in missing information refers to repairing incomplete modal information in multi-modal data sets due to collection errors, transmission loss, or human factors through technical means, aiming to restore data integrity to improve the reliability of downstream tasks such as classification and retrieval.

[0039] Step S400: Adopting a preset index construction mechanism to perform clustering processing on the fusion feature vectors in the second data set, generating a retrieval index library containing a classification index structure.

[0040] The index construction mechanism is a systematic method for converting raw data (such as text, images, multi-modal feature vectors) into efficient retrieval structures by establishing a mapping relationship between data identifiers and storage locations to speed up query response speed and reduce computational complexity. Its core goal is to balance storage cost and query efficiency, supporting real-time or near-real-time access to large-scale data sets.

[0041] Clustering processing is a process of dividing unlabeled data into similar subsets (clusters) through unsupervised learning techniques, aiming to maximize intra-cluster similarity and minimize inter-cluster similarity to reveal data distribution patterns or potential class features. This process does not rely on prior knowledge and is completely based on the attributes of the data itself for grouping.

[0042] The classification index structure is a technology that organizes and indexes data through a pre-defined classification system, aiming to achieve efficient directional retrieval and range queries based on data classification attributes such as topics, labels, and hierarchical relationships. Its core is to optimize data distribution under category constraints, reducing retrieval space complexity and improving query response speed.

[0043] The retrieval index library is a structured index collection and storage system designed for efficient data query, establishing a mapping relationship between data identifiers and physical storage locations through pre-constructed multi-dimensional indexes (such as inverted indexes, vector indexes, and classification indexes) to support fast retrieval, sorting, and recall of multi-modal data such as text, images, and videos. Its core goal is to trade space for time, reducing the computational complexity of online queries through offline index construction while ensuring throughput and low-latency response in high-concurrency scenarios.

[0044] Step S500, according to the query condition input by the user, the similarity algorithm is used to match the retrieval index library, and if the relevance of the matching result is lower than the preset threshold, the semantic query range is expanded for re-matching.

[0045] Similarity Algorithm is a mathematical model or calculation method for quantifying the consistency or correlation between two data objects. By defining the distance, angle or probability distribution relationship between objects, it evaluates their proximity or semantic relevance in the feature space. Its core goal is to serve tasks such as clustering, retrieval, recommendation, etc., and to filter out the most relevant candidate results through similarity ranking.

[0046] Expanding the semantic query range is a strategy to enhance the semantic expression ability of the query, break through the limitation of literal matching, and improve the understanding breadth of the user's intention and the recall ability of the associated content of the retrieval system. Its core goal is to expand the candidate result set from the perspective of semantic relevance rather than keyword matching, covering implicitly related but differently expressed content.

[0047] Further, the RAG-based PDF intelligent retrieval and generation method provided in the embodiment includes the following steps: Step S110, the original content is obtained from the input document data, and the pre-established classification tool is used to preliminarily analyze the document to extract the text content and image content to form an initial set.

[0048] When processing a financial report document within an enterprise, the pre-established classification tool can be used to preliminarily analyze the document to extract the textual descriptions such as financial data and explanatory text into an initial text set, and extract the image content such as charts and signatures into an initial image set.

[0049] Step S120, for the initial set, the content recognition tool is used to judge its integrity, if the text content is missing, the document scanning tool is used to obtain the complete data again, and the first text set is obtained.

[0050] For the text content, if some data is missing, such as the table data on a page is not recognized, the document scanning tool is used to rescan the page to ensure that complete data is obtained, and the first text set is formed.

[0051] Step S130, if the resolution of the image content is lower than the preset threshold, the image enhancement tool is used to process, and the first image set is obtained.

[0052] For the image content, if the resolution of a chart is found to be lower than the preset threshold, such as 300 dpi, the image enhancement tool is used to adjust the clarity to a level that meets the requirements, and the first image set is formed.

[0053] Step S140, for the first text set, use the text segmentation tool for segmentation processing, get the segmented text unit, and determine its business category through semantic matching tool according to the text unit, get the second text set.

[0054] For the first text set, use the text segmentation tool to split the text into independent word units, for example, split "annual financial report" into "annual", "financial", "report" units. Then analyze the meaning of these units through the semantic matching tool, label them as "financial" business category, and finally form the second text set. This way can effectively improve the accuracy of text classification, lay the foundation for subsequent data analysis.

[0055] Step S150, for the first image set, use the image segmentation tool to split into multiple image units, get the visual features of the image unit through the feature extraction tool, if the feature value is lower than the preset threshold, adjust through the image optimization tool, get the second image set.

[0056] For the first image set, use the image segmentation tool to split a complex financial chart into multiple small image units, such as pie chart part and column chart part, and then analyze the visual features of each unit through the feature extraction tool. If it is found that the feature value of a unit is lower than the preset threshold, such as insufficient color contrast to clearly show the data difference, adjust the brightness and contrast through the image optimization tool to ensure that the image information is intuitive and readable, form the second image set. This step significantly improves the usability of image data.

[0057] Step S160, according to the second text set and the second image set, use the data fusion tool for association mapping, form a unified business data set, detect the consistency of the data set through the content verification tool, determine whether it meets the classification target, get the final first data set.

[0058] When processing the second text set and the second image set, use the data fusion tool for association mapping, for example, match "first quarter revenue growth 10%" in the text with the corresponding column chart in the image to form a unified business data set. Then detect the consistency through the content verification tool to ensure that the text description and image data are consistent, such as verifying whether the growth rate is consistent. If found to be inconsistent, it can be traced back to the original data for correction, and finally form the first data set. This association mapping and consistency check effectively guarantee the integrity and reliability of the data, and provide a solid support for subsequent business decision-making.

[0059] In practical applications, assuming that a financial report contains multiple pages of data and complex charts, through the above process, it can be ensured that the text and image content are accurately extracted and classified, the missing data is supplemented, the low-quality images are optimized, and the final data set can fully reflect the financial situation. This method not only improves the efficiency of data processing, but also reduces the cost of manual correction, and has significant practical value.

[0060] Further, the RAG-based PDF intelligent retrieval and generation method provided by the embodiment comprises the following steps: In step S210, the image content in the first data set is divided into multiple visual element units by using an image segmentation tool, and the visual element units are processed by a feature extraction tool to obtain corresponding image features. If the value of the image features is lower than a preset threshold, the image features are adjusted by an image enhancement tool to obtain an adjusted image feature set.

[0061] When processing a financial report data of an enterprise, the processing process of the first data set can be analyzed in detail from multiple angles, and the implementation method is discussed around the processing of image content and text content.

[0062] For the processing of image content, the complex financial chart is divided into multiple visual element units by using an image segmentation tool, such as dividing a chart containing multiple data into a line chart part and a table area.

[0063] In principle, the image segmentation tool divides different areas by recognizing the boundaries and color differences in the chart. Assuming that a chart contains multiple data display areas, the feature extraction tool of each unit will analyze its line thickness and color distribution and other characteristics to obtain specific image feature values. If the feature value of a certain unit is lower than the preset threshold, such as less than 100 pixel units of clarity, the resolution and contrast of the image are adjusted by the image enhancement tool to form an adjusted image feature set.

[0064] In step S220, according to the adjusted image feature set, a feature mapping tool is used to compare the image features with the preset classification standard. If it does not meet the standard, a feature optimization tool is used for secondary processing to determine a subset of image features that meet the standard.

[0065] Subsequently, the features are compared with the preset classification standard by using the feature mapping tool. If it does not meet the requirements, such as insufficient color contrast, a feature optimization tool is used for secondary processing to ensure that the final subset of image features meets the business display requirements.

[0066] Step S230, for the text content in the first data set, the text segmentation tool is used to split it into multiple language units, the language units are processed by the semantic analysis tool to obtain corresponding text semantic features, and a text semantic set is obtained.

[0067] For the processing of text content, the text segmentation tool can split long sentences in the financial report into multiple language units, such as splitting “quarterly revenue growth data” into “quarterly”, “revenue”, “growth”, “data” and other units. The semantic analysis tool analyzes the meaning of these units based on the context relationship, such as identifying the positive association of “growth” with financial data, and then forming a text semantic set. This way can help to sort out the key information points in the text and provide a basis for subsequent classification.

[0068] Step S240, through the data integration tool, the image feature subset and the text semantic set are multi-modal fused, the content classification tool is used to match the fused data with the business, it is judged whether it meets the preset target, and a final multi-modal feature set is obtained.

[0069] In the multi-modal fusion link, the data integration tool is used to associate the image feature subset and the text semantic set, such as matching “5% revenue growth” mentioned in the text with the upward trend of the corresponding line chart in the image.

[0070] The content classification tool judges whether the fused data meets the financial analysis requirements according to the preset target, such as whether it can clearly reflect the quarterly trend, and obtains a final multi-modal feature set. This fusion method can ensure the unity of data in different forms. For example, from the business point of view, the financial report often contains complex tables and descriptive text, if only relying on single modal data, some information may be ignored.

[0071] Through the above multi-modal fusion, data features are comprehensively captured from two dimensions of image and text, such as the image directly showing the trend and the text providing specific numerical description, and the combination of the two can more completely present the financial situation. This way has a significant advantage in data presentation and business understanding.

[0072] Further, the RAG-based PDF intelligent retrieval and generation method provided in the embodiment comprises the following steps: Step S310, according to the multi-modal feature set, the information integration tool is used to uniformly process the feature coding, generate a preliminary fusion data set, detect the coding consistency of the preliminary fusion data set, if the consistency is lower than the preset threshold, adjust through the data calibration tool, and obtain a coding consistent fusion data set.

[0073] In processing enterprise internal financial report data, the integration and optimization of multi-modal feature sets are analyzed in detail. For each technical topic, detailed implementation methods and scenario examples are provided to ensure that the content is closely related to the field of financial data analysis.

[0074] The uniform processing of feature encoding by the information integration tool can be understood as the process of converting image and text features into a uniform format.

[0075] In principle, image features can be stored in vector form, while text features can be semantic labels. The integration tool needs to map both to the same dimensional space. Suppose we are processing a quarterly financial report, the image feature vector dimension is 128, and the text feature dimension is 64. The integration tool will unify them into a 100-dimensional vector through dimension reduction or dimension elevation operation to form a preliminary fusion dataset, which will help subsequent consistency detection.

[0076] Step S320, for the encoding consistent fusion dataset, the feature detection tool is used to evaluate the vector integrity. If the integrity of the fusion dataset is found to be lower than the preset threshold, the context analysis tool is used to extract supplementary information to determine the fusion dataset with improved integrity.

[0077] For the application of encoding consistency detection and data calibration tools, the principle is to ensure the coordination of features in the dataset. Suppose the preset consistency threshold is 85%, and the detection finds that the consistency of a dataset is only 70%, which may be due to the large deviation of image features and text features in some dimensions. The data calibration tool will adjust the abnormal dimension values, such as balancing the deviation through weighted average method, finally making the consistency improve to 88%, and obtaining the encoding consistent fusion dataset.

[0078] For the feature detection tool to evaluate the vector integrity, it can be understood as checking whether the fusion dataset covers the necessary information. If the integrity threshold is 90% and the detection result is 80%, it may be that some image features are missing. The context analysis tool will extract supplementary information from related text descriptions, such as inferring image trend features from the description of "income growth 10%", and forming a new fusion dataset with improved integrity of 92%.

[0079] Step S330, through the information filling tool, for the missing part in the fusion dataset with improved integrity, the content is filled in combination with the semantic supplement tool to obtain the filled fusion dataset.

[0080] For the application of the information filling tool and the semantic supplement tool, the principle is to fill in the missing content in the data set. Assuming that part of the text description in the data set after integrity improvement is still incomplete, for example, the "cost change" related information is missing, the semantic supplement tool will infer the possible content in combination with the context, such as inferring the cost downward trend from "profit increase", and filling in to form a more comprehensive data set.

[0081] Step S340, using a business matching tool, classifying and comparing the filled fusion data set, judging the fusion data set according to the pre-established rules, and obtaining the final data set meeting the business target.

[0082] For the classification comparison of the business matching tool, the principle is to filter the data set according to the financial analysis target. Assuming that the rule requires the data set to reflect the quarterly trend, the tool will compare whether the fusion data contains key indicators such as income and cost, and if some part of the data does not meet the requirements, for example, the lack of expense details, it will be excluded or marked to ensure that the final data set meets the business target. This way can effectively improve the business relevance of the data.

[0083] Further, the RAG-based PDF intelligent retrieval and generation method provided by the embodiment, step S400 includes: Step S410, for the fusion features in the second data set, a preset mechanism is used to preliminarily group the feature vectors, and a clustering operation is performed to obtain a preliminary classification result.

[0084] In the scenario of processing enterprise internal financial data analysis, the preliminary grouping processing of the fusion features in the second data set can be understood as preliminarily clustering the complex feature vectors according to similarity.

[0085] In principle, the clustering operation will group similar financial indicators into a category based on the distribution characteristics of the feature vectors, such as income-related features and cost-related features being aggregated into different groups. Assuming that an annual financial report data set is processed, the income-related feature vectors may be concentrated in a certain value interval, while the cost-related feature vectors are distributed in another interval. Through the clustering operation, two preliminary classification results can be obtained, which lay the foundation for subsequent index construction.

[0086] Step S420, according to the preliminary classification result obtained by the clustering operation, for the construction process of the classification index, an index construction tool is applied to structure the grouped feature vectors, and the hierarchical distribution of the classification index is determined.

[0087] For the construction process of the classification index, the index construction tool is applied to the structured arrangement of the grouped feature vectors, aiming to form a clear hierarchical distribution. In principle, the hierarchical distribution is divided according to the priority or importance of the financial data, such as placing core indicators at the top layer and secondary indicators at the lower layer. Assuming that the core indicators are total revenue and net profit, and the secondary indicators are various cost details, the index construction tool will place the total revenue at the first level and the cost details at the second level, forming a tree structure, which facilitates subsequent rapid positioning of data.

[0088] Step S430, through the hierarchical distribution of the classification index, the retrieval index generation tool is used to optimize and adjust the index structure, and a retrieval index framework with efficient query capability is obtained.

[0089] The retrieval index framework is obtained by the following formula: (1) In formula (1), represents the optimized index structure configuration, represents the set of all possible index structure schemes, represents the number of query samples, represents the query request, represents the corresponding index structure response, the function represents the query distance metric, the function represents the structure complexity metric, and are balance parameters. Formula (1) obtains the optimal retrieval index framework by minimizing the objective function.

[0090] For the optimization and adjustment of the index structure by the retrieval index generation tool, the purpose is to build a framework with efficient query capability. In principle, optimization and adjustment will reduce the redundant paths between index levels and improve query speed. Assuming that the original index structure requires 3-layer jump to query a cost detail, the optimized index structure can complete the query in only 2 layers, significantly shortening the response time.

[0091] Step S440, for the retrieval index framework, the data processing module is combined to batch map the feature vectors in the second data set, and if data deviation is detected during the mapping process, the calibration tool is used for correction to determine the mapping result that meets the standard.

[0092] The linear transformation process of the batch mapping of the feature vectors by the processing module is described by the following formula: (2) In formula (2), represents the mapping function, represents the a feature vector, a mapping weight matrix, a bias vector.

[0093] The degree of data deviation detected in the mapping process is quantified by the following formula: (3) In formula (3), denotes the data deviation detected in the mapping process, denotes the total number of mapping results, denotes the mapping output result, denotes the reference standard value.

[0094] The calculation process of the calibration tool to correct the deviation data is described by the following formula: (4) In formula (4), denotes the calibrated mapping result, denotes the original mapping result, denotes the calibration intensity coefficient, denotes the calibration matrix, denotes the target standard value.

[0095] For batch mapping of feature vectors by the data processing module, if data deviation is detected, calibration is performed to ensure the accuracy of the mapping results. Assuming that a certain income feature vector deviates from the normal range during the mapping process, the calibration tool will correct it according to historical data trends to ensure that the mapping results conform to the financial analysis logic. For the application of the storage management tool in the construction process of the search library, the principle is to store the optimized index structure and feature vectors in association to form a complete search library.

[0096] Step S450, according to the mapping results that meet the standards, for the construction process of the search library, the storage management tool is used to store the optimized index structure and feature vectors in association to obtain complete search library content.

[0097] The complete search library content is obtained by the following formula: (5) In formula (5), denotes the complete search library content, denotes the association storage function of the storage management tool, denotes the optimized index structure, denotes the feature vector set, denotes the index element, denotes the 1 eigenvector This indicates the total amount of data stored in the retrieval database.

[0098] Suppose that the storage management tool binds revenue metrics to corresponding vectors for storage, ensuring that relevant data can be quickly matched during queries.

[0099] Step S460: Using the complete search database content, the matching degree between the category index and the search index is detected. If the matching degree is found to be lower than the preset threshold, the index structure is locally optimized by the adjustment tool to determine the final search database architecture.

[0100] The matching degree between the category index and the retrieval index is: (6) In formula (6), Represents a category index With search index The degree of matching between them Indicates the total number of index feature dimensions. Indicates the first The weight coefficients of each feature Represents a category index In the Values ​​on each feature Indicates the search index In the Values ​​on each feature The function represents the similarity calculation function.

[0101] The preset threshold for matching degree is: (7) In formula (7), The preset threshold representing the matching degree. This represents the mean of historical matching scores. This represents the adjustment coefficient. The standard deviation of historical matching degree This indicates the total number of indexes in the search database. Indicates the number of valid matches in the index.

[0102] The optimal structure for local optimization of the index structure is: (8) In formula (8), Indicates the index structure The optimal structure after local optimization. This represents the candidate optimized structure. This indicates the number of index nodes that need optimization. Indicates the first optimized weights of the nodes, a function representing a structural difference measure, a structure of the original nodes, a structure of the optimized nodes, a structure of the optimized nodes, a structure of the optimized nodes, a regularization parameter, a structural complexity penalty term.

[0103] For the detection of the matching degree of the classification index and the retrieval index, if it is lower than the preset threshold, local optimization is performed. The principle is to improve the coordination between indexes. Assuming that the preset threshold is 80%, the actual detection is only 75%, and the adjustment tool optimizes the abnormal hierarchical structure, so that the matching degree is improved to more than 82%.

[0104] Step S470, according to the final search library architecture, the verification tool is used to test the query response ability of the search library, and the verified search library data is obtained.

[0105] For the test of the query response ability of the search library, the application of the verification tool aims to ensure the practicability of the search library. Assuming that the query response time of a certain financial index is 0.5 seconds, which meets the expected standard, the verification is passed. This way helps to ensure the efficiency of financial data analysis.

[0106] Further, the RAG-based PDF intelligent retrieval and generation method provided by the embodiment comprises the following steps: Step S510, for the query condition input by the user, the input content is structured by the parsing tool, and the decomposed query elements are obtained.

[0107] In the scenario of enterprise internal financial data analysis, the parsing tool plays a crucial role when processing the query condition input by the user. In principle, the parsing tool will decompose the complex query statement input by the user into multiple identifiable elements for subsequent matching operations. For example, assuming that the user input query is about the change of net profit in a certain year, the parsing tool will disassemble it into specific elements such as time range and financial indicator category, laying the foundation for subsequent retrieval.

[0108] Step S520, according to the decomposed query elements, the cosine similarity algorithm is used to perform preliminary matching operation on the index structure in the retrieval index library, and the initial matching result is obtained.

[0109] For the application of cosine similarity algorithm, its core is to measure the similarity between query elements and index structure. Conceptually, this algorithm judges the relevance through the angle difference of vectors, and the smaller the angle is, the higher the similarity is. Assuming that the user query element is total income data, the algorithm will compare the element with the relevant vectors stored in the retrieval index library, and preliminarily screen out the closest index node to form the initial matching result.

[0110] Specifically, the cosine similarity is calculated by vector dot product and module length, and the query vector is obtained. The cosine similarity is used to measure the similarity between the query vector and the index vector , and the cosine similarity is used to measure the consistency of the direction of the two vectors (the smaller the angle is, the higher the similarity is), and the formula is as follows: (9) In formula (9), denotes the cosine value of the angle between the query vector and the index vector , that is, the cosine similarity. denotes the dot product (inner product) of the query vector and the index vector , and denote the module length of the query vector and the index vector , respectively (also known as norm), that is, the "length" of the vector.

[0111] The calculation method of the query vector and the index vector is to multiply the corresponding components of the two vectors and then sum, that is: (10) In formula (10), is the n-dimensional component of the query vector, is the n-dimensional component of the index vector, and is the vector dimension.

[0112] The module length of the vector is: (11) In formula (11), denotes the module length of the query vector, is the n-dimensional component of the query vector, and is the vector dimension.

[0113] vector The length of the vector is: (12) In formula (12), the length of the index vector is denoted as The length of the vector is The dimensional component of the index vector is The vector dimension is

[0114] Step S530, for the initial matching result, the degree of fit between the matching result and the query condition is calculated by the correlation evaluation module, and it is judged whether the degree of fit reaches the preset threshold.

[0115] The correlation score between the query condition and the matching result is calculated by the following formula: (13) In formula (13), the correlation score between the query condition and the th matching result is denoted as The number of feature dimensions of the query condition is denoted as The weight coefficient of the th feature is denoted as The th feature value in the query condition is denoted as The th feature value in the th matching result is denoted as The function represents the similarity calculation function between two feature values.

[0116] Whether the degree of fit reaches the preset threshold is judged by the following formula: (14) In formula (14), the final judgment decision on the th matching result is denoted as The degree of fit value of the th matching result is denoted as The preset degree of fit threshold is denoted as When the degree of fit is greater than or equal to the threshold, it is judged as passing and returns 1, otherwise it is judged as not passing and returns 0.

[0117] ​The introduction of the relevance evaluation module is to further verify the accuracy of the matching results. In principle, the relevance evaluation module will score the initial results in multiple dimensions, such as content coverage and semantic consistency. Assuming that the preset fit threshold is 85%, and the initial matching result score is only 78%, further adjustment is needed to improve the matching quality.

[0118] Step S540, if the fit is lower than the preset threshold, adjust the query condition through semantic expansion tools to obtain the expanded semantic content.

[0119] The expanded semantic content is obtained by the following formula: (15) In formula (15), represents the query vector after semantic expansion, represents the original query vector, represents the expansion strength control parameter, represents the number of expanded semantic words, represents the weight coefficient of the th expanded word, represents the vector representation of the th expanded semantic word.

[0120] The purpose of using the semantic expansion tool is to broaden the query range when the fit is insufficient. Specifically, if the initial matching is not up to standard, the semantic expansion tool will expand the query condition to include near-synonymous or related indicators based on the semantic association in the financial field.

[0121] Step S550, according to the expanded semantic content, re-match the search index library to determine the updated matching result.

[0122] The updated matching result is obtained by the following formula: (16) In formula (16), represents the final matching result after updating, represents the original matching result set, represents the new matching result set based on expanded semantics, represents the retention weight factor of the original result, with a value range of 0 to 1.

[0123] Assuming that the user queries net profit, after expanding to include total revenue and cost-related concepts, re-matching is performed to obtain more comprehensive results.

[0124] Step S560, for the updated matching result, the integrity of the matching result is detected by the verification module, if the data is missing, the related information is extracted from the index structure by the supplement tool to obtain the complete matching data.

[0125] The integrity detection of the updated matching result is obtained by the following formula: (17) In formula (17), represents the integrity detection value of the matching result, represents the current matching result set, represents the reference complete matching set, represents the intersection size of the current matching and the reference matching, represents the total size of the reference matching set, represents the number of non-empty matching items, represents the expected total number of matching items, represents the integrity weight coefficient.

[0126] For the integrity detection of the updated matching result, the verification module will check whether the data is missing. In principle, if it is found that part of the financial indicator data is not covered, the supplement tool will extract the related information from the index structure. Assuming that the matching result lacks certain cost detail data, the supplement tool will be called from the secondary index level to ensure that the final result is complete and intact.

[0127] Step S570, according to the complete matching data, the storage management module is used to associate and save the matching result with the query condition input by the user, and the final query record is determined.

[0128] The association strength between the query condition and the matching result is obtained by the following formula: (18) In formula (18), represents the query condition and the matching result between the association degree score, represents the weight coefficient of the matching attribute, represents the similarity function of the attribute of the query condition and the attribute of the matching result, represents the total number of attributes participating in matching.

[0129] The final query record is obtained by the following formula: (19) In formula (19), denotes the final query record set, denotes the query condition input by the user, denotes the matching result of the user, denotes the query timestamp, denotes the relevance threshold, denotes the matching result index set related to the query condition , and the difference denotes the total number of query conditions. Formula (19) defines how the query records meeting the conditions constitute the final result set.

[0130] The application of the storage management module is to associate and save the final matching result with the user query condition for subsequent reuse or traceability. Assuming that the user queries the quarterly financial data, the system will bind and store the matching index data with the query record to form a traceable query log. This method helps to improve the standardization of data management.

[0131] Through the above multi-aspect processing, from query analysis to result saving, each link closely revolves around the demand of financial data analysis, ensuring the rigor of the query process and the reliability of the result, and also providing users with an efficient data acquisition experience.

[0132] The application relates to a PDF intelligent retrieval and generation system based on RAG, which is used for realizing the above-mentioned PDF intelligent retrieval and generation method based on RAG. The PDF intelligent retrieval and generation system based on RAG comprises a forming module, an acquisition module, a first generation module, a second generation module and a matching module. The forming module is used for acquiring input document data, adopting a pre-established classification model to analyze the document data, extracting text content and image content to form a first data set. The acquisition module is used for adopting a deep learning model to extract features of the image content in the first data set, and simultaneously applying natural language processing technology to the text content in the first data set to perform semantic analysis, so as to obtain a multi-modal feature set. The first generation module is used for applying an information integration algorithm to uniformly encode and process the multi-modal feature set to generate a second data set. If it is detected that the integrity of the fusion feature vector in the second data set is lower than a preset threshold, context semantic analysis is supplemented to fill in the missing information. The second generation module is used for adopting a preset index construction mechanism to cluster the fusion feature vector in the second data set to generate a retrieval index library containing a classification index structure. The matching module is used for adopting a similarity algorithm to match the retrieval index library according to the query condition input by a user. If the relevance of the matching result is lower than a preset threshold, the semantic query range is expanded to perform re-matching.

[0133] Further, the RAG-based PDF intelligent retrieval and generation system provided by the embodiment forms a module including a forming unit, a first obtaining unit, a second obtaining unit, a third obtaining unit, a third obtaining unit, a fourth obtaining unit, and a fifth obtaining unit. The forming unit is configured to obtain original content from input document data, perform preliminary analysis on the document by using a pre-established classification tool, and extract text content and image content to form an initial set. The first obtaining unit is configured to determine the integrity of the initial set by using a content recognition tool. If the text content is missing, complete data is reobtained by using a document scanning tool to obtain a first text set. The second obtaining unit is configured to process the image content by using an image enhancement tool if the resolution of the image content is lower than a preset threshold to obtain a first image set. The third obtaining unit is configured to perform word segmentation processing on the first text set by using a text segmentation tool to obtain segmented text units, determine the business category of the text units by using a semantic matching tool, and obtain a second text set. The fourth obtaining unit is configured to split the first image set into multiple image units by using an image segmentation tool, obtain visual features of the image units by using a feature extraction tool, and adjust the image units by using an image optimization tool if the feature values are lower than a preset threshold to obtain a second image set. The fifth obtaining unit is configured to associate and map the second text set and the second image set by using a data fusion tool to form a unified business data set, detect the consistency of the data set by using a content verification tool, determine whether the data set meets the classification target, and obtain a final first data set.

[0134] Preferably, the RAG-based PDF intelligent retrieval and generation system provided by the embodiment includes a sixth obtaining unit, a first determining unit, a seventh obtaining unit, and an eighth obtaining unit. The sixth obtaining unit is configured to split image content in the first data set into multiple visual element units by using an image segmentation tool, process the visual element units by using a feature extraction tool to obtain corresponding image features, and adjust the image features by using an image enhancement tool if the values of the image features are lower than a preset threshold to obtain an adjusted image feature set. The first determining unit is configured to compare the adjusted image feature set with a preset classification standard by using a feature mapping tool. If the comparison does not meet the standard, the image feature set is processed again by using a feature optimization tool to determine a subset of image features that meets the standard. The seventh obtaining unit is configured to split text content in the first data set into multiple language units by using a text segmentation tool, process the language units by using a semantic analysis tool to obtain corresponding text semantic features, and obtain a text semantic set. The eighth obtaining unit is configured to perform multi-modal fusion of the subset of image features and the text semantic set by using a data integration tool, perform business matching on the fused data by using a content classification tool, determine whether the data meets a preset target, and obtain a final multi-modal feature set.

[0135] Further, the RAG-based PDF intelligent retrieval and generation system provided by the embodiment further comprises a first generation module, which comprises a ninth acquisition unit, a second determination unit, a tenth acquisition unit and an eleventh acquisition unit. The ninth acquisition unit is configured to uniformly process feature codes by using an information integration tool according to the multi-modal feature set, generate a preliminary fusion data set, detect the coding consistency of the preliminary fusion data set, and if the consistency is found to be lower than a preset threshold, adjust the coding consistency by using a data calibration tool to obtain a fusion data set with consistent coding. The second determination unit is configured to evaluate vector integrity by using a feature detection tool for the fusion data set with consistent coding, and if the integrity of the fusion data set is found to be lower than a preset threshold, extract supplementary information by using a context analysis tool to determine a fusion data set with improved integrity. The tenth acquisition unit is configured to fill in the missing part of the fusion data set with improved integrity by using an information filling tool in combination with a semantic supplement tool to obtain a filled fusion data set. The eleventh acquisition unit is configured to classify and compare the filled fusion data set by using a business matching tool, judge the fusion data set according to a pre-established rule, and obtain a final data set meeting a business target.

[0136] Compared with the prior art, the RAG-based PDF intelligent retrieval and generation method and system provided by the embodiment analyzes the input document by using a pre-established classification model, extracts text and image content to form an initial data set, respectively extracts features and analyzes semantics of the image and the text by using deep learning and natural language processing technology to obtain a multi-modal feature set. Then, the information integration algorithm is applied to uniformly code to generate a fusion feature vector, and context semantics is supplemented as necessary to fill in missing information. Finally, clustering processing is used to construct a retrieval library containing a classification index, and similarity matching is performed according to the user query condition, and the semantic query range is expanded as necessary. The embodiment realizes intelligent analysis, feature extraction, information fusion and efficient retrieval of multi-modal documents, and improves the accuracy and comprehensiveness of document retrieval. The RAG-based PDF intelligent retrieval and generation method and system provided by the embodiment have the following beneficial effects: One, efficient processing of unstructured data, improving data usability The traditional PDF file contains unstructured data such as text and images, which is difficult to process and inefficient. In this embodiment, the pre-established classification model is used to analyze the document data, accurately extract the text content and image content, and form a first data set. This process can quickly disassemble complex PDF files into basic data units that can be processed, greatly reducing data processing time and improving data processing efficiency compared to manual processing or simple data extraction methods. At the same time, complete data extraction avoids information omission, ensuring that subsequent processing is based on a comprehensive data foundation, significantly improving the usability of PDF data and providing strong support for subsequent in-depth analysis and application. II. Multi-modal feature fusion, enhancing information representation ability A deep learning model is used to extract features from image content, and natural language processing techniques are used to analyze the semantic information of text content, resulting in a multi-modal feature set. This multi-modal processing method breaks down the limitations of single-modal information and can capture information in PDF files from multiple angles. The semantic information of the text and the visual features of the image complement each other, making the information representation more rich and comprehensive.

[0137] III. Information integration and missing information filling, ensuring data quality An information integration algorithm is applied to uniformly encode the multi-modal feature set to generate a second data set. In the case where the integrity of the fusion feature vector is lower than the preset threshold, context semantic analysis is used to fill in the missing information. This mechanism ensures the accuracy and integrity of the data during the integration process. In actual PDF files, some images or text may be blurry or incomplete, and through context semantic analysis, the information can be filled in to effectively repair data defects and avoid retrieval errors or inaccurate answers caused by missing information. This ensures the quality of the data used for retrieval and answer generation, improving the reliability of the system output results.

[0138] IV. Cluster construction index, speed up retrieval speed A preset index construction mechanism is used to cluster the fusion feature vectors in the second data set, generating a retrieval index library containing a classification index structure. This clustering index construction method classifies and stores massive PDF data according to features, so that when retrieving, it does not need to traverse all data, but can quickly locate relevant data areas according to the index. When the user inputs the query condition, the system can quickly narrow the search range based on the index, greatly improving the retrieval speed. V. Dynamic adjustment of retrieval strategy, improve retrieval accuracy According to the query condition input by the user, a similarity algorithm is used to match the retrieval index library, and if the relevance of the matching result is lower than a preset threshold, the semantic query range is extended for re-matching. This way of dynamically adjusting the retrieval strategy fully considers the diversity and ambiguity of the user query intention. When the initial matching result is not ideal, more potential relevant information can be mined by extending the semantic query range, and important content can be avoided from being missed due to the limitation of the query condition expression.

[0139] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they learn of the basic inventive concepts. Therefore, the appended claims are intended to encompass within their scope all possible variations and modifications of the preferred embodiments. It is apparent that those skilled in the art can, without departing from the spirit and scope of the application, make various changes and modifications of the application. Thus, the application is intended to encompass all such changes and modifications as fall within the scope of the claims, together with their equivalents.

Claims

1. A RAG-based PDF intelligent retrieval and generation method, characterized in that, The method comprises the following steps: obtaining input document data, using a pre-established classification model to analyze the document data, extracting text content and image content to form a first data set; using a deep learning model to extract features from the image content in the first data set, and simultaneously applying natural language processing techniques to the text content in the first data set for semantic analysis to obtain a multi-modal feature set; applying an information integration algorithm to uniformly encode the multi-modal feature set to generate a second data set, and if the integrity of the fusion feature vector in the second data set is detected to be lower than a preset threshold, supplementing context semantic analysis to fill in missing information; using a preset index construction mechanism to cluster the fusion feature vector in the second data set to generate a retrieval index library containing a classification index structure; according to the user input query condition, using a similarity algorithm to match the retrieval index library, and if the relevance of the matching result is lower than a preset threshold, expanding the semantic query range and re-matching.

2. The RAG based PDF intelligent retrieval and generation method of claim 1, wherein, The step of obtaining input document data, using a pre-established classification model to analyze the document data, and extracting text content and image content to form a first data set comprises: obtaining original content from the input document data, using a pre-established classification tool to preliminarily analyze the document, and extracting text content and image content to form an initial set; for the initial set, determining its integrity by a content recognition tool, if the text content is missing, re-obtaining complete data by a document scanning tool to obtain a first text set; if the image content resolution is lower than a preset threshold, processing by an image enhancement tool to obtain a first image set; for the first text set, using a text segmentation tool for segmentation processing to obtain segmented text units, and using a semantic matching tool to determine the business category of the text units to obtain a second text set; for the first image set, using an image segmentation tool to split into multiple image units, using a feature extraction tool to obtain the visual features of the image units, and if the feature value is lower than a preset threshold, adjusting by an image optimization tool to obtain a second image set; according to the second text set and the second image set, using a data fusion tool for association mapping to form a unified business data set, and using a content verification tool to detect the consistency of the data set to determine whether it meets the classification target to obtain the final first data set.

3. The RAG based PDF intelligent retrieval and generation method of claim 1, wherein, The step of using a deep learning model to extract features from the image content in the first data set, and simultaneously applying natural language processing techniques to the text content in the first data set for semantic analysis to obtain a multi-modal feature set comprises: for the image content in the first data set, using an image segmentation tool to split it into multiple visual element units, using a feature extraction tool to process the visual element units to obtain corresponding image features, and if the value of the image features is lower than a preset threshold, adjusting by an image enhancement tool to obtain an adjusted image feature set; According to the adjusted image feature set, a feature mapping tool is used to compare the image features with preset classification criteria, and if the criteria are not met, a feature optimization tool is used for secondary processing to determine an image feature subset that meets the criteria; For the text content in the first data set, a text segmentation tool is used to split it into multiple language units, a semantic analysis tool is used to process the language units to obtain corresponding text semantic features, and a text semantic set is obtained; Through a data integration tool, the image feature subset and the text semantic set are fused in multiple modalities, and a content classification tool is used to match the fused data for business to determine whether it meets the preset target, and a final multi-modal feature set is obtained.

4. The RAG based PDF intelligent retrieval and generation method of claim 1, wherein, According to the multi-modal feature set, an information integration algorithm is applied for unified encoding processing to generate a second data set, and if the integrity of the fused feature vector in the second data set is detected to be lower than a preset threshold, a step of supplementing context semantic analysis to fill in missing information includes: According to the multi-modal feature set, an information integration tool is used to uniformly process feature encoding to generate a preliminary fused data set, and the encoding consistency of the preliminary fused data set is detected, and if the consistency is detected to be lower than a preset threshold, a data calibration tool is used for adjustment to obtain a fused data set with consistent encoding; For the fused data set with consistent encoding, a feature detection tool is used to evaluate the vector integrity, and if the integrity of the fused data set is found to be lower than a preset threshold, a context analysis tool is used to extract supplementary information to determine a fused data set with improved integrity; Through an information filling tool, for the missing part in the fused data set with improved integrity, a semantic supplement tool is used for content filling to obtain a filled fused data set; A business matching tool is used to classify and compare the filled fused data set, and according to the pre-established rules, the fused data set is judged to obtain a final data set that meets the business target.

5. The RAG based PDF intelligent retrieval and generation method of claim 1, wherein, The step of using a preset index construction mechanism to cluster the fused feature vectors in the second data set to generate a retrieval index library containing a classification index structure includes: For the fused features in the second data set, a preset mechanism is used to preliminarily group the feature vectors, and a clustering operation is used to obtain a preliminary classification result; According to the preliminary classification result obtained by the clustering operation, for the construction process of the classification index, an index construction tool is used to structure the grouped feature vectors to determine the hierarchical distribution of the classification index; Through the hierarchical distribution of the classification index, a retrieval index generation tool is used to optimize and adjust the index structure to obtain a retrieval index framework with efficient query capability; For the retrieval index framework, a data processing module is combined to map the feature vectors in the second data set in batches, and if a data deviation is detected during the mapping process, a calibration tool is used for correction to determine a mapping result that meets the standard; According to the mapping result that meets the standard, for the construction process of the retrieval library, a storage management tool is used to associate and store the optimized index structure and the feature vectors to obtain complete retrieval library content; Through complete search library content, the matching degree of the classification index and the search index is detected, if the matching degree is lower than a preset threshold, the index structure is locally optimized through an adjustment tool, and a final search library architecture is determined; According to the final search library architecture, the query response capability of the search library is tested by using a verification tool, and verified search library data is obtained.

6. The RAG based PDF intelligent retrieval and generation method of claim 1, wherein, The step of expanding the semantic query range and re-matching if the relevance of the matching result is lower than the preset threshold according to the user input query condition, using a similarity algorithm to match the search index library, comprises: According to the user input query condition, the input content is structured by a parsing tool, and the decomposed query elements are obtained; According to the decomposed query elements, a cosine similarity algorithm is used to perform preliminary matching operation on the index structure in the search index library, and an initial matching result is obtained; According to the initial matching result, the matching result is calculated by a relevance evaluation module, and the matching degree of the query condition is determined whether it reaches a preset threshold; If the matching degree is lower than the preset threshold, the query condition is adjusted in range by a semantic expansion tool, and the expanded semantic content is obtained; According to the expanded semantic content, the search index library is re-matched to determine the updated matching result; According to the updated matching result, the completeness of the matching result is detected by a verification module, if the data is missing, the related information is extracted from the index structure by a supplement tool, and the complete matching data is obtained; According to the complete matching data, a storage management module is used to associate and save the matching result and the user input query condition to determine the final query record.

7. A RAG-based PDF intelligent retrieval and generation system for implementing the RAG-based PDF intelligent retrieval and generation method according to any one of claims 1 to 6, characterized in that, The RAG-based PDF intelligent search and generation system comprises: A forming module is configured to obtain input document data, parse the document data by using a pre-established classification model, extract text content and image content to form a first data set; An acquisition module is configured to extract features of image content in the first data set by using a deep learning model, and perform semantic analysis on text content in the first data set by applying natural language processing technology to obtain a multi-modal feature set; A first generation module is configured to generate a second data set by applying information integration algorithm for unified coding according to the multi-modal feature set, and supplement missing information by context semantic analysis if the completeness of the fusion feature vector in the second data set is lower than a preset threshold; A second generation module is configured to generate a search index library containing a classification index structure by clustering the fusion feature vector in the second data set by using a preset index construction mechanism; A matching module is configured to match the search index library by using a similarity algorithm according to a user input query condition, and re-match by expanding the semantic query range if the relevance of the matching result is lower than a preset threshold.

8. The RAG based PDF intelligent search and generation system of claim 7, wherein, The forming module comprises: A forming unit is configured to obtain original content from input document data, and perform preliminary analysis on the document by using a pre-established classification tool to extract text content and image content to form an initial set. The first obtaining unit is configured to determine the integrity of the initial set by using a content recognition tool, and if the text content is incomplete, obtain complete data again by using a document scanning tool to obtain a first text set; The second obtaining unit is configured to, if the image content resolution is lower than a preset threshold, process the image content by using an image enhancement tool to obtain a first image set; The third obtaining unit is configured to, for the first text set, perform word segmentation processing by using a text word segmentation tool to obtain segmented text units, perform category labeling on the text units by using a semantic matching tool to determine the business category, and obtain a second text set; The fourth obtaining unit is configured to, for the first image set, split the image set into a plurality of image units by using an image segmentation tool, obtain visual features of the image units by using a feature extraction tool, and if the feature values are lower than a preset threshold, adjust the image set by using an image optimization tool to obtain a second image set; The fifth obtaining unit is configured to, according to the second text set and the second image set, perform association mapping by using a data fusion tool to form a unified business data set, detect the consistency of the data set by using a content verification tool to determine whether the data set meets the classification target, and obtain a final first data set.

9. The RAG based PDF intelligent search and generation system as claimed in claim 7, wherein, The obtaining module comprises: The sixth obtaining unit is configured to, for the image content in the first data set, split the image content into a plurality of visual element units by using an image segmentation tool, process the visual element units by using a feature extraction tool to obtain corresponding image features, and if the values of the image features are lower than a preset threshold, adjust the image features by using an image enhancement tool to obtain an adjusted image feature set; The first determining unit is configured to, according to the adjusted image feature set, compare the image features with a preset classification standard by using a feature mapping tool, and if the comparison does not meet the standard, perform secondary processing on the image features by using a feature optimization tool to determine an image feature subset that meets the standard; The seventh obtaining unit is configured to, for the text content in the first data set, split the text content into a plurality of language units by using a text word segmentation tool, process the language units by using a semantic analysis tool to obtain corresponding text semantic features, and obtain a text semantic set; The eighth obtaining unit is configured to, by using a data integration tool, perform multi-modal fusion on the image feature subset and the text semantic set, perform business matching on the fused data by using a content classification tool to determine whether the data meets a preset target, and obtain a final multi-modal feature set.

10. The RAG based PDF intelligent search and generation system as claimed in claim 7, wherein, The first generation module comprises: The ninth obtaining unit is configured to, according to the multi-modal feature set, uniformly process feature encoding by using an information integration tool to generate a preliminary fused data set, detect the encoding consistency of the preliminary fused data set, and if the consistency is detected to be lower than a preset threshold, adjust the preliminary fused data set by using a data calibration tool to obtain a fused data set with consistent encoding. The second determining unit is configured to, for the fused data set with consistent encoding, evaluate vector integrity by using a feature detection tool, and if it is found that the integrity of the fused data set is lower than a preset threshold, extract supplementary information by using a context analysis tool to determine a fused data set with improved integrity; The tenth obtaining unit is configured to, for the missing part in the fused data set with improved integrity, fill content by combining a semantic supplement tool through an information filling tool to obtain a filled fused data set; The eleventh obtaining unit is configured to classify and compare the filled fused data set by using a business matching tool, judge the fused data set according to a pre-established rule, and obtain a final data set meeting a business target.

Images