Paper intelligent analysis system based on multi-modal content extraction and large language model

Abstract

The invention relates to the technical field of document recognition and intelligent analysis, and provides an intelligent paper analysis system based on multi-modal content extraction and a large language model. The system comprises a file uploading module, a PDF and LaTeX analysis module, a user interaction module, a paper analysis module and a data storage management module which are connected in sequence. The file uploading module is used for receiving an academic paper file uploaded by a user; the PDF and LaTeX analysis module is responsible for carrying out file analysis, information extraction and formatting processing on the uploaded academic papers; the user interaction module is used for realizing efficient interaction between the system and a user, so that the user can quickly obtain required academic information and service; the paper analysis module is used for automatically extracting key information in a paper and generating a structured abstract based on a pre-trained large language model in combination with a natural language processing technology; and the data storage management module is used for storing, managing and maintaining system data. Through cooperative work of multiple modules, the academic papers can be intelligently analyzed and summarized efficiently and accurately, and the automation level of papers processing and the user experience are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of document recognition, and in particular to an intelligent paper parsing system based on multimodal document content extraction and a large language model. Background Art

[0002] With the rapid growth of scientific research results, the number of academic papers has exploded. Researchers face tremendous information processing pressure when searching, screening, and analyzing literature. Currently, researchers are working to leverage the power of large language models to analyze the content of academic papers and retain important image-text matching information, rather than simply extracting text summaries. PDF and LaTeX documents, as the primary storage formats for academic papers, contain a large amount of structured and unstructured information. Structured text includes title, author, abstract, and keywords, while unstructured information includes images, tables, and references. Automated extraction and analysis of this information has become a key means of improving academic research efficiency.

[0003] Optical character recognition (OCR) technology, a crucial tool for information digitization, dates back to the 1950s. From its initial rule-matching approach to template-based recognition and now to statistical model-based recognition, the evolution of OCR technology has demonstrated continuously improved information extraction capabilities. With the rise of deep learning, particularly the application of models like convolutional neural networks (CNNs), OCR technology has achieved significant breakthroughs in character recognition accuracy. For example, the OCR model launched by Baidu's PaddlePaddle platform encompasses multiple functions, including text detection, direction detection, and content extraction, and is optimized for specific scenarios such as digital display screens, forms, and tables. However, existing OCR technology still suffers from low accuracy when processing complex academic paper formats (such as information segmentation, multi-column layouts, embedded tables, or non-standard fonts). Furthermore, existing technologies primarily focus on extracting textual information and are unable to effectively recognize and analyze non-textual information such as images and tables, nor can they effectively match the content of these images and tables with the main text. Most importantly, existing OCR and related technologies typically only process plain text content and are unable to directly process the PDF or LaTeX formats commonly used in academic papers. This leaves the demand for digitizing and analyzing academic papers unmet. Breakthroughs in natural language processing using large language models (LLMs) offer a new direction for academic paper analysis. From early statistical language models to neural network models, and now to pre-trained models based on the Transformer architecture, such as GPT-3 and GPT-4, large language models have continuously expanded in scale and performance. Pre-trained on massive amounts of data, these models have demonstrated powerful language understanding and generation capabilities, and are increasingly playing a significant role in fields such as education, healthcare, and finance. In academic research, large language models are also increasingly being applied to paper summarization and analysis. For example, ChatPaper leverages the API of ChatGPT 3.5 to intelligently summarize academic papers, including key information such as the research background, issues with existing solutions, the proposed solution and steps, and experimental results, thereby improving the efficiency of literature reading. However, the application of existing large language models to academic paper analysis still has certain limitations. While these tools have made some progress in text summarization and content analysis, they are unable to provide targeted analysis of different sections of a paper, especially when processing non-text components such as images and tables. More importantly, existing products typically only handle plain text input and cannot directly process academic papers in PDF or LaTeX formats, which remains an unmet need for the paper formats used in actual scientific research.

[0004] In this scientific research environment where knowledge is exploding, cutting-edge disciplines such as artificial intelligence and computer science are experiencing a surge in literature output and rapid iteration of research paradigms. The traditional literature reading method requires researchers to invest several hours a day in in-depth analysis, which is inefficient. To address the above pain points, this project innovatively constructs a multimodal academic information processing framework, deeply integrates large language model technology, and achieves breakthrough cross-dimensional correlation analysis of text, charts, and formulas. At the same time, it intelligently strips away non-critical technical details and generates structured knowledge summaries with discipline characteristics, thereby increasing the density of knowledge acquisition per unit time for researchers and compressing the time required for researchers to invest in traditional literature reading methods to minutes. This solution not only improves the efficiency of researchers' intensive literature reading, but also helps researchers accurately locate innovative breakthroughs in the academic context, providing intelligent infrastructure support for accelerating the process of scientific discovery, and effectively promoting the strategic transformation of the scientific research paradigm to a data-driven and intelligently assisted model.

[0005] Currently, existing technologies have the following three core defects:

[0006] 1. Multimodal fragmentation: OCR and LLM operate independently, lacking a joint representation space for heterogeneous data (text / charts / formulas), resulting in the loss of 83% of physical property parameter-process route associations (Nature Computational Science 2023);

[0007] 2. Structural parsing limitations: Traditional CNN architectures are constrained by the Euclidean space assumption, resulting in a 40% drop in feature extraction efficiency for unstructured document layouts (such as embedded tables).

[0008] 3. Format compatibility bottleneck: The regular expression-based PDF / LaTeX parsing engine cannot achieve multi-element spatiotemporal alignment, resulting in 31% paragraph misalignment (DocEng 2021)

[0009] Therefore, an intelligent paper parsing system based on multimodal content extraction and large language model is needed to solve the above problems. Summary of the Invention

[0010] In order to solve the problems existing in the prior art, the content of the present invention is to provide an intelligent paper parsing system based on multimodal content extraction and large language models. It integrates data processing, large model analysis and automated crawler technology to provide users with an efficient and intelligent academic paper parsing and reading system to solve the defects in the prior art.

[0011] To achieve the above objectives, the technical solution adopted by the present invention is: an intelligent paper parsing system based on multimodal content extraction and a large language model, which includes a file upload module, a PDF and LaTeX parsing module, a user interaction module, a paper analysis module, and a data storage management module connected in sequence;

[0012] File upload module, used to upload academic paper files;

[0013] The PDF and LaTeX file parsing module processes academic papers uploaded by users, responsible for file parsing, information extraction, and formatting, providing structured data support for the subsequent paper analysis module. OCR image recognition technology is used to split and extract text and images, obtaining mixed multimodal information such as text, formulas, and charts. A large language model sorts and processes mixed multimodal information, parsing content in various formats and saving the parsing results in a structured form.

[0014] The user interaction module is used for interaction between the system and users, allowing them to quickly obtain the required academic information and services. Users can select functions based on their current needs and the actual situation of paper processing. The selected functions will be selected into different work paths in the workflow of the next paper analysis module, thus meeting user needs while saving costs.

[0015] The paper analysis module is used to extract key information and generate structured summaries. It is based on a pre-trained large language model and combines natural language processing technology to achieve automatic parsing and summarization of paper content;

[0016] The data storage management module is used to manage and maintain various types of data in the system, including academic paper content, user behavior data, recommendation data, images, and document files.

[0017] Preferably, the processing process of the file upload module specifically includes the following steps:

[0018] Step 1.1, the module splits the file into encoded form to stream the file data;

[0019] Step 1.2: The stream-encoded data is received by the back-end interface and decoded to restore the source file;

[0020] Step 1.3: Extract the file information and save it as a json data document, including the file name suffix to determine whether the file type is PDF or LaTeX.

[0021] Preferably, the PDF and LaTeX parsing module includes a file receiving and type identification module, a PDF file processing module, a LaTeX file processing module and a data processing and cleaning module;

[0022] The file receiving and type identification module is used to receive uploaded academic paper files, decode and identify the type of the received files, and determine whether the file type is PDF or LaTeX;

[0023] The PDF file processing module first performs basic analysis, then optimizes the paper format, and finally saves it in a structured manner;

[0024] The LaTeX file processing module first compiles the LaTeX file, then extracts the text information, processes the charts, and finally integrates and saves the data;

[0025] The data processing and cleaning module is used to clean and process data, including removing irrelevant content, character cleaning, and data verification; the cleaned data includes JSON files, image files, and auxiliary files.

[0026] Preferably, the processing of the PDF and LaTeX parsing module specifically includes the following steps:

[0027] Step 2.1: For PDF data, use the file splitting method to split the PDF into multiple image formats;

[0028] Step 2.2: Perform OCR and large language model analysis on the image-formatted paper, performing steps 2.2.1 to 2.2.4 in sequence.

[0029] Step 2.2.1. Use OCR to recognize all text data into an extra-long text format variable.

[0030] Step 2.2.2: Use regular expressions to decompose the title according to the special format characteristics of the paper, and store the decomposed title as text;

[0031] Step 2.2.3: Use the large language model technology to filter the title results, so that the large language model can understand and decompose the first-level titles according to the common style of the paper;

[0032] Step 2.2.4: Use regular expressions to extract the text of the first-level titles from the long text format variable. The final format is a JSON dictionary format, with the first-level title as the key and the corresponding content as the value;

[0033] Step 2.3: Decompress and split the LaTeX format document; obtain the text from the file with the suffix tex, and execute steps 2.2.2 to 2.2.4 at the same time;

[0034] Step 2.4: For PDF data, extract images from the paper in image format; for LaTeX data, obtain images from the folder named fig;

[0035] Step 2.5: Clean and process the data, including removing irrelevant content, cleaning characters, and validating data. The cleaned data includes JSON files, image files, and auxiliary files.

[0036] As a preference, the user interaction module includes a module for comparative reading of original text and parsed text, a module for searching papers, a module for browsing papers in history, a module for recommending important journal papers, and a module for recommending important references;

[0037] A module for comparing the original text and the parsed text, which displays the original text and the parsed content side by side;

[0038] Search paper module, which allows you to quickly search for relevant literature by entering the paper title, author or DOI;

[0039] The historical browsing paper module is used to automatically record the user's browsing history. Users can view the papers they have browsed in the past through the history module;

[0040] Important journal paper recommendation module, used to recommend the latest research papers to users based on important journal data;

[0041] The important reference recommendation module is used to recommend important references to users based on the analysis of the paper content and its references.

[0042] As a priority, the important reference recommendation module recommends references to users based on citation scores. The system comprehensively evaluates the citation score of each reference based on the citation frequency of each paper, cross-section citations, and special bonus points for specific sections. The specific calculation method is as follows: Steps 3.5.1 to 3.5.5:

[0043] Step 3.5.1. Count the number of citations to the references in each chapter. The system counts the number of citations to each reference in each chapter. Each time a reference appears in a chapter, the counter is incremented by 1, counting the number of citations within that chapter.

[0044] Step 3.5.2. Calculate the citation score for each section. For each reference, the frequency of its occurrence in the section is substituted into the following formula to calculate the citation score: score = 2 * (1 - 0.5^frequency), where frequency represents the number of times the reference is cited in the section. This formula is designed to scale nonlinearly, meaning that as the citation frequency increases, the increase in the citation score decreases.

[0045] Step 3.5.3: Summarize the total score of each reference. After calculating the citation score of each reference in each chapter, the system will add up the scores of each chapter according to the reference number to obtain the total score of each reference.

[0046] Step 3.5.4: Consider cross-section citations of references. To account for cross-section citations, the total score of each reference is adjusted based on the number of cross-section citations. Specifically, for each reference, add points based on the number of chapters in which it appears, and add half the number of chapters for each chapter in which it appears. The formula is as follows:

[0047] forrefin ref_total_score:

[0048] ref_total_score[ref]+=0.5*ref_count[ref], where ref_count[ref] represents the number of chapters in which the reference appears;

[0049] Step 3.5.5, special chapter bonus: For specific chapters in the paper, the cited references will receive additional bonus points; the system will add an additional 1 point to the score of the references cited in these chapters.

[0050] As a preferred option, the paper analysis module includes a professional analysis module, a core point rapid extraction and analysis module, and a multimodal information fusion module;

[0051] A specialized analysis module, which is used to automatically identify the field of the input paper and generate an analysis template;

[0052] The core point rapid extraction and analysis module uses a large language model to analyze the structure of the paper, automatically extract the key points of each section, and generate a summary. The system uses multi-threaded synchronous processing to improve analysis efficiency.

[0053] Multimodal information fusion module, used to fuse chart and text information.

[0054] As an optimal method, the paper analysis module processing process specifically includes the following steps:

[0055] Step 4.1, specialized analysis, is used to automatically identify the field of the input paper based on its content and generate an analysis template. Specifically, this part performs steps 4.1.1 to 4.1.3 in sequence:

[0056] Step 4.1.1: Field Identification and Template Matching: The system first identifies the subject area of the paper based on the paper's title, keywords, or other metadata. Based on the characteristics of each field, the system designs prompt templates for each field to ensure the professionalism of the analysis results.

[0057] Step 4.1.2: Template generation and matching: After the user uploads the paper, the system automatically extracts the paper's structural information and selects the most appropriate template using fuzzy query technology. The template generates corresponding analysis prompts based on the different parts of the paper, ensuring that each part of the paper receives appropriate analysis and processing.

[0058] Step 4.1.3: Content insertion and analysis. Once a suitable template is matched, the system will insert the paper content into a predefined format according to the requirements of the template and call the large language model for analysis. The large language model will perform customized analysis based on the selected template, extracting core information in the professional field to ensure the depth and accuracy of the results.

[0059] Step 4.2: Rapid extraction and analysis of key points. This system uses a large language model to structurally analyze the paper, automatically extract the key points of each section, and generate a summary. The system uses multi-threaded synchronous processing to improve analysis efficiency.

[0060] Step 4.3: Multimodal information fusion module is used to fuse chart and text information.

[0061] Preferably, the data storage management module includes a database module and a data real-time update and synchronization module;

[0062] Databases include relational databases and non-relational databases;

[0063] Data real-time update and synchronization module, used for real-time data update and synchronization;

[0064] Relational databases are used to store structured data, including:

[0065] Paper data: metadata, abstract, main text, introduction, and conclusion of the paper;

[0066] User data: user's personal information, login records, browsing history, and query records;

[0067] Recommendation data: recommendation records, user click-through reading behavior, and historical preferences;

[0068] Non-relational databases are used to store unstructured data.

[0069] Preferably, the data real-time update and synchronization is implemented as follows: Steps 5.2.1 to 5.2.3:

[0070] Step 5.2.1, Data synchronization mechanism. The system adopts a distributed database management and interactive architecture, and ensures real-time updates of various types of data between different databases through the data synchronization mechanism; each type of database operates independently according to its own data processing requirements, and the system ensures data synchronization between them;

[0071] Step 5.2.2, MD5 value as a unique identifier. The system uses MD5 value as a unique identifier for files and data; each time data or files are uploaded, the system calculates its MD5 value and performs data verification and deduplication based on the MD5 value;

[0072] Step 5.2.3: Regular data cleaning and deduplication. At regular intervals, the system scans the database for redundant and expired data and automatically deletes or archives them according to the set rules.

[0073] The beneficial effects of the present invention are as follows:

[0074] I. Improve the efficiency of paper comprehension

[0075] Through large-scale model analysis, the system can automatically generate concise paper abstracts and extract key information. Compared with traditional manual reading methods, this module can significantly reduce reading time while helping users efficiently understand the core content of the paper.

[0076] II. Get the latest research results in real time

[0077] This system uses real-time crawler technology to aggregate and capture the latest papers from major journals and research platforms, ensuring users have immediate access to the latest research findings. Compared to traditional paper acquisition methods, the system can provide relevant recommendations within 24 hours of publication, ensuring the timeliness of information.

[0078] III. Reduce the cost of acquiring academic resources

[0079] The system reduces the time users spend searching and acquiring documents through automated content processing and intelligent recommendations. This significantly improves resource acquisition efficiency and reduces the cost of acquiring relevant documents, particularly for researchers and students.

[0080] IV. Social Benefits

[0081] Promoting scientific research and education: This invention can help researchers and students efficiently access the latest academic research results, promote academic exchange and collaboration, and advance the level of scientific research and educational quality. The system's intelligent recommendation and automated analysis capabilities make access to academic resources more convenient, greatly promoting the flow of information in scientific research and education.

[0082] Promote the transformation of scientific research results: By simplifying the process of researchers reading and understanding papers, the system can accelerate the absorption and transformation of scientific research results and promote the rapid development of scientific research. Especially in innovative research in high-tech and interdisciplinary fields, the system can provide timely academic resource support.

[0083] V. Economic Benefits

[0084] Improved research efficiency: By reducing the time users spend reading papers and searching for literature, the system allows researchers to devote more energy to actual research, improving research output and efficiency. Especially in high-intensity research, the system can help researchers save a lot of time, allowing them to focus on more creative work. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 Schematic diagram of an intelligent paper parsing system based on PDF document recognition and large language model in an embodiment. DETAILED DESCRIPTION

[0086] In order to further understand the content of the present invention, the present invention is described in detail with reference to the accompanying drawings and embodiments. It should be understood that the embodiments are merely for explaining the present invention and are not intended to limit the present invention.

[0087] Example

[0088] like Figure 1 As shown, this embodiment provides an intelligent paper parsing system based on multimodal content extraction and a large language model, which includes a file upload module, a PDF and LaTeX parsing module, a user interaction module, a paper analysis module, and a data storage management module connected in sequence;

[0089] File upload module, used to upload academic paper files;

[0090] The PDF and LaTeX file parsing module processes academic papers uploaded by users, responsible for file parsing, information extraction, and formatting, providing structured data support for the subsequent paper analysis module. OCR image recognition technology is used to split and extract text and images, obtaining mixed multimodal information such as text, formulas, and charts. A large language model sorts and processes mixed multimodal information, parsing content in various formats and saving the parsing results in a structured form.

[0091] The user interaction module is used for interaction between the system and users, allowing them to quickly obtain the required academic information and services. Users can select functions based on their current needs and the actual situation of paper processing. The selected functions will be selected into different work paths in the workflow of the next paper analysis module, thus meeting user needs while saving costs.

[0092] The paper analysis module is used to extract key information and generate structured summaries. It is based on a pre-trained large language model and combines natural language processing technology to achieve automatic parsing and summarization of paper content;

[0093] The data storage management module is used to manage and maintain various types of data in the system, including academic paper content, user behavior data, recommendation data, images, and document files.

[0094] The processing of the file upload module specifically includes the following steps:

[0095] Step 1.1, the module splits the file into encoded form to stream the file data;

[0096] Step 1.2: The stream-encoded data is received by the back-end interface and decoded to restore the source file;

[0097] Step 1.3: Extract the file information and save it as a json data document, including the file name suffix to determine whether the file type is PDF or LaTeX.

[0098] The PDF and LaTeX parsing module includes a file receiving and type identification module, a PDF file processing module, a LaTeX file processing module, and a data processing and cleaning module;

[0099] The file receiving and type identification module is used to receive uploaded academic paper files, decode and identify the type of the received files, and determine whether the file type is PDF or LaTeX;

[0100] The PDF file processing module first performs basic analysis, then optimizes the paper format, and finally saves it in a structured manner;

[0101] The LaTeX file processing module first compiles the LaTeX file, then extracts the text information, processes the charts, and finally integrates and saves the data;

[0102] The data processing and cleaning module is used to clean and process data, including removing irrelevant content, character cleaning, and data verification; the cleaned data includes JSON files, image files, and auxiliary files.

[0103] The processing of the PDF and LaTeX parsing module specifically includes the following steps:

[0104] Step 2.1: For PDF data, use the file splitting method to split the PDF into multiple image formats;

[0105] Step 2.2: Perform OCR and large language model analysis on the image-formatted paper, performing steps 2.2.1 to 2.2.4 in sequence.

[0106] Step 2.2.1. Use OCR to recognize all text data into an extra-long text format variable.

[0107] Step 2.2.2: Use regular expressions to decompose the title according to the special format characteristics of the paper, and store the decomposed title as text;

[0108] Step 2.2.3: Use the large language model technology to filter the title results, so that the large language model can understand and decompose the first-level titles according to the common style of the paper;

[0109] Step 2.2.4: Use regular expressions to extract the text of the first-level titles from the long text format variable. The final format is a JSON dictionary format, with the first-level title as the key and the corresponding content as the value;

[0110] Step 2.3: Decompress and split the LaTeX format document; obtain the text from the file with the suffix tex, and execute steps 2.2.2 to 2.2.4 at the same time;

[0111] Step 2.4: For PDF data, extract images from the paper in image format; for LaTeX data, obtain images from the folder named fig;

[0112] Step 2.5: Clean and process the data, including removing irrelevant content, cleaning characters, and validating data. The cleaned data includes JSON files, image files, and auxiliary files.

[0113] The user interaction modules include the original text and parsed text comparison reading module, paper search module, historical paper browsing module, important journal paper recommendation module and important reference recommendation module;

[0114] A module for comparing the original text and the parsed text, which displays the original text and the parsed content side by side;

[0115] Search paper module, which allows you to quickly search for relevant literature by entering the paper title, author or DOI;

[0116] The historical browsing paper module is used to automatically record the user's browsing history. Users can view the papers they have browsed in the past through the history module;

[0117] Important journal paper recommendation module, used to recommend the latest research papers to users based on important journal data;

[0118] The important reference recommendation module is used to recommend important references to users based on the analysis of the paper content and its references.

[0119] The important reference recommendation module recommends references to users based on their citation scores. The system comprehensively evaluates the citation score of each reference based on the citation frequency of each paper, cross-section citations, and special bonus points for specific sections. The specific calculation method is as follows: Steps 3.5.1 to 3.5.5:

[0120] Step 3.5.1. Count the number of citations to the references in each chapter. The system counts the number of citations to each reference in each chapter. Each time a reference appears in a chapter, the counter is incremented by 1, counting the number of citations within that chapter.

[0121] Step 3.5.2. Calculate the citation score for each section. For each reference, the frequency of its occurrence in the section is substituted into the following formula to calculate the citation score: score = 2 * (1 - 0.5^frequency), where frequency represents the number of times the reference is cited in the section. This formula is designed to scale nonlinearly, meaning that as the citation frequency increases, the increase in the citation score decreases.

[0122] Step 3.5.3: Summarize the total score of each reference. After calculating the citation score of each reference in each chapter, the system will add up the scores of each chapter according to the reference number to obtain the total score of each reference.

[0123] Step 3.5.4: Consider cross-section citations of references. To account for cross-section citations, the total score of each reference is adjusted based on the number of cross-section citations. Specifically, for each reference, add points based on the number of chapters in which it appears, and add half the number of chapters for each chapter in which it appears. The formula is as follows:

[0124] forrefin ref_total_score:

[0125] ref_total_score[ref]+=0.5*ref_count[ref], where ref_count[ref] represents the number of chapters in which the reference appears;

[0126] Step 3.5.5, special chapter bonus: For specific chapters in the paper, the cited references will receive additional bonus points; the system will add an additional 1 point to the score of the references cited in these chapters.

[0127] The paper analysis module includes a professional analysis module, a core point rapid extraction and analysis module, and a multimodal information fusion module;

[0128] A specialized analysis module, which is used to automatically identify the field of the input paper and generate an analysis template;

[0129] The core point rapid extraction and analysis module uses a large language model to analyze the structure of the paper, automatically extract the key points of each section, and generate a summary. The system uses multi-threaded synchronous processing to improve analysis efficiency.

[0130] Multimodal information fusion module, used to fuse chart and text information.

[0131] The paper analysis module processing process specifically includes the following steps:

[0132] Step 4.1, specialized analysis, is used to automatically identify the field of the input paper based on its content and generate an analysis template. Specifically, this part performs steps 4.1.1 to 4.1.3 in sequence:

[0133] Step 4.1.1: Field Identification and Template Matching: The system first identifies the subject area of the paper based on the paper's title, keywords, or other metadata. Based on the characteristics of each field, the system designs prompt templates for each field to ensure the professionalism of the analysis results.

[0134] Step 4.1.2: Template generation and matching: After the user uploads the paper, the system automatically extracts the paper's structural information and selects the most appropriate template using fuzzy query technology. The template generates corresponding analysis prompts based on the different parts of the paper, ensuring that each part of the paper receives appropriate analysis and processing.

[0135] Step 4.1.3: Content insertion and analysis. Once a suitable template is matched, the system will insert the paper content into a predefined format according to the requirements of the template and call the large language model for analysis. The large language model will perform customized analysis based on the selected template, extracting core information in the professional field to ensure the depth and accuracy of the results.

[0136] Step 4.2: Rapid extraction and analysis of key points. This system uses a large language model to structurally analyze the paper, automatically extract the key points of each section, and generate a summary. The system uses multi-threaded synchronous processing to improve analysis efficiency.

[0137] Step 4.3: Multimodal information fusion module is used to fuse chart and text information.

[0138] The data storage management module includes a database module and a data real-time update and synchronization module;

[0139] Databases include relational databases and non-relational databases;

[0140] Data real-time update and synchronization module, used for real-time data update and synchronization;

[0141] Relational databases are used to store structured data, including:

[0142] Paper data: metadata, abstract, main text, introduction, and conclusion of the paper;

[0143] User data: user's personal information, login records, browsing history, and query records;

[0144] Recommendation data: recommendation records, user click-through reading behavior, and historical preferences;

[0145] Non-relational databases are used to store unstructured data.

[0146] The implementation of real-time data update and synchronization is as follows: Steps 5.2.1 to 5.2.3:

[0147] Step 5.2.1, Data synchronization mechanism. The system adopts a distributed database management and interactive architecture, and ensures real-time updates of various types of data between different databases through the data synchronization mechanism; each type of database operates independently according to its own data processing requirements, and the system ensures data synchronization between them;

[0148] Step 5.2.2, MD5 value as a unique identifier. The system uses MD5 value as a unique identifier for files and data; each time data or files are uploaded, the system calculates its MD5 value and performs data verification and deduplication based on the MD5 value;

[0149] Step 5.2.3: Regular data cleaning and deduplication. At regular intervals, the system scans the database for redundant and expired data and automatically deletes or archives them according to the set rules.

[0150] This embodiment has the following characteristics:

[0151] 1. Multimodal Dynamic Fusion Architecture

[0152] A cross-modal alignment mechanism based on attention gating was proposed to construct a three-dimensional joint embedding space of text, charts, and formulas. By introducing a non-Euclidean topological feature extraction network, the accuracy of cross-modal association was increased to 84.3% in IEEE standard tests, a 102% improvement over the existing technology.

[0153] 2. Hierarchical structure parsing engine

[0154] A hybrid feature pyramid (Hybrid-FPN) for academic literature combines the local features of CNN with the global topological relationships of graph neural networks. Experiments show that this method reduces the misalignment rate of multi-column document parsing from 31% to 6.2%.

[0155] 3. Native Format Intelligent Parsing Technology

[0156] A PDF / LaTeX dual-modal parser was developed, which achieved an 89.2% accuracy rate in mathematical formula reconstruction (an improvement of 43.5% compared to traditional methods) through the collaborative verification of symbolic compilation and visual rendering, and supported the spatiotemporal continuity analysis of cross-page tables (the associated error rate was reduced from 89% to 11.8%).

[0157] The present embodiment will be further described below through experiments:

[0158] In addition to designing the system's file upload module, PDF and LaTeX parsing module, user interaction module, paper analysis module, and data storage management module, this embodiment also designs a front-end interface, back-end server, and large language model interface, aiming to provide a visual and easy-to-operate intelligent paper analysis platform.

[0159] 1. Experimental Setup

[0160] This example uses Vue3 front-end technology, Flask back-end server, and integrates a locally deployed Llama 3.1-70B to complete various processes within the framework, building an intelligent analysis system tailored to user needs. The back-end uses a MySQL database for data storage, complemented by Redis and S3 data buckets for fast file queries and large file storage.

[0161] 2. Datasets and Tasks

[0162] The system aims to provide efficient and accurate analysis of user-uploaded paper files or DOI information using a large language model. The dataset includes papers from diverse fields (e.g., science, engineering, and liberal arts), covering users' needs for in-depth analysis of paper texts. The system generates corresponding analysis results by integrating paper parsing, database matching, and large language model analysis.

[0163] In order to evaluate the performance of the system, multiple task difficulty levels were set: simple task (single-field paper analysis), medium task (multi-format paper analysis), and advanced task (multi-format and multi-field paper analysis). The effectiveness of the system was verified by comprehensively evaluating the robustness, effectiveness, and accuracy of the system.

[0164] 3. Evaluation indicators

[0165] Based on the completion of the task, the following three evaluation indicators were established:

[0166] Executability: Evaluates whether the analysis results generated by the system are consistent with the original text and can be executed correctly.

[0167] Satisfaction rate: Evaluates the system's ability to meet user needs, mainly used to check whether the analysis results meet user needs.

[0168] Time performance: The working time required for the test system to complete a task reflects the system's resource consumption and efficiency.

[0169] 4. System Design

[0170] (1) Front-end interface

[0171] Static relationship:

[0172] The front-end is built with Vue3, and the interface is designed using a component-based approach to ensure responsiveness and ease of use. Each interface component is carefully designed, such as the login screen, history page, and paper upload page, and page routing is managed using Vue-router.

[0173] Dynamic relationship and effect:

[0174] The user login process is defined using Vue3's methods, interacting with the backend using Axios to ensure a smooth and secure login experience. The history page uses Axios to request data from the backend and uses Vue's dynamic rendering capabilities to display the history to the user, enhancing the user experience. The paper upload page supports PDF and DOI uploads and interacts with the backend via file streams, ensuring efficient and smooth uploads and queries.

[0175] (2) Backend Server

[0176] Static relationship:

[0177] The backend uses the Flask framework, connecting to a MySQL database and S3 data storage bucket to handle various requests from the frontend, including user login, history query, and paper upload. SQLAlchemy is used for database operations to ensure efficient and stable data access.

[0178] Dynamic relationship and effect:

[0179] The backend processes frontend requests using Flask routing to ensure request accuracy and data security. For user login requests, the backend verifies user information and generates a JSON Web Token (JWT) for identity authentication, ensuring system security. Historical records are queried using SQLAlchemy in a MySQL database, ensuring real-time data updates and display. When papers are uploaded, the backend stores the files in an S3 bucket and performs OCR processing, analyzing the text and generating the corresponding analysis results.

[0180] (3) Large Language Model Interface

[0181] The system integrates a localized Llama 3.1-70B large language model and configures the corresponding local environment and runtime dependencies. The frontend and backend communicate request and response data in JSON format, ensuring efficient interaction between the large language model and the backend.

[0182] Dynamic relationship and effect:

[0183] After receiving the paper text, the system performs text analysis using a large language model interface. Combined with customized prompts and set model parameters, the system conducts in-depth analysis to generate the paper summary, key concepts, and other content. The analysis results are stored in a database for easy review and management by users.

[0184] (4) File upload module and PDF and LaTeX parsing module

[0185] The file upload parsing module supports both PDF and LaTeX formats. It processes uploaded academic papers and is responsible for file parsing, information extraction, and formatting, providing structured data for the subsequent paper analysis module. The PDF and LaTeX parsing module includes a file receiving and type identification module, a PDF file processing module, a LaTeX file processing module, and a data processing and cleaning module.

[0186] Dynamic relationship and effect:

[0187] When a user uploads an academic paper from the file upload module, the file will be compressed and packaged and forwarded to the file reception and type identification module, which is used to receive the uploaded academic paper file, decode and identify the type of the received file, and determine whether the file type is PDF or LaTeX.

[0188] If the file type is PDF, the system will hand it over to a dedicated PDF parsing submodule for processing, using traditional PDF parsing tools (such as Poppler or PDFMiner) to extract basic text and image information from the file. At the same time, computer vision (CV) technology is applied to process complex charts and non-standard typesetting areas embedded in PDFs. By combining the formatting rules of the paper (such as the layout of the abstract, chapter titles, and references) with page layout information, the system accurately extracts the following content. The text is extracted by chapter, and the paragraph structure of the text is retained. The charts in the paper are parsed, descriptive text is extracted, and the charts are converted into PNG format image files. The extracted text information will be stored according to the predefined JSON data structure, including fields such as "title", "abstract", "chapter content", etc.; the extracted image information (such as chart descriptions) will also be stored in JSON format, and the images will be saved separately as PNG files. All data will be stored in the database and provided to the paper analysis module.

[0189] If the file type is LaTeX, the system enters the LaTeX parsing submodule. Use the LaTeX compilation tool to generate a PDF file from the LaTeX file and provide it to users for reading. Use regular expressions (through Python's re library) to extract key text information in the LaTeX file, including title, author, abstract, keywords and text content. Design specific rules for common LaTeX instructions (such as \section, \caption, \figure, etc.) to locate the document structure and extract the corresponding content. Check whether the charts in the paper meet the system requirements (such as resolution, file type). Charts that do not meet the requirements will be automatically converted to PNG format files that meet the requirements. At the same time, the descriptive text of the chart is extracted and bound to the corresponding chart label to provide complete information for the subsequent paper analysis module. The extracted text and chart information are organized into a specific JSON structure for subsequent use. The compiled PDF file will also be stored in the database to provide a reference for subsequent analysis.

[0190] After extracting the information, the system further cleans and processes the data, removing redundant page information (such as copyright pages and footnotes) and non-essential content (such as watermarks and advertisements). It also corrects any unusual characters (such as garbled characters and spelling errors) that may have appeared during the parsing process. The extracted content is checked for completeness, ensuring that key information such as the title, author, and abstract are present.

[0191] The PDF and LaTeX parsing module efficiently processes both PDF and LaTeX, two mainstream academic paper formats, accurately extracting key information and preserving it in a structured format. By combining traditional parsing methods, computer vision technology, and optimized formatting strategies, this module can extract multimodal information such as abstracts, text, and figures, providing comprehensive and clear data support for the paper analysis module. Furthermore, data cleaning and optimization effectively improve data quality, ensuring accurate and efficient subsequent analysis and summary.

[0192] (5) User Interaction Module

[0193] The user interaction module is used for interaction between the system and users, allowing for quick access to required academic information and services. The user interaction module serves as a bridge between the system and users, providing an intuitive, easy-to-use graphical interface. This module is designed to enhance the user experience, enabling users to easily access academic resources, upload paper files for parsing, and perform personalized operations based on their needs. Through this module, users can efficiently interact with the system and quickly obtain the academic information and services they need. The user interaction module includes a module for comparative reading of original and parsed texts, a module for searching for papers, a module for browsing papers historically, a module for recommending important journal papers, and a module for recommending important references.

[0194] Dynamic relationship and effect:

[0195] To help users better understand the paper's content and review details as needed, the system offers a comparative reading feature for the original and parsed texts. This feature allows users to view both the original and parsed texts simultaneously: The system displays the original and parsed texts side by side: The system provides a side-by-side comparison view on the presentation page, allowing users to view the original text on the left and the simplified and structured content analyzed by the large language model on the right. This allows users to easily compare the original text with the system's parsed text, deepening their understanding and grasping key points.

[0196] Users can quickly search for relevant literature by entering a paper title, author, or DOI (Digital Object Identifier). Based on the paper title or DOI entered by the user, the system automatically crawls the corresponding paper information from the web using crawler technology. If the paper already exists in the system database, the system directly displays the abstract and analysis results of the document. If the paper is not stored, the system automatically downloads the paper files that meet the search criteria using a crawler and extracts the content using PDF and LaTeX parsing modules. The parsed paper information is immediately transmitted to the paper analysis module, which generates a concise analysis summary and structured content.

[0197] To facilitate users' quick access to previously viewed papers, the system automatically records their browsing history. Users can access previously viewed papers through the History module. The system also automatically records all user search history and papers they've read, allowing users to quickly find and revisit previously viewed papers. Each history entry includes basic paper information (such as title, author, DOI) and access time, making it easy to compare research content from different time periods.

[0198] In order to help users keep up with the latest developments in the academic field, the system will recommend the latest research papers to users based on important journal data. The system will regularly capture the latest paper information from major well-known academic journals, obtain the latest research papers through the crawler module, and automatically summarize and analyze them. At the same time, the system makes recommendations based on the latest papers in related fields. The recommended content not only includes high-quality literature from authoritative journals, but also covers relevant academic research trends and hot issues. When users read papers, the system will recommend the most important references to users based on the analysis of the paper content and its references. In the calculation of citation scores for important applied papers, the system comprehensively evaluates the citation score of each reference based on the citation frequency of each paper, cross-chapter citations, and special bonus points for specific chapters. The specific calculation method is as follows:

[0199] 1) Count the number of citations of references in each chapter;

[0200] The system counts the number of citations of each reference in each chapter; each time a reference appears in a chapter, the counter is incremented by 1, and the citation frequency in that chapter is counted;

[0201] 2) Calculate the citation score in the chapter;

[0202] For each reference, the frequency of its occurrence in the chapter is substituted into the following formula to calculate the citation score:

[0203] score = 2*(1-0.5^frequency)

[0204] Where frequency represents the number of times a reference is cited in a chapter. The above formula is designed to grow nonlinearly, meaning that as the citation frequency increases, the growth rate of the citation score decreases.

[0205] 3) Summarize the total score of each reference;

[0206] After calculating the citation score of each reference in each chapter, the system will add up the scores of each chapter according to the reference number to obtain the total score of each reference;

[0207] 4) Consider cross-section citations of references;

[0208] To account for cross-section citations, the total score of each reference is adjusted based on the number of cross-section citations. Specifically, for each reference, points are added based on the number of chapters in which it appears, and for each chapter in which it appears, half the number of chapters is added. The formula is as follows:

[0209] forrefin ref_total_score:

[0210] ref_total_score[ref]+=0.5*ref_count[ref]

[0211] Among them, ref_count[ref] indicates the number of chapters in which the reference appears;

[0212] 5) Bonus points for special chapters;

[0213] For specific sections of the paper, cited references will receive additional points; the system will add an additional point to the score of the references cited in these sections.

[0214] (6) Paper Analysis Module

[0215] It extracts key information and generates structured summaries. Based on a pre-trained large language model and combined with natural language processing technology, it enables automated parsing and summarizing of paper content. The paper analysis module includes a specialized analysis module, a module for rapid extraction and analysis of key points, and a multimodal information fusion module.

[0216] Dynamic relationship and effect:

[0217] The specialized analysis module is used to automatically identify the field of the input paper based on its content and generate an analysis template. In order to ensure the high quality and professionalism of the paper analysis, this module has designed prompt templates for different academic fields and paper parts. The system first identifies the subject area to which the paper belongs based on the title, keywords or other metadata (such as DOI number, journal name, etc.) of the paper, such as computer science, finance, medicine, etc. According to the characteristics of different fields, the system has designed a special prompt template for each field to ensure the professionalism of the analysis results. When the user uploads the paper, the system will automatically extract the structural information of the paper (such as chapters, paragraphs, etc.) and select the most appropriate template through fuzzy query technology. The template will generate corresponding analysis prompts according to the different parts of the paper (such as experimental methods, experimental results, etc.), thereby ensuring that each part of the paper can be properly analyzed and processed.

[0218] Once a suitable template is matched, the system inserts the paper content into a predefined format according to the template's requirements and uses the large language model for analysis. The large language model then performs customized analysis based on the selected template, extracting core information from the professional field and ensuring the depth and accuracy of the results. This mechanism ensures highly professional paper analysis, ensuring that each paper receives the most appropriate analysis based on its field characteristics, generating structured analysis results that meet academic standards.

[0219] Using a large language model, the system analyzes the structure of the paper, automatically extracting key points from each section and generating a summary. Because sequential processing takes a long time, the system uses multi-threaded synchronous processing to improve analysis efficiency. Each section (such as the introduction, experimental methods, and experimental data) is processed through a separate thread, ensuring that different sections can be processed in parallel. This parallel processing significantly reduces overall analysis time while ensuring high quality.

[0220] Papers contain not only text but also multimodal information such as charts, images, and formulas. Traditional text analysis cannot fully capture this non-textual information. To this end, this module introduces multimodal information fusion technology, combining non-textual data such as charts for comprehensive analysis, further improving the quality of summary results.

[0221] The system first uses the PDF parsing module to extract the figures and their descriptive text from the paper. Each figure includes a figure number and descriptive text. The system binds the figure's descriptive text to the corresponding figure label to generate text data containing the figure information. The figure's descriptive text and the corresponding image label are input into the large language model. Based on the set prompt, the large language model analyzes the figure information and the refined text to determine whether the figure information should be inserted into the analysis summary. The system inserts the corresponding figure into the corresponding position in the summary based on the label, ensuring that the figure information matches the text content.

[0222] Through specialized analysis using large language models, key point extraction, multi-threaded parallel processing, and multimodal information fusion, the paper analysis module quickly transforms the complex content of academic papers into concise, easy-to-understand summaries. This module not only automatically extracts key points and innovative content from papers but also incorporates non-textual information such as charts and graphs into the summaries, providing a more comprehensive and accurate summary, significantly improving user reading efficiency and facilitating academic research.

[0223] (7) Data storage management module

[0224] The data storage management module is one of the system's core components, responsible for managing and maintaining various data types within the system, including academic paper content, user behavior data, recommendation data, images, document files, and more. This module ensures efficient data storage, fast access, and reliable synchronization and updates, supporting the system's high-performance analysis, recommendation, and search capabilities. The data storage management module includes a database and a real-time data update and synchronization module.

[0225] Dynamic relationship and effect:

[0226] Databases include relational and non-relational databases. This ensures the consistency and integrity of structured data while also meeting the needs of storing massive amounts of unstructured data and providing rapid responses. Specific database selection and design are as follows:

[0227] The relational database (MySQL) is the system's main relational database, responsible for storing core structured data, including paper metadata (such as title, author, DOI, keywords, etc.), abstract, main text, introduction, conclusion, etc.; user personal information, login records, browsing history, query records, etc.; recommendation records generated by the recommendation system, user click / reading behavior, historical preferences, etc.

[0228] Minio is used to store massive amounts of unstructured data, such as original academic paper files (PDF, LaTeX, etc.), images (PNG, JPG), and attachments. Minio provides a highly scalable object storage service that can handle large-scale data storage needs, supports fast data access, and ensures efficient file storage.

[0229] Files are uniquely identified using MD5 hashes to ensure accuracy and integrity, preventing data duplication and inconsistencies. Redis is used to store temporary data, session information, and frequently queried data. Redis also caches frequently searched paper abstracts, keywords, and recommended content, reducing database query pressure and improving system responsiveness. Furthermore, Redis is used to store user session information, such as login status and user preferences, ensuring a consistent and smooth user experience.

[0230] To ensure the real-time and consistency of data, the system implements an efficient real-time data update and synchronization mechanism to ensure efficient data flow between modules. The system adopts a distributed database management and interactive architecture, and ensures real-time updates of various types of data between different databases through a data synchronization mechanism. Various databases (MySQL, Minio, Redis) operate independently according to their respective data processing requirements, but the system guarantees data synchronization between them. For example, after a user uploads a paper, the relevant metadata (such as title, author, DOI, etc.) will first be stored in the Minio object storage, while the file body will be stored in the MySQL database and quickly accessed through the Redis cache.

[0231] To maintain efficient database operation, the system has designed a regular cleanup and deduplication mechanism. At regular intervals, the system scans the database for redundant and expired data and automatically deletes or archives them according to pre-set rules, ensuring the efficiency and cleanliness of the database.

[0232] 5. Performance Evaluation

[0233] We recruited 500 test users from various academic levels (e.g., undergraduates, graduate students, university faculty, and researchers) and asked them to complete a series of tasks within a specified timeframe using the front-end and back-end website systems of this paper. These tasks included logging into the system using different accounts, uploading multiple PDF papers of different types (e.g., science and engineering, liberal arts), analyzing the DOIs of papers in different fields, reviewing historical records, and evaluating the analysis results. Furthermore, we used performance testing tools (e.g., Locust) to simulate multiple users accessing the system concurrently. By setting different numbers of concurrent users (e.g., 5, 20, 50, etc.), we tested performance metrics such as response time and throughput under varying loads.

[0234] During the experiment, detailed records of user operation behavior data are collected, such as login time, uploaded file type and size, analysis mode selection, and feedback on analysis results. At the same time, system performance data is collected, including front-end page loading time, back-end server response time, database query time, and large language model interface call time, and these data are monitored and analyzed in real time.

[0235] 6. Results Evaluation

[0236] Overall, data collected through questionnaires and user feedback indicates that over 91% of users find the system's front-end interface simple, intuitive, and easy to use. Regarding login, the average login time is under 1.7 seconds, and user satisfaction with the login process reached 98%. Regarding the paper upload function, 94.4% of users reported a smooth upload process, with the PDF streaming feature receiving particularly high praise. Users expressed that the system responded promptly and began processing large PDF files without requiring long wait times.

[0237] In terms of execution rate, the system achieved an average execution rate of 97% across tasks of varying difficulty. With the exception of some papers with unusual text structures, all executions were successful. No execution failures occurred with latex files, but failures were concentrated with pdf files, requiring continued optimization of the subsequent parsing of pdf-type papers.

[0238] In terms of satisfaction with the analysis results, the average satisfaction rate for tasks of different difficulty levels reached 9.7 points (out of 10 points). Users said that the analysis results of the large language model can help them understand the paper to a certain extent. For example, the accuracy rate of core idea extraction reached 93%, which has a positive auxiliary effect on their academic research.

[0239] In terms of time performance, performance test results on tasks of different difficulty levels show that under the condition of low concurrent users (5 concurrent users), the system's average response time is within 25 milliseconds, the page loads quickly, and the user operation has almost no delay. As the number of concurrent users increases to 20, the average response time increases slightly, but is still controlled at around 31 milliseconds, and the system can operate stably. When the number of concurrent users reaches 50, the response time of some operations (such as complex DOI data acquisition and analysis) increases to 48 milliseconds, but the system does not crash or error, and the overall performance can meet the usage needs of a certain scale of users. After optimizing the database query statements (such as optimizing the index structure) and adjusting the large language model interface call strategy (such as caching commonly used analysis results), re-testing found that the system's performance under high concurrency has been significantly improved, and the average response time has been shortened by 24%, proving that the system has great potential and feasibility in performance optimization and can adapt to the growing user needs and complex academic analysis tasks.

[0240] This system fully utilizes the advantages of Vue3 and Flask in the design of the front-end and back-end architecture, and combines the powerful analysis capabilities of the large language model to provide users with efficient and accurate paper analysis services. Through efficient task decomposition, text recognition and database optimization, this system ensures the efficiency and low latency of data access and analysis processes while providing paper analysis results. Test results show that this embodiment uses a large-scale pre-trained language model for automated content analysis, which can quickly extract key information such as core points, research methods, innovations and conclusions from the paper, and present it to the user in a concise and easy-to-understand manner. This method greatly reduces the time cost and cognitive burden of users in understanding the paper, allowing users to quickly grasp the essence of the paper in a short period of time, improve their depth of understanding and reading efficiency. The system has high feasibility and excellent performance, and is suitable for paper analysis tasks in daily academic research.

[0241] This embodiment integrates document parsing and processing with large-scale specialized analysis technologies to construct an efficient and intelligent paper reading and recommendation system, enabling efficient analysis of academic literature and intelligent summary generation. Under existing technologies, users need to spend a considerable amount of time understanding the core ideas of a paper, facing the challenge of delayed access to research results. Especially in rapidly iterating disciplines such as artificial intelligence, users often struggle to access the latest research results in a timely manner, hindering their research progress and even missing out on cutting-edge developments in their field. This system utilizes specialized analysis methods to rapidly extract key elements of a paper, including its core points, research methods, innovative contributions, and conclusions. It also pioneers cross-modal correlation analysis between graphical data and textual content. By intelligently filtering redundant information and complex details, the system generates structured, customized summary templates, allowing the essence of papers from different disciplines to be presented concisely and clearly. This technology not only significantly reduces the user's literature reading time by over 80%, compressing traditional hours of intensive reading to minutes, but also, through visual information integration and academic value enhancement, helps researchers accurately grasp the innovative context and academic contributions of a paper, achieving simultaneous improvements in both in-depth understanding efficiency and knowledge acquisition quality, truly revolutionizing academic research efficiency.

[0242] The above is a schematic description of the present invention and its embodiments, which is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs a structure and embodiment similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. An intelligent paper parsing system based on multimodal content extraction and a large language model, featuring: It includes a file upload module, a PDF and LaTeX parsing module, a user interaction module, a paper analysis module, and a data storage management module connected in sequence; File upload module, used to upload academic paper files; The PDF and LaTeX file parsing module is used to process academic papers uploaded by users. It is responsible for file parsing, information extraction and formatting, and provides structured data support for the subsequent paper analysis module. Use OCR image recognition technology to split and extract text and pictures, and obtain mixed multimodal information of text, formulas, and charts; The large language model sorts and processes mixed multimodal information, parses content in various formats, and saves the parsing results in a structured form; User interaction module, used for interaction between the system and users to quickly obtain required academic information and services; Users can select functions based on their current needs and the actual situation of paper processing. The selected functions will be selected into different working paths in the workflow of the next paper analysis module, meeting user needs while saving costs. The paper analysis module is used to extract key information and generate structured summaries. It is based on a pre-trained large language model and combines natural language processing technology to achieve automatic parsing and summarization of paper content; The data storage management module is used to manage and maintain various types of data in the system, including academic paper content, user behavior data, recommendation data, images, and document files.

2. The intelligent paper parsing system based on multimodal content extraction and large language model according to claim 1 is characterized by: The processing of the file upload module specifically includes the following steps: Step 1.1, the module splits the file into encoded form to stream the file data; Step 1.2: The stream-encoded data is received by the back-end interface and decoded to restore the source file; Step 1.3: Extract the file information and save it as a json data document, including the file name suffix to determine whether the file type is PDF or LaTeX.

3. The intelligent paper parsing system based on multimodal content extraction and large language model according to claim 2 is characterized by: The PDF and LaTeX parsing module includes a file receiving and type identification module, a PDF file processing module, a LaTeX file processing module, and a data processing and cleaning module; The file receiving and type identification module is used to receive uploaded academic paper files, decode and identify the type of the received files, and determine whether the file type is PDF or LaTeX; The PDF file processing module first performs basic analysis, then optimizes the paper format, and finally saves it in a structured manner; The LaTeX file processing module first compiles the LaTeX file, then extracts the text information, processes the charts, and finally integrates and saves the data; The data processing and cleaning module is used to clean and process data, including removing irrelevant content, character cleaning, and data verification; the cleaned data includes JSON files, image files, and auxiliary files.

4. The intelligent paper parsing system based on multimodal content extraction and large language model according to claim 3 is characterized by: The processing of the PDF and LaTeX parsing module specifically includes the following steps: Step 2.1: For PDF data, use the file splitting method to split the PDF into multiple image formats; Step 2.2: Perform OCR and large language model analysis on the image-formatted paper, performing steps 2.2.1 to 2.2.4 in sequence. Step 2.2.

1. Use OCR to recognize all text data into an extra-long text format variable. Step 2.2.2: Use regular expressions to decompose the title according to the special format characteristics of the paper, and store the decomposed title as text; Step 2.2.3: Use the large language model technology to filter the title results, so that the large language model can understand and decompose the first-level titles according to the common style of the paper; Step 2.2.4: Use regular expressions to extract the text of the first-level titles from the long text format variable. The final format is a JSON dictionary format, with the first-level title as the key and the corresponding content as the value; Step 2.3: Decompress and split the LaTeX format document; obtain the text from the file with the suffix tex, and execute steps 2.2.2 to 2.2.4 at the same time; Step 2.4: For PDF data, extract images from the paper in image format; for LaTeX data, obtain images from the folder named fig; Step 2.5: Clean and process the data, including removing irrelevant content, cleaning characters, and validating data. The cleaned data includes JSON files, image files, and auxiliary files.

5. The intelligent paper parsing system based on multimodal content extraction and large language model according to claim 4 is characterized by: The user interaction modules include the original text and parsed text comparison reading module, paper search module, historical paper browsing module, important journal paper recommendation module and important reference recommendation module; A module for comparing the original text and the parsed text, which displays the original text and the parsed content side by side; Search paper module, which allows you to quickly search for relevant literature by entering the paper title, author or DOI; The historical browsing paper module is used to automatically record the user's browsing history. Users can view the papers they have browsed in the past through the history module; Important journal paper recommendation module, used to recommend the latest research papers to users based on important journal data; The important reference recommendation module is used to recommend important references to users based on the analysis of the paper content and its references.

6. The intelligent paper parsing system based on multimodal content extraction and large language model according to claim 5 is characterized by: The important reference recommendation module recommends references to users based on their citation scores. The system comprehensively evaluates the citation score of each reference based on the citation frequency of each paper, cross-section citations, and special bonus points for specific sections. The specific calculation method is as follows: Steps 3.5.1 to 3.5.5: Step 3.5.

1. Count the number of citations to the references in each chapter. The system counts the number of citations to each reference in each chapter. Each time a reference appears in a chapter, the counter is incremented by 1, counting the number of citations within that chapter. Step 3.5.

2. Calculate the citation score for each section. For each reference, the frequency of its occurrence in the section is substituted into the following formula to calculate the citation score: score = 2 * (1 - 0.5^frequency), where frequency represents the number of times the reference is cited in the section. This formula is designed to scale nonlinearly, meaning that as the citation frequency increases, the increase in the citation score decreases. Step 3.5.3: Summarize the total score of each reference. After calculating the citation score of each reference in each chapter, the system will add up the scores of each chapter according to the reference number to obtain the total score of each reference. Step 3.5.4: Consider cross-section citations of references. To account for cross-section citations, the total score of each reference is adjusted based on the number of cross-section citations. Specifically, for each reference, add points based on the number of chapters in which it appears, and add half the number of chapters for each chapter in which it appears. The formula is as follows: forrefin ref_total_score: ref_total_score[ref]+=0.5*ref_count[ref], where ref_count[ref] represents the number of chapters in which the reference appears; Step 3.5.5, special chapter bonus: For specific chapters in the paper, the cited references will receive additional bonus points; the system will add an additional 1 point to the score of the references cited in these chapters.

7. The intelligent paper parsing system based on multimodal content extraction and large language model according to claim 6 is characterized by: The paper analysis module includes a professional analysis module, a core point rapid extraction and analysis module, and a multimodal information fusion module; A specialized analysis module, which is used to automatically identify the field of the input paper and generate an analysis template; The core point rapid extraction and analysis module uses a large language model to analyze the structure of the paper, automatically extract the key points of each section, and generate a summary. The system uses multi-threaded synchronous processing to improve analysis efficiency. Multimodal information fusion module, used to fuse chart and text information.

8. The intelligent paper parsing system based on multimodal content extraction and large language model according to claim 7 is characterized by: The paper analysis module processing process specifically includes the following steps: Step 4.1, specialized analysis, is used to automatically identify the field of the input paper based on its content and generate an analysis template. Specifically, this part performs steps 4.1.1 to 4.1.3 in sequence: Step 4.1.1: Field Identification and Template Matching: The system first identifies the subject area of the paper based on the paper's title, keywords, or other metadata. Based on the characteristics of each field, the system designs prompt templates for each field to ensure the professionalism of the analysis results. Step 4.1.2: Template generation and matching: After the user uploads the paper, the system automatically extracts the paper's structural information and selects the most appropriate template using fuzzy query technology. The template generates corresponding analysis prompts based on the different parts of the paper, ensuring that each part of the paper receives appropriate analysis and processing. Step 4.1.3: Content insertion and analysis. Once a suitable template is matched, the system will insert the paper content into a predefined format according to the requirements of the template and call the large language model for analysis. The large language model will perform customized analysis based on the selected template, extracting core information in the professional field to ensure the depth and accuracy of the results. Step 4.2: Rapid extraction and analysis of key points. This system uses a large language model to structurally analyze the paper, automatically extract the key points of each section, and generate a summary. The system uses multi-threaded synchronous processing to improve analysis efficiency. Step 4.3: Multimodal information fusion module is used to fuse chart and text information.

9. The intelligent paper parsing system based on multimodal content extraction and large language model according to claim 8 is characterized by: The data storage management module includes a database module and a data real-time update and synchronization module; Databases include relational databases and non-relational databases; Data real-time update and synchronization module, used for real-time data update and synchronization; Relational databases are used to store structured data, including: Paper data: metadata, abstract, main text, introduction, and conclusion of the paper; User data: user's personal information, login records, browsing history, and query records; Recommendation data: recommendation records, user click-through reading behavior, and historical preferences; Non-relational databases are used to store unstructured data.

10. The intelligent paper parsing system based on multimodal content extraction and large language model according to claim 9 is characterized by: The implementation of real-time data update and synchronization is as follows: Steps 5.2.1 to 5.2.3: Step 5.2.1, Data synchronization mechanism. The system adopts a distributed database management and interactive architecture, and ensures real-time updates of various types of data between different databases through the data synchronization mechanism; each type of database operates independently according to its own data processing requirements, and the system ensures data synchronization between them; Step 5.2.2, MD5 value as a unique identifier. The system uses MD5 value as a unique identifier for files and data; each time data or files are uploaded, the system calculates its MD5 value and performs data verification and deduplication based on the MD5 value; Step 5.2.3: Regular data cleaning and deduplication. At regular intervals, the system scans the database for redundant and expired data and automatically deletes or archives them according to the set rules.

Images