Knowledge base document processing method and system based on AI application

Abstract

The invention provides a knowledge base document processing method and system based on an AI application, and relates to the technical field of big data, and the method comprises the steps: S1, carrying out format analysis and multi-mode conversion on an uploaded to-be-processed document to obtain standard text content and text description of a picture contained in the to-be-processed document; s2, performing semantic analysis on the standard text content based on a pre-trained large model to obtain a corresponding context semantic vector, and capturing a long sequence dependency relationship based on the context semantic vector to obtain a dependency feature set; and S3, segmenting the standard text content according to a pre-acquired segmentation length threshold value and the dependency feature set to obtain segmented text files, adding the text description to each picture to obtain picture files, and outputting the text files and the picture files as document processing results. The method has the beneficial effects that the processing efficiency and scale are remarkably improved, and the semantic understanding and extraction precision is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of big data technology, and in particular to a knowledge base document processing method and system based on AI applications. Background Technology

[0002] With the rapid development of artificial intelligence technology and the deep advancement of the digital economy, knowledge bases, as core data carriers for enterprise decision support, academic research innovation, and public service optimization, are expanding exponentially. Knowledge base document processing technology, as a key link in realizing data value mining, directly determines the utilization efficiency and service capabilities of knowledge bases. Its core requirements focus on achieving efficient processing of large-scale documents, accurate semantic understanding, and multimodal information integration. However, existing technological systems are struggling to match the current complex and massive document processing scenarios.

[0003] Currently, existing technologies in the field of knowledge base document processing can be mainly divided into three categories: 1. Traditional Manual Processing Model: This model relies on personnel with specialized domain knowledge to manually classify, archive, extract core information, verify content, and maintain and update documents. It was the earliest technology applied to knowledge base processing. The core advantage of this model lies in the deep understanding of specialized semantics by human personnel. When processing highly specialized documents such as legal provisions, medical diagnostic reports, and aerospace technical manuals, the domain experience of these personnel can help avoid ambiguous interpretations and ensure the accuracy of information extraction. However, this model has extremely low processing efficiency. Limited by human energy and focus, the average number of documents processed per person per day is typically only 20-50, and this number decreases further as document complexity increases. Furthermore, it has poor scalability; when facing large-scale knowledge base update needs with hundreds of thousands or more documents, the processing cycle often lasts for months, severely lagging behind the timeliness requirements of business decisions.

[0004] 2. Rule-based Automated Processing Systems: These systems utilize pre-defined rules such as keyword matching, format template recognition, and field location to automatically extract and categorize key information from documents. For example, a company's knowledge base management system can automatically extract and input information from standardized product manuals into a database by pre-setting tag rules such as "Technical Parameters:" and "Scope of Application:". While such systems can process 200-300 highly standardized structured documents per day, they are extremely incompatible with unstructured documents. For example, the coverage of pre-defined rules is less than 30% for unstructured documents such as scanned handwritten notes, text contained in screenshots, and mixed-format research reports, making information extraction ineffective.

[0005] 3. Basic AI-Assisted Processing Technology: Its core improvement over rule-based systems lies in introducing a single machine learning model into the processing. Examples include using a Naive Bayes classifier for document topic classification, an LSTM (Long Short-Term Memory) model for text summarization, and an SVM (Support Vector Machine) model for keyword recognition. This type of technology breaks through the limitations of fixed rules, enabling the learning of document features through data training. For instance, a document management platform uses an LSTM model to generate abstracts for academic papers and an SVM model for topic classification. However, current basic AI technologies have limited task scenarios; most systems can only perform single tasks such as classification and summarization, and their generalization ability is weak. Due to the use of a single model architecture and training data often limited to specific domains, when facing cross-domain document processing needs (such as simultaneously processing mechanical engineering drawings and biomedical experimental reports), the classification accuracy is generally below 65%, and the information completeness of the extracted summary is less than 50%, still requiring significant manual correction.

[0006] In summary, the existing technology system suffers from two major problems: First, the processing efficiency is mismatched with the needs of large-scale operations. Traditional manual processing is completely unable to meet the rapid update requirements of large-scale knowledge bases. When faced with massive amounts of unstructured documents, the processing efficiency of rule-based systems and rudimentary AI technologies drops sharply to less than 100 documents per day, making it difficult to support the operation of knowledge bases with hundreds of thousands or millions of documents. Second, the accuracy of semantic understanding and information extraction is low. Rule-based systems rely only on surface feature matching and cannot understand the deep semantics of documents. Rudimentary AI technologies, due to their simple model architecture and weak generalization ability, are insufficient in processing professional terminology, cross-domain contexts, and implicit information, resulting in high error rates and omission rates in the extraction of key information.

[0007] A thorough analysis of the root causes of the problem reveals two main aspects: First, the technical architecture has inherent limitations. Rule-based systems rely on predefined logic, inherently unable to overcome the limitations of rule coverage, and lack adaptability in complex scenarios. Basic AI technologies employ a single model architecture, failing to integrate core technologies such as multimodal semantic understanding and long-sequence semantic association capture, making it difficult to handle multi-format, cross-domain, and semantically complex document content. Second, data processing capabilities are insufficient. Existing technologies have weak preprocessing capabilities for unstructured data, lacking effective algorithms for format parsing, multimodal conversion, text cleaning, and deep feature extraction. This results in a large number of mixed-format documents containing text and images being unable to be effectively parsed and utilized, limiting the scale of processing and further impacting the accuracy of information extraction due to incomplete data utilization.

[0008] Therefore, developing a knowledge base document processing technology that can achieve multi-format document parsing, deep semantic understanding, and efficient and accurate processing has become an urgent need in the current field of AI applications. Summary of the Invention

[0009] To address the problems existing in the prior art, this invention provides a knowledge base document processing method based on AI applications, comprising: Step S1: Perform format parsing and multimodal conversion on the uploaded document to be processed to obtain the standard text content and text descriptions of the images contained in the document to be processed. Step S2: Based on the pre-trained large model, perform semantic analysis on the standard text content to obtain the corresponding context semantic vector, and capture long sequence dependencies based on the context semantic vector to obtain a dependency feature set; Step S3: The standard text content is segmented according to the pre-acquired segment length threshold and the dependent feature set to obtain segmented text files, and the text descriptions are added to each of the images to obtain image files. The text files and the image files are then output as document processing results.

[0010] Preferably, step S1 includes: Step S11: Perform format parsing on the document to be processed, and extract the text content, table content and images from it when the format parsing result indicates that it contains PDF or Word documents, and convert it into editable text and extract the text content, table content and images from it when the format parsing result indicates that it contains scanned documents. Step S12: Perform text cleaning on the text and the table content to obtain the standard text content; Step S13: Convert each of the images into the corresponding text descriptions.

[0011] Preferably, the pre-trained large model is a BERT-BiLSTM fusion model. In step S2, the standard text content is semantically analyzed by the pre-trained BERT model to obtain the context semantic vector, and then each context semantic vector is input into the BiLSTM model for processing to obtain the dependent feature set.

[0012] Preferably, the segment length threshold is the maximum context length of the embedded model used by the knowledge base.

[0013] Preferably, after performing step S1, a knowledge fusion process is further included, comprising: Step A1: Use a probabilistic graphical model to extract key information from the standard text content to obtain key entities and the relationships between entities; Step A2: Entity matching is performed based on the key entities and the relationships between entities and the pre-stored knowledge graph. When the entity matching result indicates a mismatch, the corresponding key entities and the relationships between entities are stored in the knowledge graph for knowledge fusion and updating.

[0014] Preferably, before performing step A2, the method further includes displaying the extracted key entities and the relationships between entities through a human-computer interaction interface; The human-computer interaction interface also provides a correction window, allowing users to correct the key entities and the relationships between them.

[0015] Preferably, the method further includes continuously recording the corrected key entities and the relationships between entities and marking them as training samples, and triggering fine-tuning of the probabilistic graphical model when the number of training samples reaches a preset threshold.

[0016] Preferably, before performing step S2, a transfer learning process is further included, comprising: Step B1: Classify the standard text content into document categories to obtain corresponding document category tags; Step B2: When the document category label is not in the pre-stored category label set, fine-tune the pre-trained large model.

[0017] Preferably, in step S3, after segmenting the standard text content, the method further includes adding a preset segmentation identifier after each segment to obtain the segmented text file, and adding the segmentation identifier after each image to obtain the image file.

[0018] This invention also provides a knowledge base document processing system based on AI applications, which applies the above-described knowledge base document processing method. The knowledge base document processing system includes: The multimodal data processing module is used to perform format parsing and multimodal conversion on the uploaded document to be processed to obtain the standard text content and text descriptions of the images contained in the document to be processed. The semantic understanding module, connected to the multimodal data processing module, is used to perform semantic analysis on the standard text content based on a pre-trained large model to obtain the corresponding context semantic vector, and capture long sequence dependencies based on the context semantic vector to obtain a dependency feature set. The content organization module is connected to the multimodal data processing module and the semantic understanding module, respectively. It is used to segment the standard text content according to the pre-acquired segment length threshold and the dependency feature set to obtain segmented text files, and to add the text description to each of the images to obtain image files. The text files and the image files are then output as document processing results.

[0019] The above technical solution has the following advantages or beneficial effects: 1) Significantly improved processing efficiency and scale: Through format parsing and multimodal conversion, it can automatically adapt to multiple document formats without manual intervention. This completely solves the compatibility problem of traditional manual one-by-one adaptation and rule-based systems that can only handle structured documents. In addition, combined with the parallel computing capabilities of pre-trained large models, it can simultaneously process the parsing and semantic analysis tasks of multiple documents, greatly shortening the processing cycle of a single document. According to actual tests, on a server equipped with an Intel Xeon E5-2690 v4 processor and 128GB of memory, the average daily document processing volume reaches 1200 documents, which is 24 times higher than traditional manual processing and 4 times higher than rule-based systems. It can meet the monthly update needs of a knowledge base with millions of documents. 2) Significantly improved semantic understanding and extraction accuracy: Based on a pre-trained large model, semantic analysis of standard text content is performed and long-sequence dependencies are captured, solving the problem of handling semantic fragmentation in long documents by traditional models. The resulting dependency feature set can fully reflect the semantic association structure of the document. In actual testing, among 10,000 test documents covering mechanical engineering, biomedicine, and law, the key information extraction accuracy reached 92.3%, and the omission rate was 4.1%, which is 27.3 percentage points higher and 30.9 percentage points lower than that of basic AI technology (65% accuracy and 35% omission rate), respectively. Attached Figure Description

[0020] Figure 1 A flowchart illustrating a knowledge base document processing method based on AI applications is provided in a preferred embodiment of the present invention. Figure 2 This is a schematic diagram of a sub-process of step S1 in a preferred embodiment of the present invention; Figure 3 A flowchart illustrating the knowledge fusion process is shown in a preferred embodiment of the present invention. Figure 4 A flowchart illustrating the transfer learning process is provided in a preferred embodiment of the present invention. Figure 5 This is a schematic diagram of the structure of a knowledge base document processing system based on AI applications, which is a preferred embodiment of the present invention. Detailed Implementation

[0021] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment; other embodiments that conform to the spirit of the present invention may also fall within the scope of the present invention.

[0022] In a preferred embodiment of the present invention, based on the above-mentioned problems existing in the prior art, a knowledge base document processing method based on AI applications is provided, such as... Figure 1 As shown, it includes: Step S1: Perform format parsing and multimodal conversion on the uploaded document to be processed to obtain the standard text content and text descriptions of the images contained in the document to be processed. Step S2: Based on the pre-trained large model, perform semantic analysis on the standard text content to obtain the corresponding context semantic vector, and capture long sequence dependencies based on the context semantic vector to obtain a dependency feature set; Step S3: Segment the standard text content according to the pre-acquired segment length threshold and dependent feature set to obtain segmented text files, and add text descriptions to each image to obtain image files, and output the text files and image files as document processing results.

[0023] Specifically, addressing the common problems of limited format compatibility and low utilization of unstructured data in existing technologies, this embodiment first performs format parsing and multimodal conversion on the document to be processed to achieve data standardization. For example, Figure 2 As shown, step S1 includes: Step S11: Perform format parsing on the document to be processed, and extract the text content, table content and images from it when the format parsing result indicates that it contains PDF or Word documents, and convert it into editable text and extract the text content, table content and images from it when the format parsing result indicates that it contains scanned documents. Step S12: Clean the text and table content to obtain standard text content; Step S13: Convert each image into a corresponding text description.

[0024] Specifically, in this embodiment, for structured documents such as PDF and Word, text, tables, and images can be directly extracted through format parsing, avoiding the defect of traditional rule systems that miss table information; for unstructured documents such as scanned documents, the Tesseract OCR algorithm is preferred to achieve editable text conversion. Tests have shown that in the processing of scanned maintenance records in the mechanical manufacturing field, the text conversion accuracy reaches 98%, which is 13 percentage points higher than that of traditional OCR tools (average accuracy of 85%), completely solving the accuracy problem of digitizing paper documents.

[0025] For image information such as product structure diagrams and assembly schematics contained in the document, the multimodal conversion process preferably uses a CNN algorithm to construct a mapping relationship between visual features and semantic descriptions, generating structured text descriptions such as "[Part A]-[Connection Method]-[Part B]", transforming image information from unsearchable to associative data. This approach, which ignores image information processing methods used in more basic AI technologies, effectively improves the utilization rate of document information. Finally, after text cleaning (regular expression denoising + jieba word segmentation), the output standard text content lays a high-quality data foundation for subsequent semantic analysis.

[0026] After standardization, in order to address the problems of insufficient semantic understanding and weak ability to capture long sequence dependencies in existing technologies, resulting in low accuracy of key information extraction, this invention specifies that the pre-trained large model is a BERT-BiLSTM fusion model. In step S2, the pre-trained BERT model is used to perform semantic analysis on the standard text content to obtain context semantic vectors, and then each context semantic vector is input into the BiLSTM model for processing to obtain a dependency feature set.

[0027] Among them, the pre-trained BERT model has a strong ability to capture professional semantics, while the BiLSTM model, through a bidirectional propagation structure, traverses the vector sequence in both forward (from front to back) and backward (from back to front) directions, effectively capturing chain dependencies such as "technical parameters-applicable conditions-maintenance requirements" in long documents. The resulting dependency feature set can fully reflect the semantic logic of the document, solving the problem of forward dependency loss in traditional LSTM models.

[0028] Furthermore, the standard text content is segmented according to the pre-acquired segment length threshold and dependent feature set to obtain segmented text files, and text descriptions are added to each image to obtain image files. The text files and image files are then output as document processing results.

[0029] The segment length threshold is not a fixed value, but rather the maximum context length of the embedded model used by the knowledge base. This maximum context length can be obtained by consulting the official documentation of the embedded model or calling the API (e.g., the maximum context length of the commonly used Sentence-BERT model is 512 characters). This ensures that the segmented text files can directly adapt to the knowledge base embedding requirements, avoiding semantic breaks caused by traditional fixed segmentation. Simultaneously, a preset identifier "—chunk—" is added after the segmented text, and this identifier is also added after the image text description, allowing the knowledge base import system to automatically identify the segmentation nodes, effectively improving import efficiency.

[0030] The final output text file (including segments and identifiers) and image file (Word format, including images and corresponding text descriptions) are provided as download links through a web interface. Users can directly create a knowledge base after downloading without any secondary processing, effectively saving knowledge base construction time.

[0031] Existing technologies lack effective knowledge fusion mechanisms, leading to a disconnect between newly extracted information and the knowledge base. To address this technical problem, in a preferred embodiment of the present invention, after performing step S1, as follows... Figure 3 As shown, it also includes a knowledge fusion process, including: Step A1: Use a probabilistic graphical model to extract key information from the standard text content to obtain key entities and the relationships between entities; Step A2 involves matching key entities and their relationships with a pre-stored knowledge graph. If the matching result indicates a mismatch, the corresponding key entities and their relationships are stored in the knowledge graph for knowledge fusion and updating.

[0032] The existing system does not have a lightweight manual intervention entry point, and the manual correction process is cumbersome; moreover, it does not build a closed-loop mechanism of "manual correction - data annotation - model retraining", which means that human-computer collaboration is only at the level of "manual error correction" and cannot achieve collaborative optimization. Based on this, in the preferred embodiment of the present invention, before executing step A2, the extracted key entities and the relationships between entities are displayed through a human-computer interaction interface. The human-computer interaction interface also provides a correction window, allowing users to correct key entities and the relationships between them.

[0033] In a preferred embodiment of the present invention, the method further includes continuously recording the corrected key entities and the relationships between entities and marking them as training samples, and triggering fine-tuning of the probabilistic graphical model when the number of training samples reaches a preset threshold.

[0034] Specifically, in this embodiment, by providing a human-interactive interface, the time for manually correcting a single document is reduced from 15 minutes to 2 minutes, improving correction efficiency by 75%. Furthermore, after using the corrected data for model fine-tuning, the model's processing accuracy in similar documents can be further improved by 3-5 percentage points, forming a continuous optimization loop.

[0035] In a preferred embodiment of the present invention, before performing step S2, as follows: Figure 4 As shown, it also includes a transfer learning process, including: Step B1: Classify the standard text content into document categories to obtain the corresponding document category labels; Step B2 involves fine-tuning the pre-trained large model when the document category label is not in the pre-stored category label set.

[0036] Specifically, in this embodiment, through transfer learning, when adding a new energy field, only 500 samples are needed for fine-tuning in 36 hours to achieve adaptation, shortening the system iteration cycle to 10 days, which is more than 70% shorter than the existing technology (2-3 months); the dynamic update engine can automatically update the knowledge base every 24 hours to ensure the timeliness of information.

[0037] In a preferred embodiment of the present invention, step S3 further includes adding a preset segmentation identifier after each segment to obtain a separate text file, and adding a separate identifier after each image to obtain an image file.

[0038] In summary, through a comprehensive design encompassing multimodal preprocessing, deep semantic analysis, dynamic knowledge fusion, and human-machine collaborative optimization, this method thoroughly addresses the core pain points of existing technologies in terms of processing efficiency, semantic accuracy, and cross-domain adaptability. This approach not only clarifies standardized processing steps but also achieves efficient end-to-end deployment from "document input" to "knowledge base availability" through system-level hardware and software deployment and iterative mechanism design. To ensure the efficient implementation of the above methods, this invention designs a matching hardware and software deployment scheme and a standardized operating procedure, specifically including: 1. Hardware and software environment deployment – ​​supporting high concurrency and efficient model operation Hardware environment: The deployment server uses an Intel Xeon E5-2690 v4 processor, 128GB DDR4 memory, and a 2TB SSD hard drive; the client uses a regular PC (i5 processor, 8GB memory); and is equipped with a high-definition scanner (for scanning paper documents).

[0039] Software environment: The server is equipped with Ubuntu 20.04, Python 3.8, TensorFlow 2.5 deep learning framework, and MySQL 8.0 database; the client is equipped with Chrome browser and accesses the system through a web interface.

[0040] Model training data: A total of 5,000 documents, including product manuals, technical manuals, and maintenance records, were collected from the mechanical manufacturing field. 4,000 documents were used as the training set and 1,000 documents were used as the test set. LabelStudio was used to annotate entities (such as "part names" and "technical parameters") and relationships in the training set.

[0041] 2. Overall process performance: Taking the machinery manufacturing industry as an example Based on the above deployment, using 1000 mechanical manufacturing documents (including 300 PDF manuals, 200 Word handbooks, 300 scanned repair records, and 200 TXT experimental data files) as the processing objects, the system operation flow and results are as follows: Step 1: Multimodal Data Acquisition and Preprocessing: 1000 documents were uploaded in batches via a web interface, and the format was automatically parsed to obtain the text and table content of PDF and Word documents; the scanned documents were converted into editable text using the Tesseract OCR algorithm with an accuracy of 98%; for multimodal conversion, the CNN algorithm was preferred to convert the product structure diagram in the documents into a text description of "[Part A]-[Connection Method]-[Part B]"; text cleaning was performed by removing special characters using regular expressions and segmenting the text using jieba to generate standardized text data.

[0042] Step 2: AI Deep Analysis: The pre-processed text data is input into a pre-trained large model. First, the pre-trained BERT model is used to generate contextual semantic vectors for semantic understanding. The BiLSTM model is then used to capture long-sequence dependencies of "technical parameters-applicable conditions". Next, the CRF algorithm is used to extract key information and identify entities such as "part name", "rated power", and "service life" to extract the relationships between entities. Finally, the fine-tuned BERT model is combined with an attention mechanism to classify the document into four categories: "product description", "technical standard", "maintenance guide" and "experimental data".

[0043] Step 3: Dynamic Knowledge Fusion: The extracted "part names" are matched with entities in the existing knowledge graph using entity alignment algorithms (such as string matching based on edit distance), and new technical parameter information is fused. For the newly added "new energy motor" category documents, transfer learning is used to fine-tune the pre-trained model using 500 samples to achieve cross-domain adaptation. The fused knowledge is stored in the MySQL database. The update log shows that this update added 800 entities and 1200 relationships.

[0044] Step 4: Human-Machine Collaborative Optimization. The AI ​​processing results are displayed to the user through a web interface: The user browses documents on the client and finds that the "maintenance cycle" parameter extraction is incorrect in 10 documents. The parameter value is corrected by using the "information completion" button in the left toolbar. The correction operation takes an average of 1.5 minutes per document. At the same time, the corrected data is automatically labeled as training samples. After accumulating 500 corrected data, the model is fine-tuned, and the fine-tuning process takes 8 hours.

[0045] Step 5: Application Output. The knowledge base storage module provides data to the application output module, enabling services such as document retrieval (users can input "gearbox maintenance cycle" to accurately return relevant documents and key parameters) and intelligent recommendation (recommending related maintenance guides to maintenance personnel).

[0046] This invention also provides a knowledge base document processing system based on AI applications, which applies the above-described knowledge base document processing method, such as... Figure 5 As shown, the knowledge base document processing system includes: Multimodal data processing module 1 is used to parse the uploaded document to be processed in terms of format and perform multimodal conversion to obtain the standard text content and text descriptions of the images contained in the document to be processed. Semantic understanding module 2, connected to multimodal data processing module 1, is used to perform semantic analysis on standard text content based on a pre-trained large model to obtain the corresponding context semantic vector, and capture long sequence dependencies based on the context semantic vector to obtain a dependency feature set; The content organization module 3 is connected to the multimodal data processing module 1 and the semantic understanding module 2, respectively. It is used to segment the standard text content according to the pre-acquired segment length threshold and the dependency feature set to obtain segmented text files, and to add text descriptions to each image to obtain image files. The text files and image files are output as document processing results.

[0047] The above description is merely a preferred embodiment of the present invention and does not limit the implementation and protection scope of the present invention. Those skilled in the art should realize that any equivalent substitutions and obvious changes made using the content of this specification and illustrations should be included within the protection scope of the present invention.

Claims

1. A knowledge base document processing method based on AI applications, characterized in that, include: Step S1: Perform format parsing and multimodal conversion on the uploaded document to be processed to obtain the standard text content and text descriptions of the images contained in the document to be processed. Step S2: Based on the pre-trained large model, perform semantic analysis on the standard text content to obtain the corresponding context semantic vector, and capture long sequence dependencies based on the context semantic vector to obtain a dependency feature set; Step S3: The standard text content is segmented according to the pre-acquired segment length threshold and the dependent feature set to obtain segmented text files, and the text descriptions are added to each of the images to obtain image files. The text files and the image files are then output as document processing results.

2. The knowledge base document processing method according to claim 1, characterized in that, Step S1 includes: Step S11: Perform format parsing on the document to be processed, and extract the text content, table content and images from it when the format parsing result indicates that it contains PDF or Word documents, and convert it into editable text and extract the text content, table content and images from it when the format parsing result indicates that it contains scanned documents. Step S12: Perform text cleaning on the text and the table content to obtain the standard text content; Step S13: Convert each of the images into the corresponding text descriptions.

3. The knowledge base document processing method according to claim 1, characterized in that, The pre-trained large model is a BERT-BiLSTM fusion model. In step S2, the standard text content is semantically analyzed by the pre-trained BERT model to obtain the context semantic vector, and then each context semantic vector is input into the BiLSTM model for processing to obtain the dependent feature set.

4. The knowledge base document processing method according to claim 1, characterized in that, The segment length threshold is the maximum context length of the embedded model used by the knowledge base.

5. The knowledge base document processing method according to claim 1, characterized in that, After performing step S1, a knowledge fusion process is also included, comprising: Step A1: Use a probabilistic graphical model to extract key information from the standard text content to obtain key entities and the relationships between entities; Step A2: Entity matching is performed based on the key entities and the relationships between entities and the pre-stored knowledge graph. When the entity matching result indicates a mismatch, the corresponding key entities and the relationships between entities are stored in the knowledge graph for knowledge fusion and updating.

6. The knowledge base document processing method according to claim 5, characterized in that, Before performing step A2, the process also includes displaying the extracted key entities and the relationships between them through a human-computer interaction interface. The human-computer interaction interface also provides a correction window, allowing users to correct the key entities and the relationships between them.

7. The knowledge base document processing method according to claim 6, characterized in that, It also includes continuously recording the corrected key entities and the relationships between entities and marking them as training samples, and triggering fine-tuning of the probabilistic graphical model when the number of training samples reaches a preset threshold.

8. The knowledge base document processing method according to claim 1, characterized in that, Before performing step S2, a transfer learning process is also included, comprising: Step B1: Classify the standard text content into document categories to obtain corresponding document category tags; Step B2: When the document category label is not in the pre-stored category label set, fine-tune the pre-trained large model.

9. The knowledge base document processing method according to claim 1, characterized in that, In step S3, after segmenting the standard text content, the method further includes adding a preset segmentation identifier after each segment to obtain the segmented text file, and adding the segmentation identifier after each image to obtain the image file.

10. A knowledge base document processing system based on AI applications, characterized in that, The knowledge base document processing system, which employs the knowledge base document processing method as described in any one of claims 1-9, comprises: The multimodal data processing module is used to parse the uploaded document to be processed in terms of format and perform multimodal conversion to obtain the standard text content and text descriptions of the images contained in the document to be processed. The semantic understanding module, connected to the multimodal data processing module, is used to perform semantic analysis on the standard text content based on a pre-trained large model to obtain the corresponding context semantic vector, and capture long sequence dependencies based on the context semantic vector to obtain a dependency feature set. The content organization module is connected to the multimodal data processing module and the semantic understanding module, respectively. It is used to segment the standard text content according to the pre-acquired segment length threshold and the dependency feature set to obtain segmented text files, and to add the text description to each of the images to obtain image files. The text files and the image files are then output as document processing results.

Images