Graph retrieval enhancement generation method based on telecom specification multi-mode knowledge graph

Abstract

The invention discloses a graph retrieval enhancement generation method based on a telecom specification multi-modal knowledge graph, and belongs to the technical field of multi-modal knowledge graphs. The method comprises the following steps of: firstly, analyzing titles, texts, pictures and tables in a PDF (Portable Document Format) by virtue of layout analysis, OCR (Optical Character Recognition) and LLM tools through a telecom standard analysis agent; calling a telecom specification multi-modal extraction agent, and extracting chart names and description information from pictures and tables by using a multi-modal large model; the method comprises the following steps: constructing a title hierarchical structure and extracting a triple from a text block, and constructing a multi-modal, multi-hierarchy and multi-granularity telecommunication specification knowledge graph comprising a title, the text block, a picture, a table and the triple; and finally, calling a telecommunication specification multi-granularity retrieval agent, and deeply combining the knowledge graph with the large model, thereby solving the problem that the traditional RAG cannot perform multi-hop reasoning and relieving the illusion problem possibly occurring in the question and answer process of the large model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-modal knowledge graph, and particularly relates to a multi-modal knowledge graph based on a telecommunication specification and a graph retrieval enhancement generation method. BACKGROUND

[0002] A multi-agent system (MAS) is a distributed system composed of multiple autonomous or semi-autonomous agents. Each agent can independently perceive the environment, learn, make decisions, and perform tasks, while collaborating with each other through effective communication mechanisms to achieve common goals or individual tasks. Compared with single-agent systems, multi-agent systems have higher flexibility, robustness, and scalability, and can efficiently complete comprehensive tasks that single models or algorithms cannot cover. In complex environments, MAS can dynamically adjust strategies and behaviors to adapt to changing needs and conditions.

[0003] Traditional knowledge graph (KG) construction mainly relies on rule matching, template filling, and machine learning methods. These methods are less efficient and have limited accuracy when dealing with large-scale, heterogeneous data. In recent years, with the development of deep learning technology, knowledge graph construction methods based on neural networks have gradually emerged. These methods can better handle unstructured data and have stronger generalization ability. However, they usually require a large amount of labeled data for training, and still face challenges in handling complex semantic relationships and multi-modal data (such as text, images, videos, etc.). The above methods face challenges such as difficulty in multi-modal data analysis, complex knowledge level association, and poor quality of unsupervised knowledge extraction in knowledge graph construction. Multi-agent technology, through the division of labor and cooperation mechanism, combined with the generalization reasoning ability of large language models (LLM) and multi-modal large pre-training models (MLM), provides a new solution for automated, multi-modal, multi-granularity, and multi-level knowledge graph construction.

[0004] In MAS, each agent can use large models to enhance its own functions, such as layout analysis, text understanding, and image recognition. Specifically, in the process of knowledge graph construction, agents can automatically identify and analyze different data types from massive multi-modal files, and perform unsupervised data extraction through multiple calls to LLM. In addition, with the powerful representation ability of LLM, agents can also perform efficient graph retrieval and question answering tasks to provide accurate answers for users. This method based on MAS and LLM not only improves the automation and accuracy of knowledge graph construction, but also enhances the adaptability and response speed of the system, making it able to cope with more complex and diverse application scenarios.

[0005] To solve the problems of poor multi-modal data analysis effect, dependence on manual annotation, and incomplete multi-granularity knowledge association in the prior art, realize the construction of a multi-modal knowledge graph, and complete the generation of enhanced graph retrieval, the present application provides a method for generating enhanced graph retrieval based on a multi-modal knowledge graph of telecommunications specifications. SUMMARY

[0006] The technical problem to be solved by the present application is how to solve the problems of poor multi-modal data analysis effect, dependence on manual annotation, and incomplete multi-granularity knowledge association in the prior art, realize the construction of a multi-modal knowledge graph, and complete the generation of enhanced graph retrieval, and provide a method for generating enhanced graph retrieval based on a multi-modal knowledge graph of telecommunications specifications. Through the cooperation between agents in MAS, the image table understanding ability of MLM, and the unsupervised learning ability of LLM, efficient analysis and unsupervised knowledge extraction of multi-modal data such as text, images, and tables are realized, and accurate question answering of telecommunications specification knowledge is supported.

[0007] As shown in Figure 4 the present application solves the above technical problems by the following technical solutions. The present application comprises the following steps:

[0008] S1: Establishing a multi-agent collaborative architecture

[0009] The multi-agent collaborative architecture includes a telecommunications specification multi-level analysis agent, a telecommunications specification multi-modal extraction agent, and a telecommunications specification multi-granularity retrieval and question answering agent.

[0010] S2: Document content analysis

[0011] The telecommunications specification multi-level analysis agent is used to analyze each PDF document in the preset set of telecommunications specification documents.

[0012] S3: Knowledge graph construction

[0013] Based on the analysis results, the telecommunications specification multi-modal extraction agent is used to extract triplets to obtain a set of knowledge graphs.

[0014] S4: Generation of enhanced graph retrieval

[0015] Based on the set of knowledge graphs extracted, the telecommunications specification multi-granularity retrieval and question answering agent is used to realize industry specification question answering and complete the task of generating enhanced graph retrieval.

[0016] Further, in the step S2, the process of analysis using the telecommunications specification multi-level analysis agent is as follows:

[0017] S21: For any N-page document d i in the set of telecommunications specification documents, the jth page The page analysis model labels each region of the page with different types of tags, synchronously records the coordinate positioning information and confidence parameters of each region, and realizes page analysis;

[0018] S22: Each region is processed according to the label of the page analysis, and the processing result of each region is obtained;

[0019] S23: According to the processing result of each region, the document d i is converted into a Markdown format collection, and the parsing work is completed.

[0020] Further, in the step S21, the label types include text Text, image Image, table Table, heading Heading, and abandoned area Abandon.

[0021] Further, in the step S22, the specific processing process is as follows:

[0022] S221: Use RapidOCR to extract text T from the text region ;

[0023] S222: Convert the image region into Base64 encoding, store the image path Img path , and use the Qwen2.5-VL-3B-Instruct model to generate the image name Img name and the image description Img Caption ;

[0024] S223: Convert the table region into HTML format table content Tab content , use the Qwen2.5-7B-Instruct model to generate the table name Tab name and the table description Tab Caption ;

[0025] S224: Extract the title content H i of all title regions of the document d Hea , and optimize the title level.

[0026] Further, in the step S224, the process of optimizing the title level is as follows:

[0027] S2241: Build a title hierarchy graph:

[0028] G=(V,E)

[0029] where the node set represents the title set V = {h1, h2,..., hN} and the edge set E represents the parent-child relationship between titles. ∣n∣ i ∈ H Hea} and the edge set E represents the parent-child relationship between titles.

[0030] S2242: Detect illegal level jumps using depth-first search:

[0031] Find the root node h r with level L(r) = 1, recursively visit its child nodes, and if a child node is found with a level greater than the current node level + 1, record it as an illegal jump J:

[0032] J = {(h m , h l ) | L(h m ) > L(h l ) + 1}

[0033] where h m and h l represent the mth and lth titles in H Hea , and L(h m ) and L(h l ) represent the levels of h m and h l .

[0034] S2243: Adjust the level structure using breadth-first search:

[0035] For each illegal jump pair (h m , h l ) ∈ J, use breadth-first search to find the nearest legal intermediate level L' from the mth title h m downward layer by layer such that L' < L(h l ), modify the edges of L' to construct new parent-child relationships:

[0036] L(h l ) < L' < L(h m ).

[0037] S2244: Update the title hierarchy graph structure:

[0038] After completing the title level optimization, construct the title hierarchy triple Tti heading based on the tree structure of the titles:

[0039] Tti heading = {(h i , r, h j ) | {(h i , h j ) ∈ E ∧ L(h j ) = L(h​i )+1}

[0040] wherein, h i , h j represent two titles with a superior-inferior relationship, r represents the relationship of h i , h j .

[0041] Further, in the step S3, the telecom specification knowledge graph SCHEMA is obtained through triple extraction as follows:

[0042] S TS-KG =(E TS-KG ,R)

[0043] wherein, S TS-KG represents the telecom specification knowledge graph SCHEMA, the node set E TS-KG ={H Hea ,T chunk ,Img path ,Img name ,Img Caption ,Tab content ,Tab name ,Tab Caption ,Tri last} represents the hierarchical title, the text block, the image path, the image name, the image description, the table content, the table name, the table description and the triple entity set, the edge set R={(e1,r,e2)|{(e1,e2)∈E TS-KG} represents the relationship set between each entity, r represents the relationship between two entities e1 and e2; T chunk is obtained by performing block processing on M i , the text T and the table content Tab content under the same title, and M i is obtained by converting the document d i into the Markdown format.

[0044] Further, in the step S3, the specific process of triple extraction based on the structured prompt word engineering is implemented by using the telecom specification multi-modal extraction intelligent agent as follows:

[0045] S31: input the text block T chunk , the theme entity Topic and the prompt word Prompt1 to implement triple extraction Tri1=(h,r,t), wherein the theme entity Topic is the title corresponding to the current text block, and h, r and t represent the head entity, the relationship and the tail entity of the triple;

[0046] S32: input the text block T chunkThe triple Tri1 and the prompt word Prompt2 realize the relationship description Re des ;

[0047] S33: Input the text block T chunk , the triple Tri1, and the relationship description Re des , the candidate relationship set C R = {r1, r2... r n , 'none of the above' | r k ∈ r} and the prompt word Prompt3 realize relationship fusion, and obtain the fused relationship R fu ;

[0048] S34: Input the text block T chunk , the topic entity Topic, and the prompt word Prompt4 realize entity extraction, and obtain the entity E1;

[0049] S35: Input the text block T chunk , the head and tail entities {h, t} of the triple Tri1, the entity E1, and the prompt word Prompt5 complete entity fusion, and obtain the fused entity E fu ;

[0050] S36: Input the text block T chunk , the topic entity Topic, the fused relationship R fu , the fused entity E fu , and the prompt word Prompt6 complete the final triple extraction Tri last , and further obtain the telecom specification knowledge graph.

[0051] Further, in the step S4, the specific processing process is as follows:

[0052] S41: Store the entities in the telecom specification knowledge graph in the milvus vector data volume after vectorization;

[0053] S42: When receiving a question asked by a user, the task control center assigns the task to the telecom specification multi-granularity retrieval question and answer intelligent agent, extracts all topic words from the question and vectorizes the topic words, performs similarity matching with all entities in the vector database, and returns top 3 with the highest similarity;

[0054] S43: Then return the 3-hop subgraph where the top 3 are located in the telecom specification knowledge graph, input the subgraph after pruning to the LLM model for final answer generation, and return to the user.

[0055] Compared with the prior art, the method has the following advantages: the method for generating enhanced graph retrieval based on the telecommunication specification multi-modal knowledge graph first analyzes the title, text, picture and table in the PDF through the telecommunication specification analysis agent with the help of layout analysis, OCR and LLM tools; then the telecommunication specification multi-modal extraction agent is called to extract the chart name and description information from the picture and table by using the multi-modal large model; the multi-modal, multi-level and multi-granularity telecommunication specification knowledge graph including the title, text block, picture, table and triple is constructed by constructing the title hierarchical structure and extracting the triple from the text block; finally, the telecommunication specification multi-granularity retrieval agent is called to combine the knowledge graph and the large model deeply, solve the problem of multi-hop reasoning of the traditional RAG, and alleviate the illusion problem that may occur in the question answering process of the large model. In order to evaluate the quality of the construction of the telecommunication specification knowledge graph, the question answering result in the telecommunication specification field is evaluated, and the performance of the method is improved by 19% compared with the traditional RAG. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is a framework schematic diagram of the method for generating enhanced graph retrieval based on the multi-agent collaborative architecture and the telecommunication specification multi-modal knowledge graph in the embodiment of the application;

[0057] Figure 2 is an example diagram of the Markdown document block result in the embodiment of the application;

[0058] Figure 3 is a schematic diagram of the telecommunication specification knowledge graph SCHEMA in the embodiment of the application;

[0059] Figure 4 is a general flow schematic diagram of the method for generating enhanced graph retrieval based on the telecommunication specification multi-modal knowledge graph. DETAILED DESCRIPTION

[0060] The embodiments of the application will be described in detail below, and the embodiments are implemented on the premise of the technical scheme of the application, and detailed implementation modes and specific operation processes are given, but the protection scope of the application is not limited to the following embodiments.

[0061] The embodiment provides a technical scheme: a method for generating enhanced graph retrieval based on a multi-agent collaborative architecture and a telecommunication specification multi-modal knowledge graph, which reduces the construction cost, realizes automatic, high-precision, unsupervised and sustainable construction of the telecommunication specification knowledge graph, and supports dynamic expansion and efficient graph retrieval question answering service, thereby providing a more intelligent and flexible knowledge management solution for the telecommunication operator field.

[0062] The specific content of the method of the application is as follows:

[0063] The multi-agent collaborative architecture is composed of three core agents: define the agent set A = {A1, A2, A3}, wherein A1 is a multi-level parser agent for telecom standards (MPATS), A2 is a multi-modal extractor agent for telecom standards (MEATS), and A3 is a multi-granularity retrieval agent for telecom standards (MRATS).

[0064] Task description:

[0065] Let the set of telecom standard documents be D = {d1, d2, …, d ∣D∣ |d k ∈PDF} When it is necessary to implement a fast and accurate question and answer task based on a certain document d i , first parse the content of the document through the multi-level parser agent for telecom standards, then extract triplets based on the multi-modal extractor agent for telecom standards to obtain a knowledge graph set G i , and finally implement industry standard question and answer through the multi-granularity retrieval agent for telecom standards. The specific process is shown in Figure 1 . The specific operation is as follows:

[0066] (1) Multi-level parser agent for telecom standards

[0067] Five types of semantic labels are preset: text (Text), image (Image), table (Table), heading (Heading), and abandon area (Abandon). For a certain N-page document d i , the jth page is labeled with different types of labels Label x for each block area of the page through the page layout analysis model DocLayout-YOLO, and the coordinate positioning information (x0, y0, x1, y1) and the confidence parameter of the four corners of the rectangular frame are recorded synchronously to realize page layout analysis:

[0068]

[0069] Wherein, represents page layout analysis on the jth page , and Label [Tex,Img,Tab,Hea] represents the result classification of page layout analysis: text, image, table, or heading.

[0070] According to the layout analysis, the label is processed for each region respectively:

[0071] 1) Text

[0072] Use RapidOCR to process the text region Extract the text T.

[0073] 2) Picture

[0074] Convert the image region to Base64 encoding, store the image path Img path , use Qwen2.5-VL-3B-Instruct multi-modal large model to generate image name Img name and image description Img Caption .

[0075] 3) Table

[0076] Convert the table region to HTML format table content Tab content , use Qwen2.5-7B-Instruct large model to generate table name Tab name and table description Tab Caption .

[0077] 4) Title

[0078] Extract the title content H i of all title regions d of the document Hea To solve the problem of title level error and title misidentification as text, we design a title optimization scheme as follows:

[0079] a. First, construct a title hierarchy graph:

[0080] G=(V,E)

[0081] Where, the node set represents the title set V={h1,h2,…,h ∣n∣ |h i ∈H Hea}, the edge set E represents the parent-child relationship between titles.

[0082] b. Use depth-first search (DFS) to detect illegal level jumps:

[0083] Find the root node h r with level L(r)=1, recursively visit its child nodes, and if a child node is found with level greater than the current node level+1, record it as illegal jump J:

[0084] J={(h mh l )|L(h m )>L(h l )+1}

[0085] where h m , h l denote the mth and lth headings in H Hea , L(h m ), L(h l ) denote the level of h m , h l .

[0086] c. Adjust the hierarchy using Breadth-First Search (BFS):

[0087] For each illegal jump pair (h m , h l ) ∈ J, use BFS to find the nearest legal intermediate level L' such that L' < L(h m ) from the mth heading h l , modify the edges of L' to construct new parent-child relationships:

[0088] L(h l ) < L' < L(h m )

[0089] d. Update the heading hierarchy graph G':

[0090] After completing the heading optimization, construct the heading hierarchy triples Tti heading according to the tree structure of headings:

[0091] Tti heading = {(h i , r, h j ) | {(h i , h j ) ∈ E ∧ L(h j ) = L(h i ) + 1}

[0092] where h i , h j denote two headings with superior-inferior relationships, r denotes the relationship between h i , h j .

[0093] Finally, complete the multi-level parsing of the telecommunications specification and convert the document d i to Markdown format M i .

[0094] We compare M i with the text T and table content Tab contentThe text block T is processed in a block manner chunk The data size is not more than 512 tokens, and an example of the result is as follows Figure 2 As shown in the following table.

[0095] (2) Telecom specification multi-modal extraction agent

[0096] In order to build a multi-modal, multi-level and multi-granularity telecom specification knowledge graph SCHEMA, we integrated the association between the title level relationship, text block, image, table and triple:

[0097] S TS-KG =(E TS-KG ,R)

[0098] Among them, S TS-KG represents the telecom specification knowledge graph, the node set E TS-KG ={H Hea ,T chunk ,Img path ,Img name ,Img Caption ,Tab content ,Tab name ,Tab Caption ,Tri last} represents the hierarchical title, text block, image path, image name, image description, table content, table name, table description and triple entity set, and the edge set R = {(e1, r, e2) | {(e1, e2) ∈ E TS-KG} represents the relationship set between entities, and r represents the relationship between two entities.

[0099] The telecom specification knowledge graph we built not only covers traditional text information, but also integrates pictures, tables and structured data, forming a detailed and comprehensive telecom specification knowledge graph SCHEMA. As Figure 3 shown, this architecture not only supports cross-modal data query and retrieval, but also provides a solid foundation for deeper knowledge reasoning and decision support.

[0100] The telecom specification multi-level analysis agent has completed the title level relationship triple, text block, image and table analysis part. For the triple extraction of text block and table content, the telecom specification multi-granularity retrieval question and answer agent adopts a large model driven few-shot unsupervised knowledge extraction method, and realizes the accurate construction of telecom specification domain triples through structured prompt word engineering. The general prompt word content requirements include: a. Conform to the entity and relationship naming specification; b. Have compound entity splitting rules; c. Follow the terminology standards of telecom specification.

[0101] The prompt word engineering completes six sub-tasks in steps to achieve high-quality knowledge extraction:

[0102] Step 1: Input text block T chunk , topic entity Topic, and prompt word Prompt1 to implement triple extraction Tri1 = (h, r, t), where Topic is the title corresponding to the current text block, h, r, t represent the head entity, relationship, and tail entity of the triple;

[0103] Step 2: Input text block T chunk , triple Tri1, and prompt word Prompt2 to implement relationship description Re des , which is to provide detailed description for the relationship r in the triple, and Prompt2 has additional requirements: a. The description format follows'relationship name: relationship description'; b. The relationship description needs to be universal.

[0104] Step 3: Input text block T chunk , triple Tri1, relationship description Re des , candidate relationship set C R = {r1, r2...r n , 'none of the above' | r k ∈r}, and prompt word Prompt3 to implement relationship fusion R fu , which first vectorizes all relationships, and uses the bge model to select the top 10 relationships with a correlation degree TopP greater than 0.8 with the current relationship r k to form the candidate relationship set C R , and then selects an option from the candidate relationship set C m , which is the replaceable relationship r chunk or 'none of the above', Prompt3 has additional requirements: a. Semantic matching degree evaluation; b. Relationship replacement necessity judgment.

[0105] Step 4: Input text block T chunk , topic entity Topic, and prompt word Prompt4 to implement entity extraction E1.

[0106] Step 5: Input text block T chunk , head and tail entities {h, t} of triple Tri1, entity E1, and prompt word Prompt5 to complete entity fusion E fu , Prompt5 has additional requirements: a. Synonym merging rules.

[0107] Step 6: Input text block T chunk , topic entity Topic, relationship R fu , entity E fu , and prompt word Prompt6 to complete the final triple extraction, and obtain Tri lastIn this stage, the fused relationship and entity are taken as a reference into the prompt engineering, and the optimized triplets are finally output as the result of knowledge extraction, completing the construction of the telecom specification multi-modal knowledge graph.

[0108] (3) Telecom specification multi-granularity retrieval question and answer agent

[0109] The telecom specification multi-granularity retrieval question and answer agent realizes graph-based retrieval augmented generation (GraphRAG) for telecom specifications. First, all entities are vectorized and stored in the milvus vector data volume. When receiving a user question, the task control center assigns the task to the telecom specification multi-granularity retrieval question and answer agent, extracts all topic words from it, and vectorizes these topic words. Similarity matching is performed with all entities in the vector database, and the top 3 with the highest similarity are returned. The 3-hop subgraph where the top 3 are located in the knowledge graph is returned, and after pruning the subgraph, the final answer is generated by inputting it to the LLM and returned to the user.

[0110] The data source of the present application is the telecom industry specification documents collected on the Internet. We use RAG and the method proposed in the present application to conduct experiments, and the evaluation index used is the average of F1 score and Bertscore commonly used in reasoning question and answer tasks, where F1 score focuses on completely correct answers, and Bert score focuses on semantic similarity of answers.

[0111] Score = (F1 score + Bert score) / 2

[0112] Where F1 score is the harmonic mean of precision (precision) and recall (recall), and BERTScore is a text similarity measurement method based on pre-trained language models, which calculates the similarity between two sentences by comparing the embedding vectors of each word in the sentence.

[0113] Table 1 Telecom specification GraphRAG evaluation results

[0114] Data type Score Traditional RAG 0.52 Our model 0.71

[0115] According to the telecom specification GraphRAG evaluation results shown in Table 1, the traditional RAG score was 0.52, while our model scored 0.71, a 19% improvement over the traditional RAG. This demonstrates that by introducing the knowledge graph, organically combining upper-level text blocks, lower-level triples, and multimodal data, and organizing the hierarchical relationships of titles to build a complete data system, and based on graph-based retrieval enhancement generation technology, our proposed method performs better when handling telecom specification-related questions, can more accurately understand and generate answers, and thus provide users with more reliable information support.

[0116] In summary, the graph retrieval enhancement generation method based on the telecommunications specification multimodal knowledge graph in the above embodiment first uses the telecommunications specification parsing agent to parse the titles, text, images, and tables in the PDF using layout analysis, OCR, and LLM tools. It then calls the telecommunications specification multimodal extraction agent to extract chart names and description information from images and tables using a multimodal large model. By constructing a title hierarchy and extracting triples from text blocks, a multimodal, multi-level, and multi-granular telecommunications specification knowledge graph consisting of titles, text blocks, images, tables, and triples is constructed. Finally, the telecommunications specification multi-granularity retrieval agent is called to deeply integrate the knowledge graph with the large model, solving the problem of traditional RAG's inability to multi-hop reasoning and alleviating the hallucination problem that may occur with large models during question-answering. To evaluate the quality of the telecommunications specification knowledge graph construction, question-answering results in the telecommunications specification field were evaluated. Compared with traditional RAG, our method achieved a 19% performance improvement.

[0117] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A graph retrieval enhancement generation method based on a multimodal knowledge graph of telecommunication specifications, characterized by: The following steps are involved: S1: Establishing a multi-agent collaborative architecture Establish a multi-agent collaborative architecture, including a telecommunications specification multi-level parsing agent, a telecommunications specification multi-modal extraction agent, and a telecommunications specification multi-granularity retrieval and question-answering agent; S2: Document content analysis Utilize the telecom specification multi-level parsing agent to parse each PDF document in the preset telecom specification document set; S3: Knowledge Graph Construction Based on the parsing results, the telecom specification multimodal extraction agent is used to extract triples and obtain a knowledge graph set. S4: Graph Retrieval Enhancement Generation Based on the extracted knowledge graph set, industry standard question and answer are realized through the telecommunications standard multi-granularity retrieval question and answer agent, completing the graph retrieval enhancement generation task.

2. The graph retrieval enhancement generation method based on the telecommunications specification multimodal knowledge graph according to claim 1 is characterized in that: In step S2, the process of parsing using the telecom standard multi-level parsing agent is as follows: S21: For any N-page document d in the telecommunications specification document collection i , page j Through the layout analysis model, different types of labels are added to each area of ​​the page, and the coordinate positioning information and confidence parameters of each area are simultaneously recorded to achieve layout analysis; S22: Process each region separately according to the labels of the layout analysis to obtain the processing results of each region; S23: The document d is processed according to the processing results of each area. i Convert to Markdown format set to complete the parsing work.

3. The graph retrieval enhancement generation method based on the telecommunications specification multimodal knowledge graph according to claim 2 is characterized in that: In step S21 , the tag types include text, image, table, heading, and abandonment area.

4. The graph retrieval enhancement generation method based on the telecommunications specification multimodal knowledge graph according to claim 3 is characterized in that: In step S22, the specific processing process is as follows: S221: Use RapidOCR to identify text regions LabelTex Extract text T; S222: Image region Region LabelImg Convert to Base64 encoding and store the image path Img path , use Qwen2.5-VL-3B-Instruct model combined with prompt word engineering to generate image name Img name and image description Img Caption ; S223: Set the table area Region LabelTab Convert the table content to HTML format Tab content , use Qwen2.5-7B-Instruct model combined with prompt word engineering to generate table name Tab name and table description Tab Caption ; S224: Extract document d i All title regions LabelHea Title content H Hea , and optimize the title level.

5. The graph retrieval enhancement generation method based on the telecommunication specification multimodal knowledge graph according to claim 4 is characterized in that: In step S224, the process of optimizing the title level is as follows: S2241: Construct a title hierarchy diagram: G=(V,E) Among them, the node set represents the title set V = {h1,h2,…,h ∣n∣ |h i ∈H Hea }, the edge set E represents the parent-child relationship between titles; S2242: Use depth-first search to detect illegal level jumps: Find the root node h r Its level L(r) = 1, recursively visit its child nodes, if the child node level is found to be greater than the current node level + 1, then record it as an illegal jump J: J={(h m ,h l )|L(h m )>L(h l )+1} Among them, h m 、h l Indicates H Hea The mth and lth titles in L(h m )、L(h l ) represents h m 、h l level; S2243: Adjusting the hierarchy using breadth-first search: For each illegal jump pair (h m ,h l )∈J, use breadth-first search to find the mth title h m Go down layer by layer to find the nearest legal intermediate level L ′ L′ <L(h l ), modify the edge of L′ to construct a new parent-child relationship: L(h l ) <L′<L(h m ); S2244: Update the title hierarchy diagram structure: After completing the title level optimization, construct the title level triple Tti according to the tree structure of the title heading : Tti heading ={(h i ,r,h j )|{(h i ,h j )∈E∧L(h j )=L(h i )+1} Among them, h i 、h j Indicates two titles with a hierarchical relationship, r means h i 、h j relationship.

6. The graph retrieval enhancement generation method based on the telecommunication specification multimodal knowledge graph according to claim 4 is characterized in that: In step S3, the telecom specification knowledge graph SCHEMA is obtained through triple extraction as follows: S TS-KG =(E TS-KG ,R) Among them, S TS-KG Represents the telecommunications specification knowledge graph SCHEMA, node set E TS-KG ={H Hea ,T chunk ,Img path ,Img name ,Img Caption ,Tab content ,Tab name ,Tab Caption ,Tri last } represents the hierarchical title, text block, image path, image name, image description, table content, table name, table description and triple entity set, edge set R = {(e1,r,e2)|{(e1,e2)∈E TS-KG } represents the set of relationships between entities, r represents the relationship between two entities e1 and e2; T chunk Through M i Text T and table content Tab under the same title content After block processing, M i By documentd i Convert to Markdown format.

7. The graph retrieval enhancement generation method based on the telecommunication specification multimodal knowledge graph according to claim 6 is characterized in that: In step S3, the specific process of implementing triple extraction based on structured prompt word engineering using the telecommunications standard multimodal extraction agent is as follows: S31: Input text block T chunk , the subject entity Topic and the prompt word Prompt1 realize triple extraction Tri1 = (h, r, t), where the subject entity Topic is the title corresponding to the current text block, h, r, t represent the head entity, relationship, and tail entity of the triple; S32: Input text block T chunk , triple Tri1 and prompt word Prompt2 realize the relationship description Re des ; S33: Input text block T chunk , triple Tri1, relation description Re des , candidate relation set C R ={r1,r2...r n ,′None of the above′|r k ∈r} and prompt word Prompt3 to achieve relation fusion and obtain the fusion relation R fu ; S34: Input text block T chunk , the subject entity Topic and the prompt word Prompt4 are used to extract the entity and obtain the entity E1; S35: Input text block T chunk , the head and tail entities {h, t} of the triple Tri1, entity E1 and prompt word Prompt5 complete entity fusion, and obtain the fused entity E fu ; S36: Input text block T chunk , topic entity Topic, fusion relationship R fu , fusion entity E fu And prompt word Prompt6 complete the final triple extraction, and get Tri last , and then obtain the telecommunications specification knowledge graph.

8. The graph retrieval enhancement generation method based on the telecommunication specification multimodal knowledge graph according to claim 7 is characterized in that: In step S4, the specific processing process is as follows: S41: vectorize the entities in the telecommunications specification knowledge graph and store them in the milvus vector data volume; S42: When a question is received from a user, the mission control center assigns the task to the Telecom Standard Multi-Grained Retrieval Question Answering Agent, which extracts all key words from the question and vectorizes them. It then performs similarity matching with all entities in the vector database and returns the top three most similar entities. S43: Return the 3-hop subgraph where the top 3 are located in the telecommunications specification knowledge graph, prune the subgraph and input it into the LLM model to generate the final answer, which is returned to the user.

Images