Telecommunication domain knowledge graph construction and retrieval method based on multi-format data processing

Abstract

The invention discloses a telecommunication field knowledge graph construction and retrieval method based on multi-format data processing, which is used for intelligently analyzing and processing multiple types of files and realizing high-precision structured storage of file contents. Through word segmentation, named entity recognition and dependency syntax analysis, core entities and relationships in different types of files are accurately extracted, and cross-document theme communities and hierarchical structures are constructed through a knowledge graph; according to the method, community abstracts are generated according to theme information of different documents, query contexts are dynamically adjusted by combining a context-sensitive retrieval mechanism and reinforcement learning-driven sorting optimization, and a retrieval strategy is continuously optimized based on user behavior feedback, so that a more personalized and intelligent retrieval result is realized, and the user experience is improved. And the accuracy and depth of knowledge recall are obviously improved. The multi-type file data processing method provided by the invention has wide application value, and can meet the requirements of large model knowledge enhancement in the industry on accuracy, integrity and intellectualization.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for constructing and retrieving knowledge graphs in the telecommunications field based on multi-format data processing. Background Technology

[0002] Knowledge graphs are a structured way of representing knowledge. They use nodes (entities), edges (relationships), and attributes to represent entities, concepts, and their relationships in the real world. In the fields of artificial intelligence and knowledge management, knowledge graphs are widely used in scenarios such as intelligent question answering, recommendation systems, and semantic search to improve the efficiency and accuracy of information retrieval and processing.

[0003] In the telecommunications sector, document data is a critical information resource, containing a wealth of business knowledge, technical solutions, and market analysis. To efficiently utilize this document data, building intelligent knowledge bases and implementing large-scale model knowledge enhancement has become a common requirement. Knowledge bases extract and store information from document data in a database using structured knowledge representation, facilitating user retrieval and in-depth queries. However, a core technical challenge lies in how to build a high-quality knowledge base to improve the accuracy of knowledge retrieval while processing diverse document data types. Document data in the telecommunications sector typically involves multiple formats and types, including Word, PDF, PPT, and PNG. These files differ significantly in format and content, posing complex challenges to data processing. Existing methods primarily focus on simple format conversion, information extraction, and content analysis, failing to effectively address the comprehensiveness and consistency issues in processing diverse document data types.

[0004] The current methods for processing multiple types of files are mainly based on steps such as file format conversion, content extraction, and information extraction, as well as the analysis and processing of file content. The current multi-type file data processing technology still has significant defects, mainly reflected in the following aspects: (1) Insufficient efficiency and completeness of file format conversion: Existing methods are prone to losing content when processing multiple file format conversions, affecting the completeness and continuity of document information. Lack of understanding of the overall semantic structure of the document makes it difficult to form an accurate global summary; (2) Poor granularity and consistency of information extraction: Traditional information extraction methods are difficult to accurately extract key entities and relationships in the file, resulting in an imprecise or redundant knowledge base structure and low accuracy of knowledge retrieval. (3) Lack of a unified theme structure in content analysis: The strategies for file content analysis and processing are not unified, which easily leads to insufficient or inconsistent representation capabilities. (4) Lack of global summarization and summary capabilities: Traditional summary generation is difficult to generate a summary with a global perspective in the content of multiple files, affecting the depth and comprehensiveness of knowledge retrieval. Summary of the Invention

[0005] The purpose of this invention is to provide a method for constructing and retrieving knowledge graphs in the telecommunications field based on multi-format data processing.

[0006] The technical solution adopted in this invention is: A method for constructing and retrieving knowledge graphs in the telecommunications field based on multi-format data processing includes the following steps: Step 1: Receive and retrieve various types of files uploaded by users and store them on the server; Step 2: Determine the file type based on the file's prefix and content characteristics, select the corresponding text extraction method based on the file type to extract the text content from the file, and perform data cleaning and standardization processing. Specifically, the system determines the file type (such as Word, PDF, image, etc.) based on the file's characteristics (such as prefix, content characteristics, etc.).

[0007] Step 3: The extracted text is processed by word segmentation and part-of-speech tagging, and then the key entities in the text are identified as nodes of the knowledge graph by Named Entity Recognition (NER) technology. Step 4: Identify the relationships between entities based on dependency parsing as edges in the knowledge graph; Step 5: Use a community detection algorithm to cluster entities and relationships in the knowledge graph, divide closely related content into topic communities and generate topic community summaries; at the same time, combine the document content of the file and the knowledge graph to generate a file summary for each file. Step 6: Convert the text content and knowledge graph structure into vector representations and store them in a vector database; Step 7: By monitoring user-input queries in real time, analyze the relationships between the entities involved in the knowledge graph, dynamically capture changes in user intent, and combine historical query data to generate comprehensive contextual information; Step 8: Optimize and generate a new query vector by integrating the context information of historical queries with the entity information of the current query; at the same time, dynamically adjust the priority of the search results based on the user's query history and behavior data, and form search results from high to low priority.

[0008] Specifically, in step 8, query understanding and vector generation are optimized by integrating contextual information from historical queries with entity information from the current query, ensuring that the semantics of the search results better meet user expectations. Simultaneously, the system dynamically adjusts the priority of search results based on user query history and behavioral data (such as click-through rate and dwell time), optimizes the weight of entity association information, and improves the relevance and accuracy of search results, ensuring that information retrieval better meets users' personalized needs.

[0009] Furthermore, in step 1, the user uploads various types of files through a web form or API interface; when the uploaded file is an image file, the text region in the image file is detected and preprocessed; the preprocessing operations include noise reduction, cropping, rotation correction, etc.

[0010] Furthermore, in step 2, the text content of the image file is extracted using character segmentation and OCR recognition technology, while the text data of other non-image files is extracted directly.

[0011] Furthermore, in step 4, a dependency parsing and relation extraction model is pre-built for extracting relationships between entities.

[0012] Specifically, based on dependency parsing and relation extraction models, the system identifies relationships between entities (such as "cooperation" and "subordination") and constructs these relationships as edges in a knowledge graph. The graph contains nodes (entities) and edges (relationships) and is aligned with existing knowledge bases to ensure consistency and eliminate redundancy.

[0013] Furthermore, in step 5, the core themes and sub-themes of the document are obtained by analyzing the document content, providing strong support for subsequent summary generation and knowledge retrieval; the community summary summarizes the main entities and relationships of the corresponding theme community.

[0014] Furthermore, step 8 introduces a context-sensitive retrieval mechanism, a real-time updated knowledge graph, and optimizes the ranking of retrieval results; Furthermore, the specific steps of the context-sensitive retrieval mechanism in step 8 are as follows: Step 8-1: Use a pre-trained large language model to perform semantic parsing on the user-input query, and identify keywords, intent and contextual information in the query; Step 8-2: Combine the user's historical query records with the context of the current query to dynamically expand the semantic scope of the query and generate a more accurate query vector; Step 8-3: Calculate the semantic similarity between the query and document content in the vector space using an optimized semantic matching algorithm, and optimize the ranking of search results based on the similarity results to improve the relevance of the search.

[0015] This invention employs the above technical solutions to automatically extract entities and relationships from various file formats, constructing a high-quality knowledge graph with topical communities and hierarchical structures. This knowledge graph not only supports structured storage and querying but also generates global community summaries. By combining a context-sensitive retrieval mechanism with reinforcement learning-driven ranking optimization, this invention further improves the accuracy and efficiency of the retrieval module, achieving more intelligent and personalized retrieval in large-scale document data processing, and significantly enhancing the quality and depth of knowledge retrieval. Attached Figure Description

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments; Figure 1 This is a flowchart illustrating the telecommunications domain knowledge graph construction and retrieval method based on multi-format data processing according to the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0018] This invention proposes a multi-type file processing workflow that addresses the limitations of existing technologies in terms of accuracy and performance. It designs an efficient multi-branch processing flow, comprehensively improving the processing capabilities for various file types. The overall solution consists of the following core modules, each with clearly defined functions and collaborative relationships, achieving a closed-loop processing from file upload to knowledge graph construction.

[0019] This invention intelligently parses and processes various file types, achieving high-precision structured storage of file content. Through word segmentation, named entity recognition, and dependency parsing, it accurately extracts core entities and relationships from different file types and constructs cross-document topic communities and hierarchical structures using a knowledge graph. Community summaries are generated based on the topic information of different documents. A context-sensitive retrieval mechanism and reinforcement learning-driven ranking optimization are combined to dynamically adjust the query context, and the retrieval strategy is continuously optimized based on user behavior feedback. This results in more personalized and intelligent retrieval results, significantly improving the accuracy and depth of knowledge retrieval. This multi-type file data processing method has broad application value and can meet the requirements of accuracy, completeness, and intelligence for large-scale model knowledge enhancement in various industries.

[0020] like Figure 1 As shown, this invention discloses a method for constructing and retrieving knowledge graphs in the telecommunications field based on multi-format data processing, which includes the following steps: Step 1: Receive and retrieve various types of files uploaded by users and store them on the server; Specifically, it supports users uploading various types of files, including Word, Excel, PDF, PPT, image files, web page files, and text files, via web forms and API interfaces. Uploaded files are stored on the server and then processed in subsequent file handling procedures.

[0021] Step 2: Determine the file type based on the file's prefix and content characteristics, select the corresponding text extraction method based on the file type to extract the text content from the file, and perform data cleaning and standardization processing. Specifically, text content in image files is extracted using character segmentation and OCR recognition technology, while text data is directly extracted from other non-image files. Further, step 2, extracting text content from image files using character segmentation and OCR recognition technology, includes the following steps: Step 2-1, Layout Analysis and Preprocessing: Use layout analysis techniques on the image file to identify text block locations and segment text regions. Step 2-2: Enhance the text region with images (such as brightness adjustment, cropping, and noise reduction) to generate a clear text image for recognition.

[0022] Steps 2-3, Character Segmentation and OCR Recognition: Use character segmentation algorithms (such as edge detection or cluster analysis) to extract single character regions from the text image, and combine them with the OCR module to recognize the character regions and obtain high-precision text information.

[0023] Specifically, file types are determined based on prefixes, content characteristics, and other factors, categorizing them into common office documents such as Word, Excel, PDF, and PPT, as well as other types of unstructured data such as image files, web page files, and plain text files. File type determination can be achieved through rule matching and model training.

[0024] For image files, layout analysis techniques are used to determine the location of text blocks and segment the image to obtain text regions. Image enhancement preprocessing (such as brightness / contrast adjustment, cropping, rotation correction, and noise reduction) is performed on the determined text regions to output clear text image regions for recognition. On the preprocessed text image regions, image processing techniques are used to extract characters, which are then segmented into single-character images based on rules such as character position, size, and color. Finally, an OCR module is used to recognize the content of each single-character image to obtain text information, improving recognition accuracy.

[0025] Step 3: The extracted text is processed by word segmentation and part-of-speech tagging, and then the key entities in the text are identified as nodes of the knowledge graph by Named Entity Recognition (NER) technology. Specifically, the extracted and identified text undergoes preliminary processing, including noise removal, word segmentation, part-of-speech tagging, and basic cleanup, to prepare for subsequent knowledge graph construction and analysis. Named entity recognition technology is used to automatically identify key entities (such as names of people, places, companies, and products) in the text and annotate these entities as nodes in the knowledge graph. As a feasible implementation, deep learning models (such as BERT or BiLSTM-CRF) are used to extract key entities (such as names of people, places, and companies) from the text. Models are trained on domain-specific datasets to optimize entity recognition accuracy in scenarios such as telecommunications, law, or document management.

[0026] Step 4: Identify the relationships between entities based on dependency parsing as edges in the knowledge graph; Specifically, dependency parsing and relation extraction models are used to identify relationships between entities in the text (such as "cooperation," "competition," "subordination," etc.). The extracted relationships are stored as triples (entity 1, relation, entity 2), and these relationships are added to the edges of the knowledge graph to form associations between entities and establish temporal relationships. The identified entities and relations are stored in the knowledge graph, forming structured graph data composed of nodes (entities) and edges (relationships). The identified entities are aligned with entities in the existing knowledge base to ensure data consistency and eliminate redundancy. In other words, dependency parsing and relation extraction models are used to identify relationships between entities in paragraph content, generating a knowledge graph containing multi-level associations, including entity alignment (aligning identified entities with entities in the existing knowledge base). The extracted entities and relations are stored in a graph database or a triple database, forming the knowledge graph structure.

[0027] Step 5: Use a community detection algorithm to cluster entities and relationships in the knowledge graph, divide closely related content into topic communities and generate topic community summaries; at the same time, combine the document content of the file and the knowledge graph to generate a file summary for each file. Specifically, graph clustering algorithms (such as Louvain or Walktrap) are used to cluster entities and relationships in the knowledge graph, grouping closely related entities and relationships into the same topic community. Thematic associations between communities are automatically detected, providing a hierarchical structure for multi-document content. These topic communities can help analyze the core topics and subtopics of documents.

[0028] Based on the community detection results, combined with the community structure and text information, the system generates a concise community topic summary for each community, outlining the main entities and relationships within the community. This provides a global and comprehensive perspective for the knowledge graph, enabling users to quickly understand the core content of the documents.

[0029] A concise document summary is generated based on paragraph content and the community topic structure of the knowledge graph, ensuring the accuracy and representativeness of the summary information and making it easy for users to quickly obtain the core content of the document.

[0030] Step 6: Convert the text content and knowledge graph structure into vector representations and store them in a vector database; Specifically, vectorization models are used to transform the community structure of text and knowledge graphs into vectors and store them in vector databases (such as Milvus, Qdrant, and Weaviate) to enable efficient retrieval and information recall based on similarity.

[0031] Step 7: By monitoring user-input queries in real time, analyze the relationships between the entities involved in the knowledge graph, dynamically capture changes in user intent, and combine historical query data to generate comprehensive contextual information; Specifically, by combining contextual information and a knowledge graph, the retrieval strategy is dynamically adjusted to achieve efficient knowledge retrieval and information recall. By introducing a context-sensitive retrieval mechanism, a real-time updated knowledge graph, and a ranking strategy optimized by reinforcement learning, the system ensures more accurate and relevant search results, and can quickly adapt to changes in user needs, providing a more intelligent and personalized search experience. Through vectorization technology and retrieval optimization strategies, the system comprehensively improves the efficiency of similarity retrieval and information recall, achieving dynamic context awareness and intelligent ranking.

[0032] Step 8: Optimize and generate a new query vector by integrating the context information of historical queries with the entity information of the current query; at the same time, dynamically adjust the priority of the search results based on the user's query history and behavior data, and form search results from high to low priority.

[0033] Specifically, by monitoring user-input queries in real time, the system analyzes the entities involved and their relationships within the knowledge graph, dynamically captures changes in user intent, and generates comprehensive contextual information by combining historical query data. By integrating the context of historical queries with the entity information of the current query, the system optimizes query understanding and vector generation, ensuring that the semantics of the search results better meet user expectations. Simultaneously, the system dynamically adjusts the priority of search results based on user query history and behavioral data (such as click-through rate and dwell time), optimizes the weight of entity association information, and improves the relevance and accuracy of search results, ensuring that information retrieval better meets users' personalized needs.

[0034] Furthermore, step 8 introduces a context-sensitive retrieval mechanism, a real-time updated knowledge graph, and optimizes the ranking of retrieval results; furthermore, the specific steps of the context-sensitive retrieval mechanism in step 8 are as follows: Step 8-1: Use a pre-trained large language model to perform semantic parsing on the user-input query, and identify keywords, intent and contextual information in the query; Step 8-2: Combine the user's historical query records with the context of the current query to dynamically expand the semantic scope of the query and generate a more accurate query vector; Step 8-3: Calculate the semantic similarity between the query and document content in the vector space using an optimized semantic matching algorithm, and optimize the ranking of search results based on the similarity results to improve the relevance of the search.

[0035] This invention introduces a context-based intelligent retrieval algorithm that dynamically expands the semantic scope of the query based on the user's historical query information and the current context, generating accurate query vectors. Through an optimized semantic matching algorithm, the system not only improves the relevance of search results but also prioritizes displaying content that highly matches the user's query semantics through ranking optimization.

[0036] This invention utilizes cross-document association analysis and knowledge graph capabilities to identify and mine potential related entities and relationships between documents, forming cross-document knowledge links. Through graph neural network technology, the system can infer causal relationships between different documents and generate knowledge links of different dimensions within a multi-level association structure, thus providing users with more comprehensive information retrieval results. This invention introduces a reinforcement learning algorithm to continuously optimize the retrieval ranking strategy based on real-time user feedback. The reinforcement learning reward mechanism adjusts the strategy according to whether the retrieval results meet user needs. By combining with a knowledge graph, the system can not only identify the user's query intent but also continuously and adaptively adjust recommended content, further improving retrieval efficiency.

[0037] This invention employs the above technical solutions, combining OCR, NLP, LLM, and knowledge graph technologies to intelligently parse and process various file types, effectively reducing information loss and conversion errors, and achieving high-precision structured storage of file content. This invention utilizes a knowledge graph-based semantic analysis and paragraph segmentation strategy. Through word segmentation, named entity recognition, and dependency parsing, it accurately extracts core entities and relationships from different file types and constructs cross-document topic communities and hierarchical structures using knowledge graphs.

[0038] Furthermore, this invention utilizes community summary generation technology to automatically generate concise yet comprehensive community summaries based on the topic information of different documents, improving document representation capabilities and knowledge base recall accuracy. Combining a context-sensitive retrieval mechanism with reinforcement learning-driven ranking optimization, the system can dynamically adjust the query context and continuously optimize retrieval strategies based on user behavior feedback, thereby achieving more personalized and intelligent retrieval results and significantly improving the accuracy and depth of knowledge retrieval. Compared to traditional methods, this solution can provide more comprehensive and semantically deep knowledge summaries while maintaining data integrity, significantly improving the processing efficiency and knowledge value-added capabilities of multi-type file data.

[0039] The multi-type file data processing method of the present invention has wide application value and can meet the requirements of accuracy, completeness and intelligence of large model knowledge enhancement in the industry.

[0040] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. A method for constructing and retrieving knowledge graphs in the telecommunications field based on multi-format data processing, characterized by: Includes the following steps: Step 1: Receive and retrieve various types of files uploaded by users and store them on the server; Step 2: Determine the file type based on the file's prefix and content characteristics, select the corresponding text extraction method based on the file type to extract the text content from the file, and perform data cleaning and standardization processing. Step 3: The extracted text is processed by word segmentation and part-of-speech tagging, and then the key entities in the text are identified as nodes of the knowledge graph by Named Entity Recognition (NER) technology. Step 4: Identify the relationships between entities based on dependency parsing as edges in the knowledge graph; Step 5: Use a community detection algorithm to cluster entities and relationships in the knowledge graph, divide closely related content into topic communities and generate topic community summaries; at the same time, combine the document content of the file and the knowledge graph to generate a file summary for each file. Step 6: Convert the text content and knowledge graph structure into vector representations and store them in a vector database; Step 7: By monitoring user-input queries in real time, analyze the relationships between the entities involved in the knowledge graph, dynamically capture changes in user intent, and combine historical query data to generate comprehensive contextual information; Step 8: Optimize and generate a new query vector by integrating the context information of historical queries with the entity information of the current query; at the same time, dynamically adjust the priority of the search results based on the user's query history and behavior data, and form search results from high to low priority.

2. The method for constructing and retrieving telecommunications domain knowledge graphs based on multi-format data processing according to claim 1, characterized in that: In step 1, users upload various types of files through web forms or API interfaces; when uploading image files, text regions in the image files are detected and preprocessed; preprocessing operations include noise reduction, cropping, and rotation correction.

3. The method for constructing and retrieving telecommunications domain knowledge graphs based on multi-format data processing according to claim 1, characterized in that: In step 2, the text content of the image file is extracted using character segmentation and OCR recognition technology, while the text data of other non-image files is extracted directly.

4. The method for constructing and retrieving telecommunications domain knowledge graphs based on multi-format data processing according to claim 1, characterized in that: In step 4, a dependency parsing and relation extraction model is built in advance for extracting relationships between entities.

5. The method for constructing and retrieving telecommunications domain knowledge graphs based on multi-format data processing according to claim 1, characterized in that: In step 5, the core themes and sub-themes of the document are obtained by analyzing the document content, which provides strong support for subsequent summary generation and knowledge retrieval; the community summary summarizes the main entities and relationships of the corresponding theme community.

6. The method for constructing and retrieving telecommunications domain knowledge graphs based on multi-format data processing according to claim 1, characterized in that: Step 8 introduces a context-sensitive retrieval mechanism, a real-time updated knowledge graph, and optimizes the ranking of retrieval results.

7. The method for constructing and retrieving telecommunications domain knowledge graphs based on multi-format data processing according to claim 6, characterized in that: The specific steps of the context-sensitive retrieval mechanism in step 8 are as follows: Step 8-1: Use a pre-trained large language model to perform semantic parsing on the user-input query, and identify keywords, intent and contextual information in the query; Step 8-2: Combine the user's historical query records with the context of the current query to dynamically expand the semantic scope of the query and generate a more accurate query vector; Step 8-3: Calculate the semantic similarity between the query and document content in the vector space using an optimized semantic matching algorithm, and optimize the ranking of search results based on the similarity results to improve the relevance of the search.

Images