A Method and System for Constructing an Industry Knowledge Base Based on Text-Based Data Synthesis

Abstract

This invention relates to the field of knowledge base construction technology, specifically disclosing a method and system for constructing an industry knowledge base based on text-based data synthesis. The method includes entity recognition of multi-source text data, constructing a knowledge point sequence, and marking key knowledge points in the knowledge point sequence; expanding the key knowledge points based on a large language model to generate an expanded knowledge point set; constructing a knowledge graph based on the expanded knowledge point set, and constructing a preset number of knowledge connection paths in the knowledge graph; classifying the knowledge connection paths and inserting them as knowledge skeletons into the industry knowledge base. When dealing with massive amounts of multi-source text data, this invention expands the keywords based on a large language model to obtain expanded knowledge points, then constructs a knowledge graph, extracts node paths from the knowledge graph, uses the node paths as knowledge skeletons, classifies them, and stores them in the knowledge base. While the resulting knowledge base still contains a large amount of information, it significantly simplifies its size.

Description

Technical Field

[0001] This invention relates to the field of knowledge base construction technology, specifically a method and system for constructing an industry knowledge base based on text-based data synthesis. Background Technology

[0002] In industry knowledge management and intelligent question-answering scenarios, enterprises and research institutions typically need to extract, organize, and apply professional knowledge from massive amounts of text data. However, existing knowledge extraction processes for text data mostly focus on text recognition, which extracts core content from existing text and stores it in a pre-set database to obtain a knowledge base. This results in a very rich knowledge base, but correspondingly, the content is also quite diverse. As the knowledge extraction process becomes more extensive, the content in the knowledge base also increases. As an information repository, its content is extremely rich, and when faced with an access request, only a search is needed to obtain the required content. It is conceivable that, since the knowledge base itself is constantly updated, as the content increases, the search process also needs to be updated accordingly, and the construction of the search architecture becomes increasingly difficult. The reason for this phenomenon is that the knowledge base construction process does not incorporate a content filtering process. Therefore, how to simplify the size of the knowledge base as much as possible when dealing with massive amounts of text data to reduce the difficulty of retrieval is the technical problem that this invention aims to solve. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for constructing an industry knowledge base based on text-based data synthesis, so as to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A method and system for constructing an industry knowledge base based on text-based data synthesis, the method comprising:

[0006] Acquire multi-source text data from the industry, perform entity recognition on the multi-source text data, construct a knowledge point sequence, and simultaneously determine the distance change sequence of the knowledge point sequence;

[0007] Key knowledge points are marked in the knowledge point sequence based on the distance change sequence;

[0008] Based on the large language model, key knowledge points are expanded to generate an expanded knowledge point set;

[0009] A knowledge graph is constructed based on an expanded set of knowledge points, and a predetermined number of knowledge connection paths are built in the knowledge graph.

[0010] Calculate the difference between knowledge connection paths and knowledge point sequences, classify knowledge connection paths according to the difference, and insert them into the industry knowledge base as knowledge skeletons;

[0011] When an access request is received, the knowledge skeleton is input into the large language model, and the displayed content is output.

[0012] As a further aspect of the present invention: the steps of acquiring multi-source text data of the industry, performing entity recognition on the multi-source text data, constructing a knowledge point sequence, and simultaneously determining the distance change sequence of the knowledge point sequence include:

[0013] Acquire multi-source text data from the industry, perform text segmentation on the multi-source text data, and obtain a sequence of text units;

[0014] Entity recognition is performed on each text unit to extract product entities, technology entities, and application entities, and a knowledge point sequence is constructed according to the text order;

[0015] Semantic vectorization of adjacent knowledge points is performed and cosine distance is calculated to generate a distance change sequence;

[0016] The distance change sequence is normalized.

[0017] As a further aspect of the present invention: the step of marking key knowledge points in the knowledge point sequence based on the distance change sequence includes:

[0018] Abrupt changes are marked in the distance change sequence based on a preset threshold; wherein, the threshold is connected to an adjustment port;

[0019] Mark the knowledge points corresponding to the mutation points as key knowledge points;

[0020] The processing order and expansion depth of keywords are determined based on the distribution density of key knowledge points in the knowledge point sequence; whereby the processing order is inversely proportional to the distribution density, and the expansion depth is directly proportional to the distribution density.

[0021] As a further aspect of the present invention: the step of expanding key knowledge points based on a large language model to generate an expanded knowledge point set includes:

[0022] An extended instruction input large language model is constructed using key knowledge points and their contextual semantic vectors.

[0023] The large language model generates explanatory content, causal relationship content, and application extension content related to key knowledge points;

[0024] Entity recognition and relation extraction are performed on the generated extended content to obtain structural nodes, mechanism nodes and application nodes;

[0025] The nodes are then structured to form an extended knowledge point set.

[0026] As a further aspect of the present invention: the step of constructing a knowledge graph based on an extended knowledge point set, and constructing a preset number of knowledge connection paths in the knowledge graph includes:

[0027] Nodes in the expanded knowledge point set are used as graph nodes;

[0028] Based on the semantic similarity, causal relationship and hierarchical relationship between nodes, node connection edges are established to construct a knowledge graph;

[0029] In the knowledge graph, multiple knowledge connection paths are generated based on semantic continuity constraints, topic consistency constraints, and logical interpretability constraints;

[0030] Each knowledge connection path is scored based on path coherence, topic coverage, and semantic consistency.

[0031] As a further aspect of the present invention: the step of calculating the difference between the knowledge connection path and the knowledge point sequence, classifying the knowledge connection path according to the difference, and inserting it into the industry knowledge base as a knowledge skeleton includes:

[0032] Calculate the degree of difference between the semantic distribution of each knowledge connection path and the semantic distribution of the original knowledge point sequence;

[0033] Based on the degree of difference, knowledge connection paths are divided into low deviation paths, medium deviation paths, and high deviation paths;

[0034] Different types of paths are constructed into technical knowledge skeletons, pedagogical knowledge skeletons, and application knowledge skeletons, and then inserted into the industry knowledge base.

[0035] The present invention also provides an industry knowledge base construction system based on text-based data synthesis, the system comprising:

[0036] The entity recognition module is used to acquire multi-source text data of the industry, perform entity recognition on the multi-source text data, construct a knowledge point sequence, and simultaneously determine the distance change sequence of the knowledge point sequence;

[0037] A key point marking module is used to mark key knowledge points in the knowledge point sequence based on the distance change sequence;

[0038] The knowledge point expansion module is used to expand key knowledge points based on the large language model and generate an expanded knowledge point set.

[0039] The connection path construction module is used to build a knowledge graph based on an extended knowledge point set, and to construct a preset number of knowledge connection paths in the knowledge graph;

[0040] The knowledge skeleton statistics module is used to calculate the difference between knowledge connection paths and knowledge point sequences, classify knowledge connection paths according to the difference, and insert them into the industry knowledge base as knowledge skeletons.

[0041] When an access request is received, the knowledge skeleton is input into the large language model, and the displayed content is output.

[0042] As a further aspect of the present invention: the entity recognition module includes:

[0043] The text segmentation unit is used to acquire multi-source text data from the industry, segment the multi-source text data into text units, and obtain a sequence of text units.

[0044] The sequential statistics unit is used to perform entity recognition on each text unit, extract product entities, technical entities and application entities, and construct a knowledge point sequence according to the text order;

[0045] The distance calculation unit is used to perform semantic vectorization processing on adjacent knowledge points and calculate the cosine distance to generate a distance change sequence.

[0046] A normalization processing unit is used to normalize the distance change sequence.

[0047] As a further aspect of the present invention: the key point marking module includes:

[0048] A mutation point marking unit is used to mark mutation points in the distance change sequence based on a preset threshold; wherein the threshold is connected to an adjustment port;

[0049] A marking execution unit is used to mark the knowledge points corresponding to the mutation points as key knowledge points;

[0050] The processing parameter determination unit is used to determine the processing order and expansion depth of keywords based on the distribution density of key knowledge points in the knowledge point sequence; wherein, the processing order is inversely proportional to the distribution density, and the expansion depth is directly proportional to the distribution density.

[0051] As a further aspect of the present invention: the knowledge point expansion module includes:

[0052] The instruction generation unit is used to construct an extended instruction input large language model using key knowledge points and their contextual semantic vectors.

[0053] The content expansion unit is used to generate explanatory content, causal relationship content, and application extension content related to key knowledge points from the large language model;

[0054] The node extraction unit is used to perform entity recognition and relation extraction on the generated extended content to obtain structural nodes, mechanism nodes and application nodes.

[0055] The node statistics unit is used to organize the nodes in a structured way to form an extended knowledge point set.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] This invention addresses the challenge of handling massive amounts of multi-source text data. It involves text recognition and extraction to obtain keyword sequences, expansion of these keywords using a large language model, and further keyword extraction to generate expanded knowledge points. A knowledge graph is then constructed based on these expanded knowledge points, resulting in a vast amount of knowledge data related to the text content. When analyzing this massive knowledge data, node paths are extracted from the knowledge graph and used as knowledge skeletons. These paths are then categorized and stored in a database, resulting in a knowledge base with a very large amount of information but a very small overall size. Upon receiving an access request, the corresponding knowledge skeleton is matched, and the large model is used to expand the knowledge skeleton. Because the large model itself has update capabilities, the resulting expanded content is dynamic and can better meet the needs. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0059] Figure 1 The overall flowchart of the method for constructing an industry knowledge base based on text-based data synthesis is shown.

[0060] Figure 2 The diagram shows the structure of an industry knowledge base construction system based on text-based data synthesis. Detailed Implementation

[0061] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0062] Figure 1 The present invention provides a general flowchart for a method of constructing an industry knowledge base based on text-based data synthesis. In this embodiment, the method includes:

[0063] Step S100: Obtain multi-source text data of the industry, perform entity recognition on the multi-source text data, construct a knowledge point sequence, and simultaneously determine the distance change sequence of the knowledge point sequence;

[0064] Multi-source text data refers to text data from different sources. The processing procedure provided by the technical solution of this invention processes each piece of text data sequentially. For different pieces of text data, it is only necessary to repeat the technical solution of this invention. Multi-source text data is a higher-level concept of text data, representing a certain type of text data. By applying text recognition to the text data, keywords can be extracted as knowledge points. By vectorizing the keywords, the differences between adjacent keywords can be calculated and represented by vector distance. By statistically analyzing the knowledge points, a knowledge point sequence is obtained. The distance is then calculated sequentially in the knowledge point sequence to obtain a distance change sequence.

[0065] Step S200: Mark key knowledge points in the knowledge point sequence based on the distance change sequence;

[0066] Distance change sequences reflect the changes in knowledge points. By analyzing distance change sequences, we can identify the more important knowledge points and mark them. The marked knowledge points are called key knowledge points.

[0067] Step S300: Expand key knowledge points based on the large language model to generate an expanded knowledge point set;

[0068] By using key knowledge points as source data and inputting them into a large language model, we can expand the content related to those knowledge points. Then, we can extract keywords from the related content to obtain an expanded set of knowledge points, which is called the expanded knowledge point set.

[0069] Step S400: Construct a knowledge graph based on the extended knowledge point set, and build a preset number of knowledge connection paths in the knowledge graph;

[0070] An extended knowledge point set is a collection of words, each of which can be converted into a word vector to quantify the relationships between different words. Given the words and their relationships, a knowledge graph can be constructed based on the extended knowledge point set. A knowledge graph is a graph data structure composed of various words. In the graph data structure, by randomly selecting a starting point and sequentially reading subsequent knowledge points, a word string, called a knowledge connection path, can be obtained. The selection process of the word string is random, and its length can be the same as the knowledge point sequence, or a pre-set interval can be used for random selection. Different degrees of randomness result in different numbers of paths.

[0071] Step S500: Calculate the difference between the knowledge connection path and the knowledge point sequence, classify the knowledge connection path according to the difference, and insert it into the industry knowledge base as a knowledge skeleton;

[0072] After obtaining the knowledge connection path, the simplest way to compare the knowledge connection path with the knowledge point sequence is to use the intersection-union ratio (IU) to determine the degree of difference. The smaller the IU, the greater the degree of difference. In this case, the knowledge connection path and the knowledge point sequence do not need to be of equal length. Of course, calculating the intersection involves comparing two words. The comparison can use word vector distance. If it is less than a preset distance threshold, it can be considered as the same word. After obtaining the degree of difference, the knowledge connection path is classified based on the preset degree of difference range. Each category of knowledge connection path is used as a knowledge skeleton and inserted into the industry knowledge base. At this time, the elements in the industry knowledge base are only word strings, which are very small in size. Regarding the application process of the industry knowledge base, when an access request is received, the knowledge skeleton is input into the large language model. The large language model can expand the given skeleton to obtain specific content and output it as the display content.

[0073] Regarding step S100, the steps of acquiring multi-source text data of the industry, performing entity recognition on the multi-source text data, constructing a knowledge point sequence, and simultaneously determining the distance change sequence of the knowledge point sequence include:

[0074] Acquire multi-source text data from the industry, perform text segmentation on the multi-source text data, and obtain a sequence of text units;

[0075] Entity recognition is performed on each text unit to extract product entities, technology entities, and application entities, and a knowledge point sequence is constructed according to the text order;

[0076] Semantic vectorization of adjacent knowledge points is performed and cosine distance is calculated to generate a distance change sequence;

[0077] The distance change sequence is normalized.

[0078] In one example of the technical solution of this invention, the knowledge point sequence and distance calculation process are described. Multi-source text data from the industry is acquired, and the multi-source text data is segmented to obtain a text unit sequence. Text segmentation is a conventional text processing process and does not require specific explanation. Entity recognition is performed on each text unit to extract product entities, technical entities, and application entities. A knowledge point sequence is then constructed according to the text order. This process is the keyword extraction process. Generally, some thesaurus is predetermined to limit the scope of product entities, technical entities, and application entities. When extracting keywords, words are first located. If a word belongs to the thesaurus, it is marked as a keyword. Given a known knowledge point sequence, adjacent knowledge points refer to the adjacent relationships in the sequence. Semantic vectorization processing is performed on adjacent knowledge points, and cosine distance is calculated to generate a distance change sequence. Each distance represents the change in a knowledge point; the larger the absolute value, the greater the change. When the value is zero, the knowledge point remains unchanged. If it is necessary to map the distance to other numerical ranges, the data in the distance change sequence can be normalized. In fact, the cosine distance is calculated, but it can also be applied directly without normalization.

[0079] Specifically, regarding the parameters mentioned above, the data sources for multi-source text may include: technical manuals, industry forum discussions, and product documentation, etc. The text segmentation process can be based on paragraph segmentation based on punctuation rules, sliding segmentation based on a fixed window length (such as 300 characters), or topic segmentation based on a semantic segmentation model.

[0080] Regarding step S200, the step of marking key knowledge points in the knowledge point sequence based on the distance change sequence includes:

[0081] Abrupt changes are marked in the distance change sequence based on a preset threshold; wherein, the threshold is connected to an adjustment port;

[0082] Mark the knowledge points corresponding to the mutation points as key knowledge points;

[0083] The processing order and expansion depth of keywords are determined based on the distribution density of key knowledge points in the knowledge point sequence; whereby the processing order is inversely proportional to the distribution density, and the expansion depth is directly proportional to the distribution density.

[0084] In one example of the technical solution of this invention, the marking process of key knowledge points is described. Based on a preset threshold, mutation points are marked in the distance change sequence. That is, when a certain distance reaches the preset threshold, the sequence number is marked as a mutation point. The distance in the distance change sequence is calculated from two knowledge points, corresponding to two knowledge points in the knowledge point sequence. The knowledge point corresponding to the mutation point in the knowledge point sequence is marked as a key knowledge point. It is worth mentioning that even if they are all key knowledge points, they are still different. For any key knowledge point, its distribution density in the knowledge point sequence is queried. The distribution density can be determined by the proportion of keywords in the knowledge point sequence. Of course, a spacing-based adjustment process can also be introduced. For example, if the spacing between two keywords is small enough, they can be considered as the same keyword, and the quantity changes from two to one. If continuous updates are desired, for any keyword, its original quantity is one. The total quantity within a preset range can be queried first, and then a logarithmic function can be applied to the total quantity. As the quantity increases, the total quantity increases, but the rate of increase decreases, thereby reducing the impact of identical adjacent keywords on the quantity growth process.

[0085] The distribution density is directly proportional to the number of keywords. Keywords are sorted according to the distribution density to determine the processing order and expansion depth. The later the sequence number, the higher the processing order (later) and the smaller the expansion depth, and the less expansion is performed.

[0086] Regarding step S300, the step of expanding key knowledge points based on the large language model to generate an expanded knowledge point set includes:

[0087] An extended instruction input large language model is constructed using key knowledge points and their contextual semantic vectors.

[0088] The large language model generates explanatory content, causal relationship content, and application extension content related to key knowledge points;

[0089] Entity recognition and relation extraction are performed on the generated extended content to obtain structural nodes, mechanism nodes and application nodes;

[0090] The nodes are then structured to form an extended knowledge point set.

[0091] In one example of the technical solution of this invention, the process of generating the extended knowledge point set is described in detail. The extended instructions are constructed by inputting the key knowledge points and their contextual semantic vectors into a large language model. The large language model then generates explanatory content, causal relationship content, and application extension content related to the key knowledge points. These are all basic functions of the large language model. Entity recognition and relation extraction are performed on the generated extension content to obtain structural nodes, mechanism nodes, and application nodes. This is equivalent to using the large language model to extract keywords from the content. During extraction, the relationships between nodes are also identified simultaneously. The resulting extended knowledge point set will have similar knowledge points that are more closely related; in simpler terms, they will be grouped together. The keywords in the extended knowledge point set are statistically analyzed in semantic order.

[0092] Regarding step S400, the step of constructing a knowledge graph based on the extended knowledge point set and building a preset number of knowledge connection paths in the knowledge graph includes:

[0093] Nodes in the expanded knowledge point set are used as graph nodes;

[0094] Based on the semantic similarity, causal relationship and hierarchical relationship between nodes, node connection edges are established to construct a knowledge graph;

[0095] In the knowledge graph, multiple knowledge connection paths are generated based on semantic continuity constraints, topic consistency constraints, and logical interpretability constraints;

[0096] Each knowledge connection path is scored based on path coherence, topic coverage, and semantic consistency.

[0097] The technical solution of this invention describes the generation process of knowledge connection paths in detail, which consists of two steps: first, creating a knowledge graph, and then determining paths in the knowledge graph. Specifically, nodes in the extended knowledge point set are used as graph nodes, and node connection edges are established based on semantic similarity, causal relationships, and hierarchical relationships between nodes to construct the knowledge graph. In the knowledge graph, multiple knowledge connection paths are generated based on semantic continuity constraints, topic consistency constraints, and logical interpretability constraints. Each knowledge connection path is scored by path coherence, topic coverage, and semantic consistency.

[0098] In the construction of the knowledge graph, firstly, all nodes in the extended knowledge point set are uniformly represented. Each node includes an entity name, entity type (such as technology, product, application), semantic vector, and contextual source information. Then, node connections are established based on semantic similarity, causal relationships, and hierarchical relationships between nodes. Semantic similarity is calculated using cosine similarity of node semantic vectors, and a threshold (e.g., 0.8) is set to filter high-similarity nodes to form edges. Causal and hierarchical relationships are determined through dependencies extracted by a large language model or predefined rules. Next, nodes and edges are combined into a graph structure G=(V,E), and multiple knowledge connection paths are generated on this graph based on semantic continuity, topic consistency, and logical interpretability constraints. Path generation can adopt various methods. The optimal path is selected by weighting coherence, topic coverage, and semantic consistency using a path scoring function, employing either depth-first or breadth-first traversal. Subsequently, the semantic distribution difference between each path and the original knowledge point sequence is calculated, and paths are classified as low-deviation, medium-deviation, or high-deviation based on the degree of difference, serving as the framework for technical, pedagogical, or application-oriented knowledge, respectively. Finally, all paths are structurally organized according to the framework template, and the content of path nodes is hierarchically optimized and semantically validated using the contextual semantic vectors of the nodes. This ensures the integrity and consistency of the knowledge graph in logic, topic, and semantics, forming a standardized knowledge graph structure that can be directly encapsulated into industry knowledge bases, serving as input to a large language model to generate display content or answers when access requests are made.

[0099] Regarding step S500, the step of calculating the difference between the knowledge connection path and the knowledge point sequence, classifying the knowledge connection path according to the difference, and inserting it as a knowledge skeleton into the industry knowledge base includes:

[0100] Calculate the degree of difference between the semantic distribution of each knowledge connection path and the semantic distribution of the original knowledge point sequence;

[0101] Based on the degree of difference, knowledge connection paths are divided into low deviation paths, medium deviation paths, and high deviation paths;

[0102] Different types of paths are constructed into technical knowledge skeletons, pedagogical knowledge skeletons, and application knowledge skeletons, and then inserted into the industry knowledge base.

[0103] In one example of the technical solution of this invention, the construction process of an industry knowledge base is described. The difference between the semantic distribution of each knowledge connection path and the semantic distribution of the original knowledge point sequence is calculated. The knowledge connection path itself is a word sequence; when compared with the original knowledge point sequence, the degree of difference can be determined. The comparison process requires first extracting features to obtain a numerical form that can be compared. The technical solution of this invention limits the comparable numerical form to a semantic distribution. The semantic distribution can be a fixed word order, and a baseline array is determined based on this fixed word order. Then, for the knowledge connection path and the original knowledge point sequence, the position of each keyword in the baseline array is queried, and the word frequency is inserted into the corresponding position. For other... The keyword position is determined by setting the word frequency to zero. This transforms the knowledge connection path and the original knowledge point sequence into the same data structure. The array difference is then calculated and used as the difference between the knowledge connection path and the original knowledge point sequence. Based on the difference, the knowledge connection path is divided into low-deviation, medium-deviation, and high-deviation paths. Different types of paths are then constructed into technical, pedagogical, and application knowledge skeletons and inserted into the industry knowledge base. This process is unique to this invention; high deviation is not meaningless, but rather treated as other types of content and stored in the industry knowledge base. Furthermore, the industry knowledge base only stores knowledge connection paths, i.e., word combinations, resulting in a very small size.

[0104] Figure 2 A structural diagram of an industry knowledge base construction system based on text-based data synthesis is shown. In a preferred embodiment of the technical solution of the present invention, an industry knowledge base construction system based on text-based data synthesis is also provided, the system 10 comprising:

[0105] Entity recognition module 11 is used to acquire multi-source text data of the industry, perform entity recognition on the multi-source text data, construct a knowledge point sequence, and simultaneously determine the distance change sequence of the knowledge point sequence;

[0106] Key point marking module 12 is used to mark key knowledge points in the knowledge point sequence based on the distance change sequence;

[0107] Knowledge point expansion module 13 is used to expand key knowledge points based on the large language model and generate an expanded knowledge point set;

[0108] The connection path construction module 14 is used to construct a knowledge graph based on the extended knowledge point set and to construct a preset number of knowledge connection paths in the knowledge graph.

[0109] The knowledge skeleton statistics module 15 is used to calculate the difference between knowledge connection paths and knowledge point sequences, classify knowledge connection paths according to the difference, and insert them into the industry knowledge base as knowledge skeletons.

[0110] When an access request is received, the knowledge skeleton is input into the large language model, and the displayed content is output.

[0111] Furthermore, the entity recognition module 11 includes:

[0112] The text segmentation unit is used to acquire multi-source text data from the industry, segment the multi-source text data into text units, and obtain a sequence of text units.

[0113] The sequential statistics unit is used to perform entity recognition on each text unit, extract product entities, technical entities and application entities, and construct a knowledge point sequence according to the text order;

[0114] The distance calculation unit is used to perform semantic vectorization processing on adjacent knowledge points and calculate the cosine distance to generate a distance change sequence.

[0115] A normalization processing unit is used to normalize the distance change sequence.

[0116] Specifically, the key point marking module 12 includes:

[0117] A mutation point marking unit is used to mark mutation points in the distance change sequence based on a preset threshold; wherein the threshold is connected to an adjustment port;

[0118] A marking execution unit is used to mark the knowledge points corresponding to the mutation points as key knowledge points;

[0119] The processing parameter determination unit is used to determine the processing order and expansion depth of keywords based on the distribution density of key knowledge points in the knowledge point sequence; wherein, the processing order is inversely proportional to the distribution density, and the expansion depth is directly proportional to the distribution density.

[0120] Furthermore, the knowledge point expansion module 13 includes:

[0121] The instruction generation unit is used to construct an extended instruction input large language model using key knowledge points and their contextual semantic vectors.

[0122] The content expansion unit is used to generate explanatory content, causal relationship content, and application extension content related to key knowledge points from the large language model;

[0123] The node extraction unit is used to perform entity recognition and relation extraction on the generated extended content to obtain structural nodes, mechanism nodes and application nodes.

[0124] The node statistics unit is used to organize the nodes in a structured way to form an extended knowledge point set.

[0125] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for constructing an industry knowledge base based on text-based data synthesis, characterized in that, The method includes: Acquire multi-source text data from the industry, perform entity recognition on the multi-source text data, construct a knowledge point sequence, and simultaneously determine the distance change sequence of the knowledge point sequence; Key knowledge points are marked in the knowledge point sequence based on the distance change sequence; Based on the large language model, key knowledge points are expanded to generate an expanded knowledge point set; A knowledge graph is constructed based on an expanded set of knowledge points, and a predetermined number of knowledge connection paths are built in the knowledge graph. Calculate the difference between knowledge connection paths and knowledge point sequences, classify knowledge connection paths according to the difference, and insert them into the industry knowledge base as knowledge skeletons; When an access request is received, the knowledge skeleton is input into the large language model, and the displayed content is output. The process of acquiring multi-source text data from the industry, performing entity recognition on the multi-source text data, constructing a knowledge point sequence, and simultaneously determining the content of the distance change sequence of the knowledge point sequence includes: Acquire multi-source text data from the industry, perform text segmentation on the multi-source text data, and obtain a sequence of text units; Entity recognition is performed on each text unit to extract product entities, technology entities, and application entities, and a knowledge point sequence is constructed according to the text order; Semantic vectorization of adjacent knowledge points is performed and cosine distance is calculated to generate a distance change sequence; The distance change sequence is normalized. The step of marking key knowledge points in the knowledge point sequence based on the distance change sequence includes: Abrupt changes are marked in the distance change sequence based on a preset threshold; wherein, the threshold is connected to an adjustment port; Mark the knowledge points corresponding to the mutation points as key knowledge points; The processing order and expansion depth of keywords are determined based on the distribution density of key knowledge points in the knowledge point sequence; whereby the processing order is inversely proportional to the distribution density, and the expansion depth is directly proportional to the distribution density. The steps of calculating the difference between knowledge connection paths and knowledge point sequences, classifying knowledge connection paths based on the difference, and inserting them as knowledge skeletons into the industry knowledge base include: Calculate the degree of difference between the semantic distribution of each knowledge connection path and the semantic distribution of the original knowledge point sequence; Based on the degree of difference, knowledge connection paths are divided into low deviation paths, medium deviation paths, and high deviation paths; Different types of paths are constructed into technical knowledge skeletons, pedagogical knowledge skeletons, and application knowledge skeletons, and then inserted into the industry knowledge base.

2. The method for constructing an industry knowledge base based on text-based data synthesis according to claim 1, characterized in that, The steps for expanding key knowledge points based on a large language model to generate an expanded knowledge point set include: An extended instruction input large language model is constructed using key knowledge points and their contextual semantic vectors. The large language model generates explanatory content, causal relationship content, and application extension content related to key knowledge points; Entity recognition and relation extraction are performed on the generated extended content to obtain structural nodes, mechanism nodes and application nodes; The nodes are then structured to form an extended knowledge point set.

3. The method for constructing an industry knowledge base based on text-based data synthesis according to claim 2, characterized in that, The step of constructing a knowledge graph based on an extended knowledge point set, and constructing a preset number of knowledge connection paths in the knowledge graph, includes: Nodes in the expanded knowledge point set are used as graph nodes; Based on the semantic similarity, causal relationship and hierarchical relationship between nodes, node connection edges are established to construct a knowledge graph; In the knowledge graph, multiple knowledge connection paths are generated based on semantic continuity constraints, topic consistency constraints, and logical interpretability constraints; Each knowledge connection path is scored based on path coherence, topic coverage, and semantic consistency.

4. An industry knowledge base construction system based on text-based data synthesis, characterized in that, The system includes: The entity recognition module is used to acquire multi-source text data of the industry, perform entity recognition on the multi-source text data, construct a knowledge point sequence, and simultaneously determine the distance change sequence of the knowledge point sequence; A key point marking module is used to mark key knowledge points in the knowledge point sequence based on the distance change sequence; The knowledge point expansion module is used to expand key knowledge points based on the large language model and generate an expanded knowledge point set. The connection path construction module is used to build a knowledge graph based on an extended knowledge point set, and to construct a preset number of knowledge connection paths in the knowledge graph; The knowledge skeleton statistics module is used to calculate the difference between knowledge connection paths and knowledge point sequences, classify knowledge connection paths according to the difference, and insert them into the industry knowledge base as knowledge skeletons. When an access request is received, the knowledge skeleton is input into the large language model, and the displayed content is output. The entity recognition module includes: The text segmentation unit is used to acquire multi-source text data from the industry, segment the multi-source text data into text units, and obtain a sequence of text units. The sequential statistics unit is used to perform entity recognition on each text unit, extract product entities, technical entities and application entities, and construct a knowledge point sequence according to the text order; The distance calculation unit is used to perform semantic vectorization processing on adjacent knowledge points and calculate the cosine distance to generate a distance change sequence. A normalization processing unit is used to normalize the distance change sequence; The key point marking module includes: A mutation point marking unit is used to mark mutation points in the distance change sequence based on a preset threshold; wherein the threshold is connected to an adjustment port; A marking execution unit is used to mark the knowledge points corresponding to the mutation points as key knowledge points; The processing parameter determination unit is used to determine the processing order and expansion depth of keywords based on the distribution density of key knowledge points in the knowledge point sequence; wherein, the processing order is inversely proportional to the distribution density, and the expansion depth is directly proportional to the distribution density. The calculation of the difference between knowledge connection paths and knowledge point sequences, the classification of knowledge connection paths based on the difference, and the inclusion of these paths as the knowledge skeleton in the industry knowledge base include: Calculate the degree of difference between the semantic distribution of each knowledge connection path and the semantic distribution of the original knowledge point sequence; Based on the degree of difference, knowledge connection paths are divided into low deviation paths, medium deviation paths, and high deviation paths; Different types of paths are constructed into technical knowledge skeletons, pedagogical knowledge skeletons, and application knowledge skeletons, and then inserted into the industry knowledge base.

5. The industry knowledge base construction system based on text-based data synthesis according to claim 4, characterized in that, The knowledge point expansion module includes: The instruction generation unit is used to construct an extended instruction input large language model using key knowledge points and their contextual semantic vectors. The content expansion unit is used to generate explanatory content, causal relationship content, and application extension content related to key knowledge points from the large language model; The node extraction unit is used to perform entity recognition and relation extraction on the generated extended content to obtain structural nodes, mechanism nodes and application nodes. The node statistics unit is used to organize the nodes in a structured way to form an extended knowledge point set.

Images