Knowledge graph construction method, device, equipment and storage medium

Abstract

The present invention relates to the field of data processing technology, and in particular to a knowledge graph construction method, device, equipment and storage medium. The method comprises: obtaining a general knowledge graph in the industrial field to construct a training data set, and training a slicing model based on the training data set; inputting the acquired private data to be constructed into the slicing model to obtain a text block; obtaining sample data, and inputting the text block and the sample data into a pre-constructed large language model to obtain graph data; generating a weighted graph based on the graph data, and performing graph recursive clustering processing on the weighted graph to obtain community structure information; processing the community structure information using a graph embedding algorithm to obtain a knowledge graph corresponding to the private data to be constructed; the method disclosed in the present application can improve the slicing flexibility and slicing accuracy by pre-training the slicing model when facing various types of industrial private data, so as to balance the information fidelity and time complexity, and greatly improve the quality of the generated text block.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a knowledge graph construction method, device, equipment and storage medium. Background Art

[0002] In the industrial field, the construction of private data knowledge graphs has become a key means to reveal potential problems in the production process, optimize production processes, improve product quality, and reduce production costs; the constructed knowledge graphs provide solid support for intelligent decision-making, enabling companies to respond to market changes and internal challenges more accurately.

[0003] Currently, the latest trend in the industry is to use a method that combines GraphRAG and LLM (Large Language Models) to build a private data knowledge graph in the industrial field. The knowledge graph construction process involves multiple steps such as data preprocessing, graph data generation, and community structure construction. It aims to integrate scattered industrial private data into a unified and structured knowledge graph.

[0004] However, in practical applications, given the diversity and complexity of private data in the industrial field, the process of building a knowledge graph faces many challenges; specifically, existing knowledge graph construction methods have limitations in adaptability. For example, in the data preprocessing stage, relying solely on LLM to perform chunking operations on unstructured data will be limited by the number of LLM tokens and face challenges when processing long texts. Specifically, if a smaller chunking parameter is set to ensure the fidelity of the chunking information, the resource consumption of LLM will be greatly increased. If a larger chunking parameter is set to ensure a smaller time complexity, it will lead to the loss of key information, that is, in the data preprocessing stage, it is difficult to find a balance between information fidelity and time complexity; in addition, in the process of constructing the community structure, existing clustering methods usually adopt a local optimal search strategy, which makes the algorithm easy to fall into the local optimal solution, thereby affecting the accuracy of the information indexing results, resulting in an imbalance between local details and global consistency.

[0005] It can be seen that the existing technology still needs to be improved and enhanced. Summary of the invention

[0006] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a knowledge graph construction method, which has the advantages of flexible and accurate segmentation, can balance information fidelity and time complexity, and greatly improve the quality of the generated text blocks.

[0007] The first aspect of the present invention provides a method for constructing a knowledge graph, comprising: obtaining a general knowledge graph in an industrial field to construct a training data set, and training a segmentation model based on the training data set; obtaining private data to be constructed, and inputting the acquired private data to be constructed into the segmentation model to obtain a text block; obtaining sample data, and inputting the text block and the acquired sample data into a pre-constructed large language model to obtain graph data; generating a weighted graph based on the graph data, and performing graph recursive clustering processing on the weighted graph to obtain community structure information; using a graph embedding algorithm to process the community structure information to obtain a knowledge graph corresponding to the private data to be constructed.

[0008] Optionally, in a first implementation method of the first aspect of the present invention, obtaining a general knowledge graph in the industrial field to construct a training data set includes: obtaining a general knowledge graph in the industrial field, obtaining entity information from the general knowledge graph in the industrial field based on a deep learning method, and calculating the weight of each entity included in the entity information; classifying the entity information according to the calculated weight to obtain classification information; and converting the classification information into vector matrix information for use as a training data set.

[0009] Optionally, in a second implementation manner of the first aspect of the present invention, the training of the chunking model based on the training data set includes: pre-building an initial model based on the ALBERT network, and setting the chunking parameters of the initial model; the initial model includes a first input embedding layer and twelve first Transformer encoders respectively connected to the first input embedding layer, the first Transformer encoder including a multi-head self-attention layer, two feedforward neural network layers and an output layer; the output of the multi-head self-attention layer is respectively input to the two feedforward neural network layers after residual connection and layer normalization processing, and the output of the two feedforward neural network layers is respectively input to the output layer after residual connection and layer normalization processing; a Segment-level loop mechanism and relative position encoding technology are introduced into the initial model to obtain an optimized model; and the training data set is input into the optimized model for training to obtain the chunking model.

[0010] Optionally, in a third implementation manner of the first aspect of the present invention, the step of obtaining the private data to be constructed, inputting the obtained private data to be constructed into the segmentation model, and obtaining a text block includes: obtaining the private data to be constructed, converting the private data to be constructed into vector matrix information to be constructed, and calculating token data of the private data to be constructed; obtaining token restriction information of a pre-constructed large language model, and confirming segmentation information based on the token data and the token restriction information; adjusting segmentation parameters of the segmentation model based on the segmentation information, and inputting the vector matrix information to be constructed into the adjusted segmentation model to obtain a text block corresponding to the private data to be constructed.

[0011] Optionally, in a fourth implementation manner of the first aspect of the present invention, the acquiring of example data, inputting the text block and the acquired example data into a pre-built large language model to obtain graph data, comprises: acquiring a pre-built first LLM large language model, the pre-built first LLM large language model comprising a second input embedding layer, six second Transformer encoders respectively connected to the second input embedding layer, a third input embedding layer, six Transformer decoders respectively connected to the third input embedding layer, a linear layer and a softmax layer; inputting the text block into the second input embedding layer, the outputs of the second input embedding layer are respectively input into the stacked six second Transformer encoders, and in the second Transformer encoder, they are sequentially processed through a multi-head self-attention layer, a residual connection, a layer normalization operation, a feedforward neural network layer, a residual connection and a layer normalization operation. to obtain a first key vector and a first value vector; obtain sample data, input the sample data into a third input embedding layer, and input the output of the third input embedding layer into six stacked Transformer decoders. In the Transformer decoder, the output of the third input embedding layer is subjected to a multi-head self-attention layer to be masked, a residual connection, and a layer normalization operation, and then integrated with the first key vector and the first value vector and input into the multi-head self-attention layer, and then sequentially subjected to a residual connection, a layer normalization operation, a feedforward neural network layer, a residual connection, and a layer normalization operation to obtain an output vector; the output vector is processed in sequence by a linear layer and a softmax layer to obtain a graph element; obtain a pre-constructed second LLM large language model, and drive the second LLM large language model to perform text conversion analysis on the graph element to obtain an element summary corresponding to the graph element; integrate the graph element and its corresponding element summary to obtain graph data.

[0012] Optionally, in a fifth implementation manner of the first aspect of the present invention, a weighted graph is generated based on the graph data, and graph recursive clustering is performed on the weighted graph to obtain community structure information, including: based on the graph data, an undirected weighted graph corresponding to the private data to be constructed is constructed, and the undirected weighted graph is initialized using the Leiden algorithm; the initialized undirected weighted graph is divided into multiple batches, and the undirected weighted graph of each batch is iteratively updated by combining the Leiden algorithm and the Mini Batch SGD algorithm; according to the iteration results, the node similarity is calculated, and the nodes in the undirected weighted graph are divided into different communities according to the node similarity to obtain community classification information; when a preset iteration stop condition is met, all community classification information in the iteration process is integrated to obtain community structure information.

[0013] Optionally, in a sixth implementation manner of the first aspect of the present invention, the community structure information is processed by a graph embedding algorithm to obtain a knowledge graph corresponding to the private data to be constructed, including: obtaining a pre-trained node2vec model and a pre-trained Skip-gram model; inputting the community structure information into the node2vec model to obtain node vector information corresponding to the community structure information; inputting the node vector information into the Skip-gram model to obtain relationship vector information corresponding to the node vector information; integrating the community structure information, the node vector information and the relationship vector information to obtain a knowledge graph corresponding to the private data to be constructed.

[0014] The second aspect of the present invention provides a knowledge graph construction device, including: a training module, used to obtain a general knowledge graph in the industrial field to construct a training data set, and train a segmentation model based on the training data set; a segmentation module, used to obtain private data to be constructed, and input the acquired private data to be constructed into the segmentation model to obtain a text block; a processing module, used to obtain sample data, and input the text block and the acquired sample data into a pre-constructed large language model to obtain graph data; a clustering module, used to generate a weighted graph based on the graph data, and perform graph recursive clustering processing on the weighted graph to obtain community structure information; an embedding module, used to process the community structure information using a graph embedding algorithm to obtain a knowledge graph corresponding to the private data to be constructed.

[0015] The third aspect of the present invention provides a knowledge graph construction device, which includes: a memory and at least one processor, wherein instructions are stored in the memory; at least one of the processors calls the instructions in the memory so that the knowledge graph construction device executes each step of the knowledge graph construction method described in any one of the above items.

[0016] The fourth aspect of the present invention provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed by a processor, the various steps of any of the above-mentioned knowledge graph construction methods are implemented.

[0017] The technical solution of the present invention can significantly improve the flexibility and accuracy of segmentation when processing various types of industrial private data through a pre-trained segmentation model; the trained segmentation model can effectively balance the time complexity while ensuring the fidelity of information, thereby greatly improving the quality of the generated text blocks, providing a solid foundation for the subsequent knowledge graph construction, and making the knowledge graph construction process more accurate and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1A logical flow chart of a knowledge graph construction method provided in an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of the structure of a knowledge graph construction device provided in an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of the structure of a knowledge graph construction device provided in an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of the layer structure of the initial model provided by an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of the layer structure of the LLM large language model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present invention provides a method, apparatus, device and storage medium for constructing a knowledge graph. In the present invention, the terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the knowledge graph construction method in the embodiment of the present invention includes:

[0025] 101. Obtain a general knowledge graph in the industrial field to construct a training data set, and train a slicing model based on the training data set;

[0026] 102. Acquire private data to be constructed, and input the acquired private data to be constructed into the block cutting model to obtain a text block;

[0027] 103. Obtain sample data, and input the text block and the obtained sample data into a pre-built large language model to obtain graph data;

[0028] 104. Generate a weighted graph based on the graph data, and perform graph recursive clustering processing on the weighted graph to obtain community structure information;

[0029] 105. Use graph embedding algorithm to process community structure information and obtain the knowledge graph corresponding to the private data to be constructed.

[0030] The present application discloses a knowledge graph construction method, which can significantly improve the flexibility and accuracy of segmentation when processing various types of industrial private data through a pre-trained segmentation model; the trained segmentation model can effectively balance the time complexity while ensuring the fidelity of information, thereby greatly improving the quality of the generated text blocks, providing a solid foundation for the subsequent knowledge graph construction, and making the knowledge graph construction process more accurate and efficient; further, combined with a pre-constructed large language model, high-quality graph data is generated, providing strong support for the construction of the knowledge graph. The large language model has a powerful ability in processing natural language text, can extract rich semantic information from the text, and convert it into graph data, laying a solid foundation for the subsequent knowledge graph generation; further, through graph recursive clustering processing and graph embedding algorithm, the extraction of community structure information and the generation of knowledge graph are realized; graph recursive clustering can deeply explore the potential structure in the data, and the graph embedding algorithm can map high-dimensional graph data to low-dimensional space, retaining its key features, so that the generated knowledge graph is more accurate and useful.

[0031] In the knowledge graph construction method of the embodiment of the present invention, the step of obtaining a general knowledge graph in the industrial field to construct a training data set includes:

[0032] 201. Obtain a general knowledge graph in the industrial field, obtain entity information from the general knowledge graph in the industrial field based on a deep learning method, and calculate the weight of each entity included in the entity information;

[0033] In this embodiment, graph neural network technology is used to obtain entity information from the general knowledge graph in the industrial field. The entity information includes entities, i.e. proper nouns, attributes of entities, and connection relationships between entities; and then the PageRank algorithm is used to calculate the weights corresponding to the entities.

[0034] 202. Classify the entity information according to the calculated weights to obtain classification information;

[0035] In this embodiment, entities are classified according to the calculated weights, so that entity information becomes more orderly and easier to manage, which facilitates subsequent data processing and analysis.

[0036] 203. Convert the classification information into vector matrix information used as a training data set;

[0037] In this embodiment, the vector matrix corresponding to the entity can be expressed as:

[0038] ;

[0039] in, Represents a triple The vector matrix formed by Indicates an entity (a specific term in the industry), Represents the set of all entities, Represents the relationship between entities in the general knowledge graph of the industry, represents the set of all relations, Indicates the category into which the entity is divided.

[0040] See also Figure 4 In the knowledge graph construction method of the embodiment of the present invention, the training of the slicing model based on the training data set includes:

[0041] 301. Pre-construct an initial model based on the ALBERT network, and set the block parameters of the initial model; the initial model includes a first input embedding layer and twelve first Transformer encoders respectively connected to the first input embedding layer, the first Transformer encoder includes a multi-head self-attention layer, two feedforward neural network layers and an output layer; the output of the multi-head self-attention layer is respectively input to the two feedforward neural network layers after residual connection and layer normalization processing, and the output of the two feedforward neural network layers is respectively input to the output layer after residual connection and layer normalization processing;

[0042] In this embodiment, the activation function of the first input embedding layer is a GELU activation function. By using the GELU activation function, the relation embedding output and the type embedding output of the first input embedding layer can be made smoother. The block parameters can be pre-set by the designer according to actual task requirements.

[0043] In this embodiment, based on the initial model pre-built by the ALBERT network, ALBERT, as a lightweight variant of BERT, effectively reduces the number of model parameters while maintaining high performance through techniques such as parameter sharing and factor decomposition embedding matrix; this optimization not only improves the basic performance and generalization ability of the model, but also reduces the demand for computing resources, making the model more suitable for resource-constrained environments; each of the twelve Transformer encoders included in the initial model is equipped with a multi-head self-attention layer, so that the model can capture complex relationships and patterns in the input data, and the output of the multi-head self-attention layer is processed by residual connection and layer normalization, which further improves the stability and training efficiency of the model; subsequently, these processed outputs are sent to two feedforward neural network layers, which further refine the feature representation by transmitting information layer by layer; finally, after another residual connection and layer normalization, the output layer generates the final model output.

[0044] In this embodiment, based on the parameter sharing mechanism of ALBERT, the query vector of each word input is , key vector Sum value vector have:

[0045] ;

[0046] ;

[0047] in, Representation vector , , The unified transformation matrix, , , Respectively indicate the activated , , vector; let the number of samples processed in parallel be , the number of attention heads is , the number of positions in the sequence is , the dimension of the feature vector of each head is , using Transformer's multi-head attention mechanism , get the segmented query vector , key vector Sum value vector :

[0048] ;

[0049] ;

[0050] ;

[0051] Then the relationship after activation Weighted Sum , Indicates the first Line Column, the obtained relation attention score Integrate into the knowledge graph relationship and preset basic attention score Activate together to get an enhanced attention score , attention weight It is obtained by mask processing and softmax function calculation, specifically:

[0052] ;

[0053] ;

[0054] ;

[0055] ;

[0056] ;

[0057] in, represents the scaling factor, represents the attention mask matrix, represents the enhanced relation embedding output, , represents the relation embedding matrix, represents the relationship ID matrix, GELU represents the activation function; bring in the attention weight , the knowledge-enhanced context vector :

[0058] ;

[0059] ;

[0060] ;

[0061] in, represents the basic context vector, represents the relation context vector, Indicates that the matrix is Line A node of a column; the node refers to a specific position in a matrix structure, which is determined by two dimensions: row and column; specifically, each node corresponds to a unique coordinate point in the matrix, which is identified by its row number and column number;

[0062] Next, activate the merged longs And output transformation , and then The input is processed in two feedforward neural network layers in sequence, and the output results are recorded as , :

[0063] ;

[0064] ;

[0065] ;

[0066] ;

[0067] in, represents the output of the first layer, Represents the output result of the second layer; represents the long-term merger weight, , The two represent the weights of the feedforward neural network layer, and Represent the biases of the two feedforward neural network layers respectively;

[0068] Finally, through residual connection and layer normalization operations, the final result is output , complete the pre-training of the ALBERT model:

[0069] .

[0070] 302. Introduce Segment-level loop mechanism and relative position encoding technology into the initial model to obtain an optimized model;

[0071] In this embodiment, the segment-level recurrence mechanism and relative position encoding technology are introduced to optimize the initial model, that is, the Transformer framework of ALBERT is integrated with the Transformer-XL framework; the segment-level recurrence mechanism allows the model to better handle long-distance dependency problems, while the relative position encoding technology provides the model with sensitivity to the relative positions of words in the input sequence. The combination of these two technologies significantly improves the model's ability to understand language structure and contextual relationships.

[0072] In this embodiment, during the process of extracting industrial private data text, the present invention integrates the relative position encoding technology of the Transformer-XL framework, so that the position information of the long text can be retained when embedding words; specifically, using the shared query matrix and key matrix , and the relative position representation matrix , relative position attention The calculation is as follows:

[0073] ;

[0074] in, Relative position encoding. Indicates the relative relationship between positions. , Indicates the input encoding. , X is the input matrix; , represents the learnable position bias vector, represents transposition; further, the present invention also introduces the segmented memory mechanism of the Transformer-XL framework, so that ALBERT can achieve context migration; specifically, the segmented memory mechanism splices the first Tier The sequence of memory states of the stage cache and hidden state ( represents sequence splicing), and then combined , input Transformer encoder to get the Tier Hidden state of the stage :

[0075] ;

[0076] Next, we integrate the memory-enhanced multi-head attention mechanism into the ALBERT model to improve the self-attention mechanism:

[0077] ;

[0078] ;

[0079] ;

[0080] ;

[0081] With one of the heads For example, ,in, , , Respectively represent the head , , The corresponding weight matrix is, Indicates the operation of the attention mechanism; Represents the projection matrix of the multi-head attention mechanism output, , , Represents the linear transformation of query, key and value respectively. The hidden state of the layer, Indicates The number of attention heads in the layer, Represents the output of splicing all heads; linear transformation parameters ( ) All layers share constraints, , , Respectively represent the query, key and value linear transformation parameters, Represents the final output obtained by merging the outputs of multiple attention heads;

[0082] Then, through the joint loss function and the accumulated clipped gradient Optimize training objectives:

[0083] ;

[0084] ;

[0085] in, , , represents the loss weight, represents the masked language model loss, represents the sentence order prediction loss, represents the autoregressive language model loss, Indicates the parameters Find the partial derivative, express The gradient of the moment; finally, based on the input token and memory buffer, the cache state of the hidden state at the current moment is updated.

[0086] 303. Input the training data set into the optimization model for training to obtain a slicing model;

[0087] In this embodiment, the slicing model is pre-trained using the structural information in the general knowledge graph in the industrial field, which can improve the slicing model's ability to understand and apply industrial knowledge. Specifically, the general knowledge graph in the industrial field is first represented as a vector with a triplet of structural information, so that the model can process and understand the information in the knowledge graph. Among them, entities (proper nouns in the industrial field), the relationship between entities in the knowledge graph, and the classification of entities are all converted into vector form for subsequent processing by the model. Next, in the input embedding layer calculation process, the Transformer based on the GELU activation function is used to activate the word embedding and position encoding to convert the input tokenID matrix into a vector form that the model can understand, and input it into the ALBERT model. The ALBERT model is a pre-trained deep bidirectional encoder that can capture the correlation between words and generate high-quality word embeddings. In the knowledge graph enhancement operation process, the GELU activation function is used to make the relationship embedding output and the type embedding output smoother, thereby enhancing the model's ability to understand the relationship between entities and entity classification information in the knowledge graph. This improves the performance of the model. In addition, based on ALBERT's parameter sharing mechanism, the query vector, key vector and value vector of each input word can be uniformly transformed. This transformation enables the model to more effectively capture the correlation between words and generate high-quality context vectors. When processing samples in parallel, by pre-setting the number of parallel processing samples, the number of attention heads, the number of positions in the sequence and the dimension of the feature vector of each head, and using the Transformer's multi-head attention mechanism, the model can process the segmented query vector, key vector and value vector, thereby capturing richer contextual relationships. After obtaining the relational attention scores, these scores are integrated into the knowledge graph relationships by weighted summation, and activated together with the basic attention scores to obtain enhanced attention scores, so that the model can more accurately capture the relationships between entities and the classification information of entities. Finally, the model integrates the processed information and outputs the final result through residual connections and layer normalization techniques. This process not only completes ALBERT pre-training, but also ensures the efficiency and accuracy of the model in industrial knowledge graph enhancement tasks.

[0088] In the knowledge graph construction method of the embodiment of the present invention, the step of obtaining the private data to be constructed and inputting the obtained private data to be constructed into the block cutting model to obtain the text block includes:

[0089] 401. Obtain the private data to be constructed, convert the private data to be constructed into vector matrix information to be constructed, and calculate the token data of the private data to be constructed;

[0090] In this embodiment, the calculated token data includes the number of tokens of entities, the number of tokens of relationships, and the number of tokens of levels.

[0091] 402. Obtain token restriction information of a pre-built large language model, and confirm block information based on the token data and the token restriction information;

[0092] In this embodiment, the token restriction information is the token budget of the LLM large language model. A pre-built hierarchy adjustment function is obtained. According to the number of tokens of the entity, the number of tokens of the relationship, the number of tokens of the hierarchy, and the token budget of the LLM large language model, the hierarchy adjustment function is used to calculate the block information, and the block information is the number of blocks.

[0093] In this embodiment, an adaptive slicing parameter adjustment algorithm is proposed to achieve a balance between information fidelity and time resource consumption. First, the number of tokens of entities, relationships, and levels in the private data to be constructed is calculated respectively:

[0094] ;

[0095] ;

[0096] in, Represents a single entity The number of tokens, Indicates the basic token number of the entity. represents the entity complexity coefficient (in this embodiment, it is 2), Representing Entities The degree (i.e. total number of relationships), Indicates the total number of tokens of all entities. Indicates the total number of entities;

[0097] ;

[0098] ;

[0099] in, Represents a single relationship The number of tokens, Represents an entity, The basic token number representing the relationship, represents the relational complexity coefficient, represents the relationship weight, Indicates the total number of tokens of all relations, Indicates the total number of relationships (degree);

[0100] ;

[0101] in The number of tokens in the representation layer. Indicates The height coefficient of the layer, Indicates The number of entities contained in the layer; the total number of tokens It can be expressed as:

[0102] .

[0103] Next, we use the restriction on the number of tokens imposed by the first LLM large language model and combine it with hierarchical optimization to calculate the minimum number of tokens. and performance parameters :

[0104] ;

[0105] in, Indicates the token budget of the first LLM large language model. represents the budget coefficient (in this embodiment, ), Indicates vocabulary;

[0106] ;

[0107] in, represents the optimal level, represents the average branching factor, and Represent the total number of entities and the total number of relationships, respectively. Indicates the density of the relationship; It is an operation of rounding up, that is, taking the smallest integer greater than or equal to the integer in the brackets;

[0108] Minimize the number of tokens The value range of needs to meet the following three conditions at the same time: Twh≤Tbu, < < , and Ps ≥ , the specific value can be determined in advance by the designer according to the specific situation; among them, , Represents the minimum number of layers, the maximum number of layers, and the number of layers after optimization Should be within this range, Indicates the minimum value of performance parameters. The calculation formula is as follows:

[0109] ;

[0110] in, represents the knowledge tree search time, Indicates the search hit rate, Indicates the level balance (in this embodiment, it is 0.5);

[0111] Finally, the layer adjustment function is introduced to calculate the corresponding expected total number of layers , and derive the new total number of tokens after adjustment , to achieve dynamic adjustment of the number of tokens, that is, to obtain the block information corresponding to the private data to be constructed:

[0112] ;

[0113] ;

[0114] in, represents the target performance score, Indicates the current layer performance score, Indicates the total number of tokens before adjustment. represents the adjustment coefficient (0.3 in this embodiment).

[0115] 403. Adjust the block parameters of the block model based on the block information, and input the vector matrix information to be constructed into the adjusted block model to obtain a text block corresponding to the private data to be constructed;

[0116] In this embodiment, segmentation information is generated according to the token data of the private data to be constructed and the token budget of the first LLM large language model to achieve adaptive adjustment of the text segmentation scale, that is, the segmentation size can be dynamically adjusted, the segmentation strategy of the segmentation model is optimized, the fidelity of the information is ensured, and a balance point is reached in the consumption of time resources, so that the segmentation model can maintain high efficiency and accuracy when processing texts of different lengths and complexities.

[0117] See also Figure 5 In the knowledge graph construction method of the embodiment of the present invention, the step of obtaining sample data and inputting the text block and the obtained sample data into a pre-built large language model to obtain graph data includes:

[0118] 501. Obtain a pre-constructed first LLM large language model, wherein the pre-constructed first LLM large language model includes a second input embedding layer, six second Transformer encoders respectively connected to the second input embedding layer, a third input embedding layer, six Transformer decoders respectively connected to the third input embedding layer, a linear layer, and a softmax layer;

[0119] 502. Input the text block into the second input embedding layer, and the outputs of the second input embedding layer are respectively input into six stacked second Transformer encoders. In the second Transformer encoder, the text block is processed by a multi-head self-attention layer, a residual connection, a layer normalization operation, a feedforward neural network layer, a residual connection, and a layer normalization operation in sequence to obtain a first key vector and a first value vector.

[0120] 503. Obtain sample data, input the sample data into a third input embedding layer, input the output of the third input embedding layer into six stacked Transformer decoders, and in the Transformer decoder, after the output of the third input embedding layer is subjected to a multi-head self-attention layer to be masked, a residual connection, and a layer normalization operation, the output is integrated with the first key vector and the first value vector and input into the multi-head self-attention layer, and then sequentially subjected to a residual connection, a layer normalization operation, a feedforward neural network layer, a residual connection, and a layer normalization operation to obtain an output vector;

[0121] 504. The output vector is processed in sequence by using a linear layer and a softmax layer to obtain a graph element;

[0122] 505. Obtain a pre-built second LLM large language model, and drive the second LLM large language model to perform text conversion analysis on the graph element to obtain an element summary corresponding to the graph element;

[0123] In this embodiment, the pre-constructed second LLM large language model is an existing large language model.

[0124] 506. Integrate the graph elements and their corresponding element summaries to obtain graph data.

[0125] In this embodiment, by adopting the pre-built first LLM large language model, the efficiency of processing text data is greatly improved; the first LLM large language model can not only quickly receive and analyze text blocks, but also, through its internal structure, can deeply mine the key information in the text; specifically, the text block is first input into the second input embedding layer, which converts the text into numerical form, and then the numerical information is passed to six closely connected second Transformer encoders; in these encoders, the text data undergoes a series of complex processing steps, including fine analysis of multi-head self-attention layers, residual connections to ensure the continuity of information, and layer normalization operations to stabilize the training process of the model, which work together to enable the model to generate high-quality first key vectors and first value vectors, which are the key to understanding the content of the text ; In order to further enhance the performance of the model, sample data is introduced and input into the third input embedding layer. After a similar processing flow, rich output vectors are generated. These vectors are integrated with the previously generated key vectors and value vectors. Through further processing by the multi-head self-attention layer, it is ensured that the model can capture the complex relationship between the data; finally, through fine-tuning of the linear layer and the softmax layer, the output vectors are converted into specific graph elements, which are a direct reflection of the model's understanding of the text content; next, the pre-built second LLM large language model is obtained and driven to perform in-depth text conversion analysis and processing on the graph elements to generate corresponding element summaries. The generated element summaries not only summarize the core content of the graph elements, but also provide a solid foundation for subsequent analysis and application.

[0126] In the knowledge graph construction method of the embodiment of the present invention, generating a weighted graph based on the graph data, performing graph recursive clustering processing on the weighted graph, and obtaining community structure information includes:

[0127] 601. Based on the graph data, construct an undirected weighted graph corresponding to the private data to be constructed, and use the Leiden algorithm to initialize the undirected weighted graph;

[0128] In this embodiment, firstly, a set of vertices in the graph data is identified, and a set of edges in the graph data is determined, including the weight of each edge; then an empty undirected weighted graph structure is created, which can store vertices and their corresponding weights; all vertices in the graph data are traversed, and they are added to the private data graph, and then all edges in the graph data are traversed, and for each edge, its starting point and end point are checked to see if they already exist in the graph, and if so, the corresponding edge is added to the graph according to the weight; finally, each edge is ensured to be undirected, that is, for any edge (u, v), the edge (v, u) is added at the same time to complete the construction of the undirected weighted graph; further, the Leiden algorithm is used to initialize the undirected weighted graph, which not only improves the processing speed, but also ensures the accuracy and reliability of data processing, laying a solid foundation for subsequent data analysis and mining.

[0129] 602. Divide the initialized undirected weighted graph into multiple batches, and iteratively update the undirected weighted graph of each batch by combining the Leiden algorithm and the Mini Batch SGD algorithm;

[0130] In this embodiment, combining the powerful ability of the Leiden algorithm in graph structure learning with the efficient computing characteristics of the Mini Batch SGD algorithm when processing large-scale data sets, the technical solution disclosed in this application can quickly and accurately iterate and update the undirected weighted graph of each batch, which not only significantly improves the computing efficiency, but also ensures that the model can capture the complex structure and patterns in the graph data during the learning process; in addition, due to the use of random sampling and batch processing methods, the technical solution of this application is more economical in memory usage, enabling it to process larger data sets without being troubled by the memory limitations of traditional algorithms, and increases the diversity of data and reduces the risk of falling into local optimality.

[0131] 603. Calculate node similarity according to the iteration result, and divide the nodes in the undirected weighted graph into different communities according to the node similarity to obtain community classification information;

[0132] In this embodiment, the node similarity is calculated to reveal the intrinsic connection between data nodes, so as to facilitate the effective division of nodes in the undirected weighted graph into different communities.

[0133] 604. When a preset iteration stop condition is met, all community classification information in the iteration process is integrated to obtain community structure information;

[0134] In this embodiment, the preset iteration stop condition may be a preset maximum number of iterations, that is, when the preset maximum number of iterations is reached, the iteration is stopped to obtain community structure information.

[0135] In the knowledge graph construction method of the embodiment of the present invention, the community structure information is processed by using a graph embedding algorithm to obtain a knowledge graph corresponding to the private data to be constructed, including:

[0136] 701. Obtain a pre-trained node2vec model and a pre-trained Skip-gram model;

[0137] 702. Input the community structure information into the node2vec model to obtain node vector information corresponding to the community structure information;

[0138] In this embodiment, the node2vec model can effectively capture the complex relationship between nodes, thereby obtaining node vector information that is highly corresponding to the community structure information; the node2vec model can map the nodes in the network into a low-dimensional space by simulating random walks, while retaining the local and global structural information of the nodes.

[0139] 703. Input the node vector information into the Skip-gram model to obtain the relationship vector information corresponding to the node vector information;

[0140] In this embodiment, the node vector information is input into the Skip-gram model, which further mines the potential relationships between nodes and generates relationship vector information that is highly matched with the node vector information. The Skip-gram model focuses on predicting words in the context. In this way, it can learn the subtle relationships between node vectors, thereby capturing richer and more accurate relationship features.

[0141] 704. Integrate the community structure information, the node vector information, and the relationship vector information to obtain a knowledge graph corresponding to the private data to be constructed;

[0142] In this embodiment, community structure information, node vector information, and relationship vector information are comprehensively considered to construct a comprehensive and accurate knowledge graph corresponding to the private data to be constructed; the construction of the knowledge graph not only helps to reveal implicit patterns and associations in the data, but also provides strong support for various applications, such as intelligent search, recommendation systems, and natural language understanding; by constructing a knowledge graph, data can be better understood and utilized, providing users with more intelligent and personalized services.

[0143] The above describes the knowledge graph construction method in the embodiment of the present invention. The following describes the knowledge graph construction device in the embodiment of the present invention. Figure 2 , an embodiment of the knowledge graph construction device in the embodiment of the present invention includes:

[0144] A training module 801 is used to obtain a general knowledge graph in an industrial field to construct a training data set, and train a slicing model based on the training data set;

[0145] A block cutting module 802 is used to obtain the private data to be constructed, and input the obtained private data to be constructed into the block cutting model to obtain a text block;

[0146] The processing module 803 is used to obtain sample data, input the text block and the obtained sample data into a pre-built large language model to obtain graph data;

[0147] A clustering module 804 is used to generate a weighted graph based on the graph data, and perform graph recursive clustering processing on the weighted graph to obtain community structure information;

[0148] The embedding module 805 is used to process the community structure information using a graph embedding algorithm to obtain a knowledge graph corresponding to the private data to be constructed.

[0149] Based on the same idea as the method in the above embodiment, the device provided by the present application can implement the method in the above embodiment.

[0150] above Figure 2 The knowledge graph construction device in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the knowledge graph construction device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0151] Figure 3 It is a structural schematic diagram of a knowledge graph construction device provided by an embodiment of the present invention. The knowledge graph construction device 900 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 910 (for example, one or more processors) and memory 920, and one or more storage media 930 (for example, one or more massive storage devices) storing application programs 933 or data 932. Among them, the memory 920 and the storage medium 930 can be short-term storage or persistent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the knowledge graph construction device 900. Furthermore, the processor 910 can be configured to communicate with the storage medium 930, and execute a series of instruction operations in the storage medium 930 on the knowledge graph construction device 900 to implement the steps of the knowledge graph construction method provided in the above-mentioned method embodiments.

[0152] The knowledge graph construction device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input and output interfaces 960, and / or one or more operating systems 931, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that Figure 3 The structure of the knowledge graph construction device shown does not constitute a limitation of the knowledge graph construction device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0153] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the knowledge graph construction method.

[0154] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0156] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A knowledge graph construction method, characterized in that: include: Obtain a general knowledge graph in the industrial field to construct a training data set, and train a slicing model based on the training data set; including: pre-constructing an initial model based on the ALBERT network, and setting the slicing parameters of the initial model; the initial model includes a first input embedding layer and twelve first Transformer encoders respectively connected to the first input embedding layer, the first Transformer encoder includes a multi-head self-attention layer, two feedforward neural network layers and an output layer; the output of the multi-head self-attention layer is respectively input to the two feedforward neural network layers after residual connection and layer normalization processing, and the output of the two feedforward neural network layers is respectively input to the output layer after residual connection and layer normalization processing; Segment-level loop mechanism and relative position encoding technology are introduced into the initial model to obtain an optimized model; the training data set is input into the optimized model for training to obtain a slicing model; Acquire the private data to be constructed, and input the acquired private data to be constructed into the block cutting model to obtain a text block; The method comprises: obtaining sample data, inputting the text block and the obtained sample data into a pre-built large language model to obtain graph data; comprising: obtaining a pre-built first LLM large language model, wherein the pre-built first LLM large language model comprises a second input embedding layer, six second Transformer encoders respectively connected to the second input embedding layer, a third input embedding layer, six Transformer decoders respectively connected to the third input embedding layer, a linear layer and a softmax layer; inputting the text block into the second input embedding layer, and inputting the outputs of the second input embedding layer into the stacked six second Transformer encoders respectively, and in the second Transformer encoder, sequentially undergoing multi-head self-attention layer processing, residual connection, layer normalization operation, feedforward neural network layer, residual connection and layer normalization operation to obtain a first key vector and a first value vector; obtain sample data, input the sample data into the third input embedding layer, the output of the third input embedding layer is input into the stacked six Transformer decoders, in the Transformer decoder, the output of the third input embedding layer is subjected to the multi-head self-attention layer to be masked, the residual connection and the layer normalization operation, and then integrated with the first key vector and the first value vector and input into the multi-head self-attention layer, and then sequentially subjected to the residual connection, the layer normalization operation, the feedforward neural network layer, the residual connection and the layer normalization operation to obtain the output vector; the output vector is processed in sequence by the linear layer and the softmax layer to obtain the graph element; obtain the pre-constructed second LLM large language model, and drive the second LLM large language model to perform text conversion analysis on the graph element to obtain the element summary corresponding to the graph element; integrate the graph element and its corresponding element summary to obtain the graph data; generating a weighted graph based on the graph data, and performing graph recursive clustering processing on the weighted graph to obtain community structure information; The community structure information is processed using a graph embedding algorithm to obtain a knowledge graph corresponding to the private data to be constructed.

2. The knowledge graph construction method according to claim 1, characterized in that: The obtaining of a general knowledge graph in the industrial field to construct a training data set includes: Obtain a general knowledge graph in the industrial field, obtain entity information from the general knowledge graph in the industrial field based on a deep learning method, and calculate the weight of each entity included in the entity information; Classify the entity information according to the calculated weights to obtain classification information; The classification information is converted into vector matrix information used as a training data set.

3. The knowledge graph construction method according to claim 2, characterized in that: The step of obtaining the private data to be constructed and inputting the obtained private data to be constructed into the block cutting model to obtain a text block includes: Obtain the private data to be constructed, convert the private data to be constructed into vector matrix information to be constructed, and calculate the token data of the private data to be constructed; Obtaining token restriction information of a pre-built large language model, and confirming block information based on the token data and the token restriction information; The block parameters of the block model are adjusted based on the block information, and the vector matrix information to be constructed is input into the adjusted block model to obtain a text block corresponding to the private data to be constructed.

4. The knowledge graph construction method according to claim 1, characterized in that: The step of generating a weighted graph based on the graph data and performing graph recursive clustering processing on the weighted graph to obtain community structure information includes: Based on the graph data, construct an undirected weighted graph corresponding to the private data to be constructed, and initialize the undirected weighted graph using the Leiden algorithm; The initialized undirected weighted graph is divided into multiple batches, and the Leiden algorithm and Mini Batch SGD algorithm are combined to iteratively update the undirected weighted graph of each batch; According to the iteration results, the node similarity is calculated, and the nodes in the undirected weighted graph are divided into different communities according to the node similarity to obtain community classification information; When the preset iteration stop condition is met, all community classification information in the iteration process is integrated to obtain the community structure information.

5. The knowledge graph construction method according to claim 1, characterized in that: The community structure information is processed by using a graph embedding algorithm to obtain a knowledge graph corresponding to the private data to be constructed, including: Get the pre-trained node2vec model and pre-trained Skip-gram model; Inputting the community structure information into the node2vec model to obtain node vector information corresponding to the community structure information; Inputting the node vector information into the Skip-gram model to obtain the relationship vector information corresponding to the node vector information; The community structure information, the node vector information, and the relationship vector information are integrated to obtain a knowledge graph corresponding to the private data to be constructed.

6. A knowledge graph construction device, characterized in that: include: A training module is used to obtain a general knowledge graph in the industrial field to construct a training data set, and train a slicing model based on the training data set; including: pre-constructing an initial model based on the ALBERT network, and setting the slicing parameters of the initial model; the initial model includes a first input embedding layer and twelve first Transformer encoders respectively connected to the first input embedding layer, the first Transformer encoder includes a multi-head self-attention layer, two feedforward neural network layers and an output layer; the output of the multi-head self-attention layer is respectively input to the two feedforward neural network layers after residual connection and layer normalization processing, and the output of the two feedforward neural network layers is respectively input to the output layer after residual connection and layer normalization processing; Segment-level loop mechanism and relative position encoding technology are introduced into the initial model to obtain an optimized model; the training data set is input into the optimized model for training to obtain a slicing model; A block cutting module, used for obtaining the private data to be constructed, and inputting the obtained private data to be constructed into the block cutting model to obtain a text block; A processing module is used to obtain sample data, input the text block and the obtained sample data into a pre-built large language model to obtain graph data; including: obtaining a pre-built first LLM large language model, the pre-built first LLM large language model includes a second input embedding layer, six second Transformer encoders respectively connected to the second input embedding layer, a third input embedding layer, six Transformer decoders respectively connected to the third input embedding layer, a linear layer and a softmax layer; inputting the text block into the second input embedding layer, and the outputs of the second input embedding layer are respectively input into the stacked six second Transformer encoders, and in the second Transformer encoder, multi-head self-attention layer processing, residual connection, layer normalization operation, feedforward neural network layer, residual connection and layer normalization operation are sequentially performed to obtain a first key vector and the first value vector; obtain sample data, input the sample data into the third input embedding layer, the output of the third input embedding layer is input into the stacked six Transformer decoders, in which the output of the third input embedding layer is subjected to the multi-head self-attention layer to be masked, the residual connection and the layer normalization operation, and then integrated with the first key vector and the first value vector and input into the multi-head self-attention layer, and then sequentially subjected to the residual connection, the layer normalization operation, the feedforward neural network layer, the residual connection and the layer normalization operation to obtain the output vector; the output vector is processed in sequence by the linear layer and the softmax layer to obtain the graph element; obtain the pre-constructed second LLM large language model, and drive the second LLM large language model to perform text conversion analysis on the graph element to obtain the element summary corresponding to the graph element; integrate the graph element and its corresponding element summary to obtain the graph data; A clustering module, used to generate a weighted graph based on the graph data, and perform graph recursive clustering processing on the weighted graph to obtain community structure information; The embedding module is used to process the community structure information using a graph embedding algorithm to obtain a knowledge graph corresponding to the private data to be constructed.

7. A knowledge graph construction device, characterized in that: The knowledge graph construction device includes: a memory and at least one processor, wherein instructions are stored in the memory; At least one of the processors calls the instructions in the memory so that the knowledge graph construction device performs each step of the knowledge graph construction method as described in any one of claims 1-5.

8. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by the processor, each step of the knowledge graph construction method as described in any one of claims 1 to 5 is implemented.

Images