Enterprise knowledge graph construction method, system, device and storage medium

Abstract

The present application relates to the technical field of knowledge graph construction, and in particular to a method, system, device and storage medium for constructing an enterprise knowledge graph. The method includes collecting enterprise data and obtaining a test data set, including enterprise data in each field and its corresponding identified entities. The neural network model is trained with the enterprise data in each field as input and the corresponding identified entities as output to generate a first entity recognition model for enterprises in each field; the first entity recognition model corresponding to the enterprise in each field is fused into a second entity recognition model through knowledge distillation; the enterprise data is input into the second entity recognition model to identify the entities in the enterprise data and extract the relationships between the entities; and the enterprise knowledge graph is constructed based on entities and relationships. The present application can effectively overcome the problem of insufficient entity recognition accuracy caused by field differences in the prior art, enhance the adaptability of the model when facing data from different fields, and thus improve the efficiency and quality of knowledge graph construction.

Description

Technical Field

[0001] The present application relates to the technical field of knowledge graph construction, and in particular to a method, system, device and storage medium for constructing an enterprise knowledge graph. Background Art

[0002] With the rapid growth and diversification of enterprise data, knowledge graphs, as an important knowledge management tool, have played a vital role in helping enterprises tap into the potential value in data, improve decision-making efficiency and the accuracy of information retrieval. By mapping various data and information into structured graphs, knowledge graphs can effectively represent entities, relationships and their attributes, and reveal the complex associations between different data elements. In recent years, with the development of artificial intelligence technology, especially deep learning, the construction process of knowledge graphs has gradually shifted from traditional manual construction to automated and intelligent entity recognition and relationship extraction methods. This shift has promoted the efficiency and intelligence of enterprise knowledge management, especially in big data and multimodal data environments, improving the quality and efficiency of graph construction.

[0003] At present, the process of building an enterprise knowledge graph usually includes key links such as entity recognition, relationship extraction and graph construction. Entity recognition is one of the most basic and important tasks. Existing entity recognition technologies are mostly based on deep learning models, especially pre-trained language models such as recurrent neural networks (RNN), long short-term memory networks (LSTM) and BERT, which can automatically recognize entities in texts through training on a large amount of annotated data.

[0004] However, the main problem with existing entity recognition technology when dealing with multi-domain tasks is that it cannot fully utilize the knowledge of various fields. Due to the large differences in professional terminology and naming conventions in various fields, general entity recognition models often lack adaptability when facing different fields. For example, industries such as medical, financial, and manufacturing have specific terms and entity types. If these terms are not supported by specialized domain knowledge, general models often cannot accurately recognize them, resulting in insufficient entity recognition accuracy. Therefore, existing technologies face great challenges in the accuracy of entity recognition of multi-domain enterprise data, and cannot fully utilize professional knowledge in various fields, which easily affects the efficiency and quality of subsequent knowledge graph construction. Summary of the invention

[0005] This application provides a method, system, device and storage medium for constructing an enterprise knowledge graph, which can effectively overcome the problem of insufficient entity recognition accuracy caused by field differences in the prior art, enhance the adaptability of the model when facing data from different fields, and thus improve the efficiency and quality of knowledge graph construction. This application provides the following technical solutions:

[0006] In a first aspect, the present application provides a method for constructing an enterprise knowledge graph, the method comprising:

[0007] Collect enterprise data and pre-process the enterprise data;

[0008] Obtain a test data set, the test data set including enterprise data in each field and its corresponding identified entities; train a preset neural network model with the enterprise data in each field as input and the corresponding identified entities as output to generate a first entity recognition model corresponding to the enterprises in each field;

[0009] The first entity recognition model corresponding to each field enterprise is fused into the second entity recognition model through knowledge distillation;

[0010] Inputting the enterprise data into a second entity recognition model, identifying entities in the enterprise data, and extracting relationships between the entities;

[0011] Build an enterprise knowledge graph based on the identified entities and extracted relationships.

[0012] In a specific possible implementation scheme, fusing the first entity recognition model corresponding to each field enterprise into the second entity recognition model by means of knowledge distillation includes:

[0013] After generating the first entity recognition model corresponding to each domain enterprise, the knowledge of these domain models is transferred as teacher models to a unified student model through the knowledge distillation process;

[0014] Design a loss function, calculate the gradient of the loss function and update the parameters of the student model through the back-propagation algorithm until the loss function reaches the minimum value or converges, and generate a second entity recognition model.

[0015] In a specific implementation scheme, the loss function is as follows:

[0016]

[0017] Where N is the number of teacher models, λ i is the weight coefficient of the i-th teacher model, and H student Respectively represent the feature representation of the i-th teacher model and the student model at the same layer; represents the multi-scale distillation loss between the feature representation of the i-th teacher model and the student model at each level; α is the parameter for adjusting the weight between the KL divergence loss and the multi-scale distillation loss, P teacher and P student Respectively represent the probability distribution of the teacher model and the student model in the output layer; D KL (P teacher ,Pstudent ) is the KL divergence;

[0018] The calculation formula of multi-scale distillation loss is as follows:

[0019]

[0020] The calculation formula of KL divergence is as follows:

[0021]

[0022] Among them, P teacher (y j ) and P studen t(y j ) represent the teacher model and the student model in the output layer for category y j The probability distribution of .

[0023] In a specific implementation scheme, the loss function is as follows:

[0024]

[0025] Among them, q i (x) k and p i (x) k Respectively represent the predicted probability of category k by the teacher model and the student model under the i-th category; w i is the weighting factor for each category, β is a parameter used to balance the impact between KL divergence and similarity loss, and N represents the total number of categories.

[0026] In a specific implementation scheme, the relationship between the extracted entities includes:

[0027] Using the identified entities as keywords, we build a keyword co-occurrence matrix by scanning each paragraph or sentence in the enterprise data;

[0028] After constructing the co-occurrence matrix, the relationship strength between entities is calculated, and the relationship strength is measured by the co-occurrence frequency;

[0029] After the relationship strength is calculated, entity pairs whose relationship strength is greater than a preset threshold are selected according to the calculated relationship strength;

[0030] The existing dependency parsing method is used to extract the relationship between the above entity pairs.

[0031] In a specific implementation scheme, constructing a keyword co-occurrence matrix includes:

[0032] Each item M in the keyword co-occurrence matrix ij Represents entity E i and entity Ej The number of co-occurrences in the same paragraph or sentence is as follows:

[0033] M ij = count(E i ,E j )

[0034] Among them, M ij Represents entity E i and entity E j The number of times each pair of entities co-occurs in the same paragraph or sentence is calculated by traversing all the texts in the enterprise data, counting the frequency of each pair of entities co-occurring in the same paragraph or sentence, and generating a co-occurrence matrix.

[0035] In a specific embodiment, calculating the relationship strength between entities includes calculating the relationship strength R between each pair of entities using the following formula: ij :

[0036]

[0037] Among them, M ii and M jj Represents entity E i and entity E j The frequency of each occurrence in all paragraphs or sentences.

[0038] In the second aspect, the present application provides an enterprise knowledge graph construction system, which adopts the following technical solutions:

[0039] A system for building an enterprise knowledge graph, comprising:

[0040] A data collection module, used to collect enterprise data and pre-process the enterprise data;

[0041] A model building module is used to obtain a test data set, which includes enterprise data in each field and its corresponding identified entities; the enterprise data in each field is used as input, and the corresponding identified entities are used as output to train a preset neural network model to generate a first entity recognition model corresponding to the enterprises in each field;

[0042] A knowledge distillation module is used to fuse the first entity recognition model corresponding to each field enterprise into a second entity recognition model through knowledge distillation;

[0043] A relationship extraction module, used to input the enterprise data into a second entity recognition model, identify entities in the enterprise data, and extract relationships between the entities;

[0044] The graph construction module is used to build an enterprise knowledge graph based on the identified entities and extracted relationships.

[0045] In a third aspect, the present application provides an electronic device comprising a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement a method for constructing an enterprise knowledge graph as described in the first aspect.

[0046] In a fourth aspect, the present application provides a computer-readable storage medium, in which a program is stored. When the program is executed by a processor, it is used to implement a method for constructing an enterprise knowledge graph as described in the first aspect.

[0047] In summary, the beneficial effects of this application include at least:

[0048] 1) By building independent entity recognition models for each field (such as medical, financial, manufacturing, etc.), we can fully utilize the expertise and terminology of each field, effectively solving the accuracy problem of general entity recognition models when processing field-specific data. The entity recognition model in each field is optimized for specific industry terminology and naming conventions, thereby improving the accuracy of entity recognition.

[0049] 2) Through the knowledge distillation method, the knowledge of multiple domain-specific teacher models (i.e., domain models) is transferred to a unified student model. This process optimizes the loss function and adjusts the model parameters so that the student model can integrate the advantages of models in various fields, improve its adaptability and recognition accuracy in multi-domain tasks, and thus solve the challenge of integrating multi-domain knowledge.

[0050] 3) By designing a combined loss function of weighted cross entropy and KL divergence based on entity similarity, the student model not only learns the output probability distribution of the teacher model during training, but also learns the deep semantic information extracted by the teacher model in the feature space. This enables the student model to more accurately identify the relationship between entities, especially in complex cross-domain scenarios, effectively improving the accuracy and robustness of relationship extraction.

[0051] By building multiple domain-specific entity recognition models and using knowledge distillation to transfer the knowledge of these domain models to a general student model, the entity recognition problem in multi-domain enterprise data is solved. Specifically, a dedicated entity recognition model is first trained for each field (such as medical, financial, manufacturing, etc.), and then the knowledge of each domain model is integrated into a unified model through knowledge distillation technology, so that the student model can recognize entities more accurately in multiple fields. This solution effectively overcomes the problem of insufficient entity recognition accuracy caused by domain differences in the existing technology, enhances the adaptability of the model when facing data from different fields, and improves the efficiency and quality of knowledge graph construction.

[0052] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of the method for constructing an enterprise knowledge graph in an embodiment of the present application.

[0054] Figure 2 It is a schematic diagram of the overall process of the enterprise knowledge graph construction method in the embodiment of the present application.

[0055] Figure 3 It is a structural block diagram of the enterprise knowledge graph construction system in the embodiment of the present application.

[0056] Figure 4 It is a block diagram of an electronic device for constructing an enterprise knowledge graph in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but are not intended to limit the scope of the present application.

[0058] Optionally, the present application uses the enterprise knowledge graph construction method provided in each embodiment as an example for explanation in an electronic device, where the electronic device is a terminal or a server. The terminal may be a computer, a tablet computer, etc. This embodiment does not limit the type of electronic device.

[0059] Reference Figure 1 , is a flow chart of a method for constructing an enterprise knowledge graph provided by an embodiment of the present application, and the method includes at least the following steps:

[0060] Step S101: collect enterprise data and pre-process the enterprise data.

[0061] In step S101, multi-source enterprise data of the target enterprise for which the knowledge graph needs to be constructed is first collected, and necessary preprocessing is performed on the collected enterprise data to ensure the quality and consistency of the data, thereby providing reliable support for subsequent entity recognition, relationship extraction and knowledge graph construction.

[0062] Specifically, this application collects enterprise data from the following ways: First, enterprises use CRM systems (such as Salesforce, HubSpot) to manage customer information. CRM systems usually contain basic customer information (such as name, company name, position, contact information, etc.), as well as interaction records with customers (such as communication history, order records, etc.). Extract data from CRM through API interfaces, or export customer information tables using database queries. Specific customer data can be extracted as needed, such as new customers and key customers within a certain period of time. The data in CRM is usually structured, which is convenient for direct storage in relational databases (such as MySQL). Preprocessing is performed through standardized processing (such as unified time format, contact name format, etc.) to prepare for subsequent entity recognition. Secondly, the contracts and agreements signed by enterprises with suppliers and customers usually contain multiple entities and complex relationships between them. For example, the company name, signing date, contract amount, contracting parties, etc. in the contract. Extract the text content of the contract through automated tools in document management systems (such as SharePoint, Google Drive), or directly through OCR technology (for scanned documents). Contract data may contain a large amount of text, which needs to be cleaned and annotated through natural language processing technology (NLP), such as removing standard terms in the contract and removing noise information. In addition, the entities in the contract (such as company name, signatory, contract amount, etc.) need to be annotated to provide training data for entity recognition and relationship extraction. Finally, the company's human resources system contains employees' basic information, positions, departments, personal information, entry dates, etc. Obtain employee information through the HR system API interface, or directly export Excel tables. The information content usually includes structured fields, such as employee ID, name, position, department, entry date, etc.

[0063] It should be noted that the above data collection methods are all based on the information disclosed by the target enterprise or actively provided by the target enterprise. In practice, this application adopts methods including but not limited to the above methods to collect enterprise data. This application does not limit the method of collecting enterprise data.

[0064] In implementation, after collecting enterprise data, duplicate records in the data are removed. For missing values ​​in the data, they can be processed by filling, interpolating or deleting. Then, extra spaces, punctuation marks, special characters and other irrelevant content in the text are removed to ensure that the text data is clean. Sensitive information (such as personal identity information, financial information, etc.) is desensitized to ensure the privacy and security of the data during use. Finally, the enterprise data is standardized and the data format is unified.

[0065] Step S102: Acquire a test data set, which includes enterprise data in each field and its corresponding identified entities. Take the enterprise data in each field as input and the corresponding identified entities as output to train a preset neural network model, and generate a first entity recognition model corresponding to each field enterprise.

[0066] In step S102, the data of the enterprises that have constructed enterprise graphs in various fields and identified entities are obtained, and the data are trained using a preset neural network model to generate an entity recognition model specific to each field.

[0067] Specifically, we first need to obtain test data from enterprise data in different fields. The test data set includes a large amount of data and identified entities of enterprises in each field. For each field, such as finance, manufacturing, retail, etc., the test data includes the text content of documents, contracts, invoices, emails, etc. that have been made public or authorized to be viewed in the field, as well as the entity annotations therein. Then, the test data set is used to train the preset neural network model with the enterprise data of each field as input and the entity annotations corresponding to the enterprise data as output, so as to obtain the first entity recognition model exclusive to each field.

[0068] Optionally, the existing BERT model is selected as the preset neural network model for training in this application. The advantage of BERT is its bidirectional encoding characteristics, which can capture contextual relationships and is applicable to text data in different fields. Other types of neural network models can also be selected, such as convolutional neural networks or long short-term memory networks. This application does not limit the specific type of neural network model.

[0069] Step S103: The first entity recognition model corresponding to each field enterprise is fused into a second entity recognition model through knowledge distillation.

[0070] In step S103, the purpose of knowledge distillation is to extract knowledge from the first entity recognition model (i.e., teacher model) of multiple domain enterprises and transfer this knowledge to a unified second entity recognition model (i.e., student model). The key to the process is how to use the soft labels of the teacher model (i.e., the probability distribution output by the teacher model) to guide the student model to learn these richer information during the training process, thereby improving its ability to recognize cross-domain entities.

[0071] Specifically, after generating the first entity recognition model corresponding to each field enterprise, the knowledge of these field models is transferred to a unified student model through the existing knowledge distillation process. The specific process of existing knowledge distillation is as follows: the first entity recognition model of each field (i.e., the teacher model, for the convenience of description, the teacher model is used to refer to the first entity recognition model in this step) will generate a set of outputs based on enterprise data in different fields. These outputs include: category probability distribution (i.e., the classification probability of each word or entity) and intermediate layer features, which are usually hidden layer features extracted by the model during training. The above outputs will serve as the learning goal of the student model to help the student model learn entity recognition knowledge in different fields. The training goal of the student model is to make its output as close as possible to the output of all teacher models, and to optimize the performance of the student model by comparing the output probability distribution of the teacher model with the output probability distribution of the student model; on each hidden layer, by comparing the intermediate layer features of the teacher and student models, the student model is guided to learn the useful features extracted at each layer.

[0072] It should be noted that in actual situations, different teacher models (from different fields) may have different numbers of layers or structures. Regardless of whether the specific number of layers of the teacher model and the student model is the same, the key is how to compare their "features" or "knowledge" rather than simply the number of layers. If the structures of the teacher model and the student model are different, the features of the teacher model can be converted to the same dimension as the student model through a mapping mechanism (for example, using an additional network layer). In addition, you can choose to compare the features of the same level of the teacher model and the student model, such as a middle layer of the teacher model and the layer at the same position of the student model, so as to ensure that the student model learns valuable features.

[0073] In addition, as a preferred embodiment, in order to guide the student model to learn the knowledge of the teacher model, the present application designs a loss function, which not only considers the difference between the teacher and the student in the output layer, but also considers the difference between the teacher and the student in the feature representation of the intermediate layer. The specific loss function can be expressed as:

[0074]

[0075] Where N is the number of teacher models, i.e., the number of entity recognition models in different fields used in this application, and λ i is the weight coefficient of the i-th teacher model, which indicates the contribution of the model to the student model training process. and H student They respectively represent the feature representations of the i-th teacher model and the student model at the same layer. represents the multi-scale distillation loss between the i-th teacher model and the student model at each level of feature representation (such as hidden layer, fully connected layer, etc.). α is a parameter that adjusts the weight between KL divergence loss and multi-scale distillation loss to balance the impact of the two. teacher and P student Represent the probability distribution of the teacher model and the student model at the output layer. KL (P teacher ,P student ) is the KL divergence, which is used to measure the difference between the probability distribution of the teacher model and the student model at the output layer. By minimizing the KL divergence, the student model will move closer to the predicted distribution of the teacher model, thereby obtaining more accurate entity recognition capabilities.

[0076] The calculation formula of multi-scale distillation loss is as follows:

[0077]

[0078] The calculation formula of KL divergence is as follows:

[0079]

[0080] Among them, P teacher (y j ) and P student (y j ) represent the teacher model and the student model in the output layer for category y j The probability distribution of . In the above formula, the multi-scale distillation loss is implemented by calculating the cosine similarity between the feature representations of the teacher model and the student model at each layer (such as hidden layer, fully connected layer, etc.). It can measure the similarity between two vectors (i.e., the output of the model at the same layer), avoiding direct element-by-element comparison of the features, thereby better retaining the overall information of the features. The student model can imitate the internal feature representation of the teacher model layer by layer, thereby obtaining a more accurate and detailed knowledge transfer. This helps the student model better capture the deep semantic features of the teacher model during the learning process, rather than just the output of the surface layer. When the two loss terms are optimized together, both the knowledge transfer from the teacher model to the student model is considered (through the multi-scale distillation loss), and the output distribution of the student model is ensured to be consistent with the output distribution of the teacher model (through the KL divergence). This design ensures that the student model is not only close to the teacher model at the implicit feature level, but also highly consistent with the teacher model in the final classification decision, thereby improving the accuracy of entity recognition.

[0081] It should be noted that for and H student, feature representation refers to the high-dimensional representation obtained by the model through the output (or activation value) of each layer during the training process. They reflect the characteristic information of the input data after being processed by each layer. For deep learning models, the output of each layer can be regarded as a feature representation. Represents the feature representation of the i-th teacher model at a certain layer. For example, assuming that the i-th teacher model is processed by several layers, the output (activation value) at a certain hidden layer or fully connected layer can be used as a feature representation. This output is obtained by passing the input data to the neurons in this layer. The value output by each neuron is the activation value of this layer, which is usually calculated using ReLU or other activation functions. student It is the feature representation of the student model at the same layer. The structure of the student model is usually similar to that of the teacher model, or different at some levels. After the student model passes the input data through the same network structure, it outputs activation values ​​at the same level, which are also feature representations.

[0082] In addition, the probability distribution is the predicted probability of each category given by the model in the classification task. Usually, the last layer of the model is a Softmax layer, which converts the final output into a probability distribution. teacher Represents the probability distribution of the teacher model at the output layer. For classification problems, the teacher model will output a vector containing the predicted values ​​for each category based on the input data. After processing, these values ​​are converted into "probabilities" for each category, indicating the likelihood that the input data belongs to each category. The sum of these probabilities is 1. Similarly, the student model will also make predictions on the input data and output a vector representing the predicted probabilities for each category. This vector represents the likelihood that the student model believes the input data belongs to each category.

[0083] In summary, and H student are the intermediate layer outputs (i.e., feature representations) obtained at each layer (such as hidden layer, fully connected layer, etc.) by passing the input data to the teacher model and the student model. teacher and P student The class probability distribution is obtained by passing the input data to the output layer of the teacher model and the student model using the Softmax function. The above operations are all commonly used model forward propagation steps in the prior art, and this application will not be described in detail here.

[0084] Through the loss function designed above, the gradient of the loss function is calculated and the parameters of the student model are updated through the back propagation algorithm until the loss function reaches the minimum value or converges. Finally, the student model converges to a model that can recognize entities in multiple fields, namely the second entity recognition model.

[0085] In another feasible embodiment, the loss function can also be expressed as:

[0086]

[0087] Among them, q i (x) k and p i (x) k Respectively represent the predicted probability of the teacher model and the student model for category k under the i-th category. i is a weighting factor for each category, the purpose of which is to adjust the importance of different categories in the total loss. Different entity categories contribute differently to model training, so we can assign different weights to each category according to the needs of the task. Usually, important categories are given greater weights to ensure that they have a greater impact on the model. β is a parameter used to balance the impact between KL divergence and similarity loss, and N represents the total number of categories, that is, the number of entity categories recognized in the student and teacher models. In the above formula, by combining KL divergence and similarity metrics, the new loss can more comprehensively capture the information in the teacher model. KL divergence ensures that the probability distribution of the student model output is close to that of the teacher model output, while similarity loss ensures that the student model can learn the deep semantic information extracted by the teacher model at the feature level. Through this improved loss function, the student model can not only fit the output of the teacher model, but also be as close to the representation of the teacher model as possible in the feature space, thereby improving the performance and generalization ability of the student model in complex cross-domain tasks. This loss function measures the difference between the teacher model and the student model by comparing their probability distributions at the output layer. What it pays attention to is the final prediction ability of the model, and directly evaluates whether the classification decision of the model on a specific category is consistent. This metric pays attention to the consistency of probability distribution, that is, the output probability of the student model should be as close to the output probability of the teacher model as possible. Therefore, it pays more attention to the final prediction consistency and improves the accuracy of the model in the classification task. Compared with the loss function designed in the previous embodiment, the previous loss function measures the difference between the teacher and the student model by comparing the feature representations of the same layer (such as hidden layer) to measure the similarity of two vectors in the feature space. What this metric pays attention to is the consistency of feature representation, that is, whether the student model can learn the semantic features extracted by the teacher model. Therefore, it pays more attention to the consistency of feature representation, helps the student model learn deeper semantic features from the hidden layer of the teacher model, and improves the generalization ability of the model, especially when processing complex tasks.

[0088] It should be noted that in this application i (x) k and p i (x) kare the output probabilities obtained by the teacher model and the student model through the softmax function, respectively, which represent the predicted probability of the input x for each category k. The operation is a prior art, so this application will not describe it in detail.

[0089] Step S104: input the enterprise data into a second entity recognition model, identify entities in the enterprise data, and extract relationships between entities.

[0090] In step S104, first, the enterprise data is input into the trained second entity recognition model. This model has undergone knowledge distillation of the multi-domain teacher model and has the ability to recognize cross-domain entities. Specifically, the second entity recognition model can identify various entities in the text, such as company name, product, location, time, etc., based on the text information in the enterprise data. The recognition of these entities is based on the cross-domain knowledge learned during the previous training, and can accurately identify entities with different names and features in different fields.

[0091] After entity recognition, the next step is to extract the relationship between entities from the enterprise data. First, use the recognized entities (keywords) to build a keyword co-occurrence matrix by scanning each paragraph or sentence in the enterprise data. Each item M in the keyword co-occurrence matrix ij Represents entity E i and entity E j The number of co-occurrences in the same paragraph or sentence is as follows:

[0092] M ij = count(E i ,E j )

[0093] Among them, M ij Represents entity E i and entity E j The number of times each pair of entities co-occurs in the same paragraph or sentence is calculated by traversing all the texts in the enterprise data, counting the frequency of each pair of entities co-occurring in the same paragraph or sentence, and generating a co-occurrence matrix.

[0094] After constructing the co-occurrence matrix, the relationship strength between entities is calculated. The relationship strength is measured by the co-occurrence frequency. The relationship strength R between each pair of entities is calculated using the following formula: ij :

[0095]

[0096] Among them, M ii and M jj Represents entity E i and entity E jThe frequency of occurrence of each entity in all paragraphs or sentences. The above formula is used to obtain the strength of the relationship between each pair of entities. If entity E i and entity E j If entities often appear in the same paragraph or sentence, the strength of the relationship between them is greater. This value will be used to determine which entities have significant relationships and then extract the relationship between them.

[0097] In another feasible embodiment, the following formula can also be used to calculate the relationship strength R between each pair of entities: ij :

[0098]

[0099] Among them, sim(E i ,E j ) is entity E i and entity E j The cosine similarity between word vectors. By introducing semantic similarity, even if the entity E i and entity E j If they have a high co-occurrence frequency, their relationship strength will not be too high if they are not semantically similar. On the contrary, if their semantic relationship is strong, the relationship strength can be improved even if they have a low co-occurrence frequency. At the same time, the weight of semantic information can be flexibly adjusted. Through semantic weighting, the calculation of relationship strength not only depends on the frequency of entities, but also comprehensively considers their semantic similarity, making the model more accurate in capturing true semantic relationships.

[0100] After the relationship strength is calculated, entity pairs with a relationship strength greater than a preset threshold are selected based on the calculated relationship strength. These entity pairs are considered to have significant relationships. Subsequently, the existing dependency syntactic analysis method is used to extract the relationship between the above entity pairs. The dependency syntactic analysis method can help extract the grammatical relationship between words in a sentence, thereby inferring the relationship between entities. This method first analyzes the structure of the sentence through a dependency syntactic analyzer to find the dependency relationship between each word in the sentence. Then, by analyzing the dependency graph, the relationship between entities is extracted. Specifically, the text is converted into a dependency tree through a dependency syntactic analyzer (such as tools such as Stanford Parser or Spacy), and the relationship between entity pairs is extracted based on the structure of the dependency graph. For example, for the sentence "Company A Supplier B", the analysis result may show that "supplier" as a verb has a dependency relationship with "Company A" and "Supplier B". The verb "supply" is usually used as a relationship identifier to indicate the type of relationship between "Company A" and "Supplier B". The extracted relationship needs to be manually verified by domain experts to ensure the correctness and accuracy of the relationship. Experts can ensure the quality of relational data by reviewing the extracted relations, manually correcting erroneous relations, or supplementing missing relations.

[0101] Step S105: construct an enterprise knowledge graph based on the identified entities and extracted relationships.

[0102] In step S105, based on the entities identified in step S104 and the extracted relationships, an enterprise knowledge graph is constructed. Specifically, all entities identified in step S104 are taken as nodes and classified by entity type. According to the extracted relationships, the entity nodes are connected through relationship edges, and the relationship types and related attributes are marked. Relevant attributes (such as time, place, position, etc.) are added to each entity and relationship edge. The knowledge graph is deduplicated, normalized, and abnormal data is processed to ensure data consistency. The knowledge graph is displayed through a graph visualization tool for further analysis and use.

[0103] In summary, combined with Figure 2 , by building multiple domain-specific entity recognition models and using knowledge distillation to transfer the knowledge of these domain models to a general student model, the entity recognition problem in multi-domain enterprise data is solved. Specifically, a dedicated entity recognition model is first trained for each field (such as medical, financial, manufacturing, etc.), and then the knowledge of each domain model is integrated into a unified model through knowledge distillation technology, so that the student model can recognize entities more accurately in multiple fields. This solution effectively overcomes the problem of insufficient entity recognition accuracy caused by domain differences in the existing technology, enhances the adaptability of the model when facing data from different fields, and improves the efficiency and quality of knowledge graph construction.

[0104] Figure 3 This is a structural block diagram of an enterprise knowledge graph construction system provided by an embodiment of the present application. The device includes at least the following modules:

[0105] A data collection module, used to collect enterprise data and pre-process the enterprise data;

[0106] A model building module is used to obtain a test data set, which includes enterprise data in each field and its corresponding identified entities. The enterprise data in each field is used as input, and the corresponding identified entities are used as output to train the preset neural network model to generate a first entity recognition model corresponding to the enterprises in each field;

[0107] A knowledge distillation module is used to fuse the first entity recognition model corresponding to each field enterprise into a second entity recognition model through knowledge distillation;

[0108] A relationship extraction module, used to input the enterprise data into a second entity recognition model, identify entities in the enterprise data, and extract relationships between the entities;

[0109] The graph construction module is used to build an enterprise knowledge graph based on the identified entities and extracted relationships.

[0110] For relevant details, refer to the above method embodiment.

[0111] Figure 4 4 is a block diagram of an electronic device provided by an embodiment of the present application. The device at least includes a processor 401 and a memory 402.

[0112] The processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 401 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0113] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction, which is used to be executed by the processor 401 to implement the enterprise knowledge graph construction method provided in the method embodiment of the present application.

[0114] In some embodiments, the electronic device may further optionally include: a peripheral device interface and at least one peripheral device. The processor 401, the memory 402 and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Schematically, the peripheral devices include but are not limited to: a radio frequency circuit, a touch display screen, an audio circuit, and a power supply.

[0115] Of course, the electronic device may also include fewer or more components, which is not limited in this embodiment.

[0116] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the enterprise knowledge graph construction method of the above method embodiment.

[0117] Optionally, the present application also provides a computer product, which includes a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the enterprise knowledge graph construction method of the above method embodiment.

[0118] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for constructing an enterprise knowledge graph, characterized in that: The method comprises: Collect enterprise data and pre-process the enterprise data; Obtain a test data set, the test data set including enterprise data in each field and its corresponding identified entities; train a preset neural network model with the enterprise data in each field as input and the corresponding identified entities as output to generate a first entity recognition model corresponding to the enterprises in each field; The first entity recognition model corresponding to each field enterprise is fused into a second entity recognition model by means of knowledge distillation; the fusion of the first entity recognition model corresponding to each field enterprise into the second entity recognition model by means of knowledge distillation comprises: after generating the first entity recognition model corresponding to each field enterprise, the knowledge of these field models is transferred as a teacher model to a unified student model by means of knowledge distillation; the loss function is designed, and the gradient of the loss function is calculated and the parameters of the student model are updated by means of a back-propagation algorithm until the loss function reaches a minimum value or converges, thereby generating a second entity recognition model; Inputting the enterprise data into a second entity recognition model, identifying entities in the enterprise data, and extracting relationships between the entities; Build an enterprise knowledge graph based on the identified entities and extracted relationships.

2. The enterprise knowledge graph construction method according to claim 1 is characterized in that: The loss function is as follows: Where N is the number of teacher models, λ i is the weight coefficient of the i-th teacher model, and H student Respectively represent the feature representation of the i-th teacher model and the student model at the same layer; represents the multi-scale distillation loss between the feature representation of the i-th teacher model and the student model at each level; α is the parameter for adjusting the weight between the KL divergence loss and the multi-scale distillation loss, P teacher and P student Respectively represent the probability distribution of the teacher model and the student model in the output layer; D KL (P teacher , P student ) is the KL divergence; The calculation formula of multi-scale distillation loss is as follows: The calculation formula of KL divergence is as follows: Among them, P teacher (y j ) and P student (y j ) represent the teacher model and the student model in the output layer for category y j The probability distribution of .

3. The enterprise knowledge graph construction method according to claim 1 is characterized in that: The loss function is as follows: Among them, q i (x) k and p i (x) k Respectively represent the predicted probability of category k by the teacher model and the student model under the i-th category; w i is the weighting factor for each category, β is a parameter used to balance the impact between KL divergence and similarity loss, and N represents the total number of categories.

4. The enterprise knowledge graph construction method according to claim 1, characterized in that: The relationship between the extracted entities includes: Using the identified entities as keywords, we build a keyword co-occurrence matrix by scanning each paragraph or sentence in the enterprise data; After constructing the co-occurrence matrix, the relationship strength between entities is calculated, and the relationship strength is measured by the co-occurrence frequency; After the relationship strength is calculated, entity pairs whose relationship strength is greater than a preset threshold are selected according to the calculated relationship strength; The existing dependency parsing method is used to extract the relationship between the above entity pairs.

5. The enterprise knowledge graph construction method according to claim 4 is characterized in that: The constructing of the keyword co-occurrence matrix comprises: Each item M in the keyword co-occurrence matrix ij Represents entity E i and entity E j The number of co-occurrences in the same paragraph or sentence is as follows: M ij =count(E i ,E j ) Among them, M ij Represents entity E i and entity E j The number of times each pair of entities co-occurs in the same paragraph or sentence is calculated by traversing all the texts in the enterprise data, counting the frequency of each pair of entities co-occurring in the same paragraph or sentence, and generating a co-occurrence matrix.

6. The enterprise knowledge graph construction method according to claim 5, characterized in that: The calculation of the relationship strength between entities includes using the following formula to calculate the relationship strength R between each pair of entities ij : Among them, M ii and M jj Represents entity E i and entity E j The frequency of each occurrence in all paragraphs or sentences.

7. The enterprise knowledge graph construction method according to claim 5, characterized in that: The calculation of the relationship strength between entities includes using the following formula to calculate the relationship strength R between each pair of entities ij : Among them, M ii and M jj Represents entity E i and entity E j The frequency of occurrence of each in all paragraphs or sentences, sim(E i , E j ) is entity E i and entity E j Cosine similarity between word vectors.

8. An enterprise knowledge graph construction system, characterized in that: include: A data collection module, used to collect enterprise data and pre-process the enterprise data; A model building module, used for obtaining a test data set, which includes enterprise data in each field and its corresponding identified entities; Taking the enterprise data of each field as input and the corresponding identified entities as output, the preset neural network model is trained to generate a first entity recognition model corresponding to the enterprises in each field; A knowledge distillation module is used to fuse the first entity recognition model corresponding to each field enterprise into a second entity recognition model through knowledge distillation; The method of fusing the first entity recognition model corresponding to each field enterprise into the second entity recognition model by means of knowledge distillation includes: after generating the first entity recognition model corresponding to each field enterprise, migrating the knowledge of these field models as teacher models to a unified student model by means of knowledge distillation; designing a loss function, calculating the gradient of the loss function and updating the parameters of the student model by means of a back propagation algorithm until the loss function reaches a minimum value or converges, thereby generating a second entity recognition model; A relationship extraction module, used to input the enterprise data into a second entity recognition model, identify entities in the enterprise data, and extract relationships between the entities; The graph construction module is used to build an enterprise knowledge graph based on the identified entities and extracted relationships.

9. An electronic device, characterized in that: The device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement a method for constructing an enterprise knowledge graph as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a program, which, when executed by a processor, is used to implement a method for constructing an enterprise knowledge graph as described in any one of claims 1 to 7.

Images