Conceptual dependency relationship identification method, medium, and device for open knowledge base

Abstract

The application discloses an open knowledge base-oriented concept dependency relationship identification method, medium and equipment, relates to the technical field of natural language processing, and mainly comprises the following steps: training a concept representation learning model by using a concept description text set and a contrast learning target function on a pre-trained language model; obtaining a concept dependency relationship identification model by using a deep mutual learning method according to the concept representation learning model; and identifying a concept dependency relationship by using the concept dependency relationship identification model. The open knowledge base-oriented concept dependency relationship identification method, medium and equipment provided by the application can improve the robustness and computational efficiency of concept dependency relationship identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and more particularly, to a concept dependency relationship identification method, medium and device for an open knowledge base. BACKGROUND

[0002] Existing knowledge concept dependency relationship identification technologies mainly fall into two categories: one is based on traditional machine learning methods: through manual design of features and combination of classic classification models to identify relationships. For example: Talukdar and Cohen first explored the use of Wikipedia to identify concept dependency relationships. They integrated hyperlinks, editing records and text content in the page, constructed features for each pair of concepts, and used a maximum entropy classifier to predict the dependency relationship between the concept pairs. Liang et al. used the hyperlinks between Wikipedia pages to calculate the reference distance (RefD) between knowledge concepts, thereby identifying the dependency relationship between the concepts. Pan et al. extracted Wikipedia concepts from MOOC video subtitles and combined text, structure and semantic information to construct features for concept pairs to discover the dependency relationship between the concepts. In addition, Sayyadiharikandeh et al. predicted concept dependency relationships based on user click behavior on Wikipedia article pages. They assumed that users continuously access two articles to obtain related background information, and based on this assumption, they constructed features for multiple concept pairs to identify relationships. The second is a deep learning-based method: automatically extract semantic features through neural networks to reduce dependence on manual features. For example: Miaschi et al. compared the effects of text embedding features, manual features and their fusion in concept dependency relationship identification. Roy et al. proposed the PREREQ method to identify concept dependency relationships using a twin neural network, which infers unknown concept dependency relationships based on known learning resource dependency relationships and partial concept dependency relationships. Li et al. proposed the R-VGAE method based on the unsupervised learning characteristics of the variational graph autoencoder to generate concept dependency graphs. Zhang et al. proposed the MHAVGAE and wMHAVGAE methods based on semi-supervised learning and weakly supervised learning, respectively. Sun et al. proposed the ConLearn method to identify concept dependency relationships using a pre-trained language model and a gated graph neural network. Mazumder et al. proposed a graph attention network model based on a heterogeneous "document-concept" graph, which added document and concept nodes and multiple relationship edges, used graph attention network to generate node embeddings, and then predicted relationship labels.

[0003] 1. Deficiencies of traditional machine learning methods: Strong feature dependency and weak generalization ability. Model performance is highly dependent on manually designed features (such as hyperlinks, click behavior, etc.), making it difficult to adapt to the complex and ever-changing conceptual semantic scenarios in open knowledge bases, and feature engineering is time-consuming and labor-intensive. Insufficient computational efficiency and scalability. When processing large-scale data, manual feature extraction and model training are inefficient and difficult to handle massive data scenarios such as Wikipedia.

[0004] 2. Limitations of Deep Learning Methods: Data Dependence and Overfitting Risk. Complex neural networks (such as graph neural networks and multi-layer Transformers) require a large amount of labeled data for training, making them prone to overfitting when data is sparse or labels are insufficient, and training costs are high (e.g., GPU resource consumption). Model Complexity and Interpretability Issues. Deep models (such as variational autoencoders and multi-head attention networks) have complex structures and exhibit "black box" characteristics, making it difficult to interpret prediction logic and limiting their application in fields requiring interpretability, such as education and intelligence analysis. Limitations in Semantic Representation Capabilities. Traditional deep learning methods are insufficient at capturing subtle differences in conceptual semantics; for example, they cannot effectively distinguish the semantic boundaries between "similar concepts" and "dependent concepts."

[0005] Existing methods struggle to strike a balance between performance, efficiency, and interpretability. Traditional methods rely on feature engineering, which is ill-suited to the dynamic and heterogeneous nature of open knowledge bases. While deep learning methods improve automation, their high complexity leads to significantly increased computational costs and overfitting risks, and they lack lightweight design. Summary of the Invention

[0006] The purpose of this invention is to provide a method, medium, and device for identifying concept dependencies in open knowledge bases, which can improve the robustness and computational efficiency of concept dependency identification.

[0007] This invention provides a method for identifying concept dependencies in open knowledge bases, comprising the following steps: S1: training a pre-trained language model using a set of concept description texts and a contrastive learning objective function to obtain a concept representation learning model; S2: obtaining a concept dependency identification model using a deep mutual learning method based on the concept representation learning model; S3: identifying concept dependencies using the concept dependency identification model.

[0008] Furthermore, the objective function for the above contrastive learning is as follows:

[0009] ,

[0010] in, The contrastive learning objective function is the contrastive learning training objective for each training batch. Indicates batch size; Represents the cosine similarity between two embeddings; and Representing concepts respectively Description text and copying concept Embedding of descriptive text; and Representing concepts respectively Description text and copying concept Embedding of descriptive text, It is the temperature coefficient.

[0011] Furthermore, the aforementioned pre-trained language model is the BERT model.

[0012] Further, step S2 specifically includes: S21: replicating the concept representation learning model to obtain a first isomorphic model and a second isomorphic model; S22: constructing a first total loss function and a second total loss function; S23: training the first isomorphic model and the second isomorphic model using the first total loss function and the second total loss function to obtain a trained first isomorphic model and a trained second isomorphic model; S24: testing the trained first isomorphic model and the trained second isomorphic model using a validation set, selecting the isomorphic model with the best performance to obtain a concept dependency recognition model.

[0013] Furthermore, both the first and second isomorphic models mentioned above include a concept representation learning model and a fully connected layer connected sequentially.

[0014] Further, step S22 specifically includes: constructing a first total loss function and a second total loss function, as shown in the formula:

[0015] ,

[0016] ,

[0017] ,

[0018] ,

[0019] ,

[0020] ,

[0021] ,

[0022] in, and These are the first total loss function and the second total loss function, respectively. and These are the objective functions for the first isomorphic model and the second isomorphic model, respectively. and These are the loss functions for mutual learning between the first isomorphic model and the second isomorphic model, respectively. The number of samples; It is a sample Tags; It is an index function, if but ,otherwise ; For the first isomorphic model, for the samples The probability of the output; For the second isomorphic model for samples The probability of the output.

[0023] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method for identifying concept dependencies for open knowledge bases.

[0024] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method for identifying concept dependencies for open knowledge bases.

[0025] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for identifying concept dependencies for open knowledge bases.

[0026] This invention also provides an application of the above-described concept dependency identification method for open knowledge bases, which is applied to the development of learning systems.

[0027] The concept dependency identification method, medium, and device for open knowledge bases provided by this invention have the following beneficial effects:

[0028] This invention addresses the shortcomings of traditional methods in terms of data sparsity, overfitting, and computational efficiency by proposing a lightweight staged concept prerequisite relation recognition framework (LSCPL). LSCPL primarily comprises a contrastive learning stage and a deep mutual learning stage. In the contrastive learning stage, a pre-trained language model (such as BERT) is used to perform bi-branch dropout masking encoding on the concept description text, generating differentiated embeddings (positive pairs) for the same concept and embeddings (negative pairs) for different concepts. The model is optimized using a contrastive learning loss function (InfoNCE), forcing positive pairs to be closer together in the semantic space and negative pairs to be further apart, thereby improving the discriminative power of concept semantic representation. In the deep mutual learning stage, the model optimized by contrastive learning (BERT-S) is replicated into two isomorphic models (…). and This approach achieves knowledge sharing and prediction distribution alignment between models through collaborative training using cross-entropy loss (CE) and KL divergence loss (KL). This dual-model approach not only learns the true labels but also forces consistent predictions between models through KL divergence, thus improving robustness.

[0029] This invention enhances semantic representation through contrastive learning: in terms of unsupervised optimization, no manual annotation is required; contrastive signals are constructed through differential encoding of the same text to capture subtle semantic differences between concepts; in terms of model output, an enhanced model BERT-S is generated, which significantly improves the semantic discrimination ability of concept embedding.

[0030] This invention achieves dual-model knowledge sharing and anti-overfitting capability through a collaborative mechanism of deep mutual learning. In terms of dual-model knowledge sharing, implicit knowledge transfer is achieved through prediction distribution alignment (KL divergence), which improves performance under lightweight design without the need for complex graph structures or additional parameters. In terms of anti-overfitting capability, the collaboration between models reduces the dependence on a single model and makes it more robust in data sparse scenarios.

[0031] This invention achieves phased optimization and improved computational efficiency through an efficient end-to-end recognition process. In terms of phased optimization, it first enhances semantic representation through contrastive learning, and then optimizes prediction capabilities through mutual learning, forming a "representation..." The end-to-end pipeline for "prediction"; in terms of improving computational efficiency, the overall framework is based on the standard BERT architecture, without introducing complex neural network layers, and the computational cost is significantly lower than that of graph neural networks and other methods. Attached Figure Description

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0033] Figure 1This is a flowchart of the concept dependency identification method for open knowledge bases provided by the present invention;

[0034] Figure 2 This is a general framework diagram of the concept dependency identification method for open knowledge bases provided by the present invention;

[0035] Figure 3 This is a comparison result diagram of the deep mutual learning model based on two identical initial models provided by the present invention;

[0036] Figure 4 This is a comparison diagram of the deep mutual learning models based on two different initial models provided by the present invention;

[0037] Figure 5 This is a schematic diagram illustrating how the performance of the LSCPL framework provided by this invention changes with parameter variations;

[0038] Figure 6 This is a structural block diagram of the computer device provided by the present invention. Detailed Implementation

[0039] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0040] Figure 1 A schematic diagram of the concept dependency identification method for open knowledge bases according to this embodiment is shown. In this embodiment, the concept dependency identification method for open knowledge bases includes the following steps:

[0041] S1: Train the pre-trained language model using a set of concept description texts and a contrastive learning objective function to obtain a concept representation learning model;

[0042] In one exemplary embodiment, the contrastive learning objective function is as follows:

[0043]

[0044] in The contrastive learning objective function is the contrastive learning training objective for each training batch. Indicates batch size; Indicates the cosine similarity between two embeddings; and Representing concepts respectively Description text and copying concept Embedding of descriptive text; and Representing concepts respectively Description text and copying concept Embedding of descriptive text; It is the temperature coefficient;

[0045] In one exemplary embodiment, the pre-trained language model is a BERT model;

[0046] S2: Based on the concept representation learning model, a concept dependency recognition model is obtained using the deep mutual learning method;

[0047] In one exemplary embodiment, step S2 specifically includes:

[0048] S21: By replicating the concept representation learning model, we obtain the first isomorphic model and the second isomorphic model;

[0049] In one exemplary embodiment, both the first isomorphic model and the second isomorphic model include a concept representation learning model and a fully connected layer connected in sequence;

[0050] S22: Construct the first total loss function and the second total loss function;

[0051] In one exemplary embodiment, step S22 specifically includes: constructing a first total loss function and a second total loss function, as shown in the formula:

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058] ,

[0059] in, and These are the first total loss function and the second total loss function, respectively. and These are the objective functions for the first isomorphic model and the second isomorphic model, respectively. and These are the loss functions for mutual learning between the first isomorphic model and the second isomorphic model, respectively. The number of samples; It is a sample Tags; It is an index function, if but ,otherwise ; For the first isomorphic model, for the samples The probability of the output; For the second isomorphic model for samples The probability of the output;

[0060] S23: Train the first isomorphic model and the second isomorphic model using the first total loss function and the second total loss function to obtain the trained first isomorphic model and the trained second isomorphic model;

[0061] S24: Test the trained first isomorphic model and the trained second isomorphic model using the validation set, and select the isomorphic model with the best performance to obtain the concept dependency recognition model.

[0062] S3: Use the concept dependency identification model to identify concept dependencies.

[0063] In this embodiment, the concept dependency identification method for open knowledge bases is applied to the development of a learning system, including the following steps:

[0064] Step 1: Train the pre-trained language model using a set of concept description texts and a contrastive learning objective function to obtain a concept representation learning model;

[0065] Among them, the conceptual description text set is a subject-specific corpus text set, especially the geological conceptual description text set;

[0066] Step 2: Based on the concept representation learning model, use deep mutual learning methods to obtain the concept dependency recognition model;

[0067] Step 3: Use the concept dependency identification model to identify concept dependencies and obtain geological concept dependencies.

[0068] In some embodiments, the above-described method for identifying concept dependencies for open knowledge bases can also be implemented in the following ways.

[0069] This method consists of two phases: a contrastive learning phase and a deep mutual learning phase, aiming to improve the model's ability to identify dependencies between knowledge concepts through collaborative strategies. The overall framework is as follows: Figure 2 As shown, the following is a detailed description of the specific technical solution:

[0070] 1. Two-stage optimization strategy for LSCPL

[0071] 1.1 Comparative Learning Phase

[0072] In the contrastive learning phase, concept description text is input into the BERT model, and two forward passes are performed using different dropout masks. Because the dropout mask is different in each forward pass, different embedding representations are generated even if the input text is the same. Specifically, for each concept description text, BERT generates two embeddings, which are considered a positive pair. On the other hand, embeddings of different concepts are considered negative pairs. By comparing these positive and negative pairs, the model can better capture subtle differences and similarities between concepts, thereby enhancing its representational capabilities. This process ultimately produces a concept representation learning model called BERT-S.

[0073] Specifically, this embodiment first describes the text set using concepts. As input, where The number of concepts. For the concept The description text. Copy To obtain and to and By applying different dropout masks, two embeddings of the same input text form positive sample pairs. However, within the same training batch of the model, embeddings of any two different concepts form negative sample pairs. Let... and They represent and The embedding. The training objective for the contrastive learning in each training batch is:

[0074] (1)

[0075] in Indicates batch size, The cosine similarity between two embeddings is represented. It is the temperature coefficient; a suitable temperature coefficient allows the model to more accurately capture the semantic similarity between texts.

[0076] 1.2 Deep Learning Phase

[0077] Following the contrastive learning phase, the BERT-S model is used to encode concepts. First, in this embodiment, the BERT-S model trained in the contrastive learning phase is replicated into two independent models for co-training in the deep mutual learning phase. The embeddings generated by BERT-S from the [CLS] token are fed into a fully connected layer for concept dependency prediction. The combination of BERT-S and the fully connected layer is denoted as... ,in , and These are the two models that will engage in deep mutual learning in this stage.

[0078] Given a pair of concepts Their corresponding descriptive text and ,in Representing concepts and In the description text The quantity. For ease of data processing, this embodiment uses [CLS] and [SEP] to separate text representing different concepts. The specific format is as follows:

[0079] (2)

[0080] Recorded as .

[0081] Given Sample For the model ,definition = Let be the output probability, where , This represents the probability that a dependency exists between two concepts, while This represents the probability that no relationship exists.

[0082] Use the cross-entropy function as the training model. Objective function:

[0083] (3)

[0084] in It is a sample The tag, It is an index function, if but ,otherwise Similarly, the model The objective function can be defined as:

[0085] (4)

[0086] During training, the two models learn from each other. Specifically, the Kullback-Leibler (KL) divergence between the predicted outputs of the two models is used to measure the difference between the two distributions, thereby promoting consistency between the two models. The loss function for mutual learning between the two models can be defined as:

[0087] (5)

[0088] (6)

[0089] Based on this, the total loss function of the two models can be defined as:

[0090] (7)

[0091] (8)

[0092] In this way, each model not only learns how to accurately predict the true labels of the training samples, but also improves by matching the probability estimates of the other model with KL divergence. In each iteration, the predictions of both models are calculated, and the parameters are updated based on the predictions of the other model. Continuous optimization is then performed. and The process continues until convergence. Finally, the two trained models are tested on a validation set, and the model with the best performance is selected for testing. The deep mutual learning algorithm is shown in Table 1.

[0093] Table 1: Deep Mutual Learning Algorithms

[0094]

[0095] 2. Experiment and Results Analysis

[0096] 2.1 Experimental Setup

[0097] Datasets. Three public datasets were used to evaluate the proposed framework and baselines. The datasets include:

[0098] University Course: This dataset includes concepts of 654 computer science courses offered by 11 universities in the United States.

[0099] MOOC: This dataset consists of texts from 38 MOOC video lectures, containing a large number of concept-topic and concept-dependency pairs, similar to university course datasets.

[0100] LectureBank: This dataset covers learning resources in fields such as natural language processing, machine learning, deep learning, information retrieval, and artificial intelligence.

[0101] Evaluation metrics. Widely used evaluation metrics were employed, including accuracy (ACC), precision (P), recall (R), F1 score (F1), and area under the ROC curve (AUC).

[0102] Baseline model. To validate the framework's performance, it was compared with several mainstream methods for predicting concept dependencies, as follows:

[0103] HGAPNet is based on graph neural networks and uses an attention mechanism on nodes in heterogeneous graphs to extract and predict dependencies between concepts.

[0104] ConLearn uses BERT for knowledge transfer and employs graph neural networks to capture context-aware concept dependencies.

[0105] MHAVGAE constructs a heterogeneous graph connecting resources and concepts. It uses word2vec to initialize node features and employs multi-head attention and gating mechanisms to improve concept representations. Finally, it applies a variational graph autoencoder to predict concept relationships.

[0106] M3 combines LSTM with handcrafted features to define and predict dependencies between concepts.

[0107] RefD captures dependencies between Wikipedia concepts by measuring the citation differences between two concepts.

[0108] Implementation details. The code for the relevant methods is implemented using PyTorch on an RTX 4090 GPU with 24GB of memory. To increase the number of samples in the dataset, the number of positive samples is expanded transitively; that is, if concept A is a prerequisite for concept B, and concept B is a prerequisite for concept C, then concept A is also considered a prerequisite for concept C. Negative samples are generated by randomly pairing concept pairs that do not appear in the positive samples. To evaluate model performance, the dataset is divided into training, validation, and test sets in an 8:1:1 ratio. All baseline methods use the same data partitioning to ensure fair and accurate comparisons between different methods.

[0109] For the proposed LSCPL method, this embodiment extracts the first 100 words from the Wikipedia article for each concept as their descriptive text. During the contrastive learning phase, the BERT-base model (BERT-base-uncased) is used as the base pre-trained language model. The learning rate is set to... Dropout is set to 0.1, and batch size is set to 64. Set to 0.05. Concept descriptions used for contrastive learning are derived from all concepts across three datasets. During the deep mutual learning phase, the learning rate and epochs of the BERT-S model are set to... The batch sizes for the three datasets were set to 32, 32, and 16, respectively.

[0110] Table 2: Comparison of experimental results between the framework (LSCPL) and baseline methods on three datasets (University Course, MOOC, and LectureBank).

[0111]

[0112] 2.2 Comparison with baseline methods

[0113] The experimental results are shown in Table 2, demonstrating that the proposed LSCPL method performs exceptionally well across multiple metrics. Except for recall, LSCPL outperforms other baseline models in accuracy, precision, F1 score, and AUC. For example, on the three datasets, the F1 score of the LSCPL method is 4.7%, 8.6%, and 11.8% higher than the best baseline method, respectively. The Improvement rate represents the performance improvement of the LSCPL method over the second-best performing baseline method.

[0114] Experimental results show that: (1) The LSCPL method significantly outperforms other baseline models on multiple metrics. Through knowledge transfer in the contrastive learning and deep mutual learning stages, LSCPL fully learns and captures the relationships between concepts, improving overall prediction performance; (2) The performance of LSCPL and ConLearn methods based on pre-trained language models is significantly better than other types of methods, showing that the pre-trained models provide powerful semantic representation capabilities, enabling these models to perform well on multiple metrics; (3) The ConLearn method combines pre-trained BERT and graph neural networks, and its performance is better than other baseline models, but still lower than the LSCPL method. This is because LSCPL has unique advantages in contrastive learning and deep mutual learning, which gives it a comprehensive performance advantage.

[0115] 2.3 Analysis of the Role of Contrastive Learning

[0116] To verify the effectiveness of contrastive learning in improving model performance, this embodiment compares and analyzes the original BERT model and the BERT-S model optimized by contrastive learning on three datasets. The results are shown in Table 3.

[0117] Experimental results show that BERT-S outperforms the original BERT on most metrics, validating the effectiveness of the contrastive learning stage. For example, on the University Course dataset, BERT-S significantly outperforms BERT in both accuracy (0.913) and F1 score (0.914). This demonstrates that contrastive learning significantly improves the overall performance of the model. Similar results were obtained on the other two datasets. These results indicate that through contrastive learning, the model can better identify dependencies between concepts, thus providing a more reliable performance improvement in practical applications.

[0118] Table 3: Comparison of BERT and BERT-S results

[0119]

[0120] 2.4 Analyzing the role of deep mutual learning

[0121] To further verify the contribution of deep mutual learning to improving the accuracy and reliability of concept dependency identification, this embodiment designed the following experiments. First, the performance of deep mutual learning was evaluated using two identical initial model settings to verify the impact of the mutual learning mechanism on model improvement under the same structure and parameters. Second, the performance of deep mutual learning was evaluated using two different initial model settings to explore the complementary and collaborative effects of models with different structures or parameters in mutual learning.

[0122] To verify the performance of deep mutual learning under the same initial model settings, this embodiment compares models without deep mutual learning with those with deep mutual learning on the UniversityCourse dataset. These models use... and It means that among them .here, This indicates a model without deep mutual learning, and This indicates starting from two identical initial models. Initially, the model that achieves best performance on the validation set through deep mutual learning is shown in the comparison results. Figure 3 As shown.

[0123] The experimental results lead to the following conclusions:

[0124] (1) When two initial models are identical, the model that has undergone deep mutual learning almost always outperforms the single model on various metrics, confirming the effectiveness of deep mutual learning. For example, The model significantly outperforms the single BERT model in F1 score and AUC, indicating that the two initial models can enhance each other through mutual learning, significantly improving overall performance.

[0125] (2) Superior This further validates the effectiveness of the contrastive learning phase. Specifically, on the university course dataset, It achieved state-of-the-art results in both F1 score (0.929) and AUC (0.963), demonstrating the powerful effect of combining contrastive learning with deep mutual learning. Furthermore, similar results were observed in experiments on two other datasets.

[0126] To verify the performance of deep mutual learning under different initial models, this embodiment conducted experiments on the MOOC dataset using three models: R-GCN, BERT, and BERT-S, and compared the performance of these models before and after mutual learning in pairs. Model A (Model B) represents the result of Model A learning Model B. The experimental results are as follows: Figure 4 As shown.

[0127] Experimental results show that when the initial models are different, the model after deep mutual learning generally outperforms the single model on all metrics. For example, BERT-S (BERT) significantly outperforms the single BERT-S model in both F1 score and AUC. These results validate the effectiveness of deep mutual learning based on two different initial models. Through mutual learning, different models can fully leverage their respective strengths and enhance each other, significantly improving overall performance. Although using different models for mutual learning can improve performance, experimental results indicate that mutual learning between two identical BERT-S models yields the best results. This is because the structural similarity between identical models makes it easier for them to learn from each other and share information, thereby better optimizing the model's representation capabilities.

[0128] 2.5 Model Parameter Analysis

[0129] In the LSCPL method, learning rate, batch size, and the length of the text describing the concept are key parameters. This invention uses F1 score and AUC metric to evaluate the model's performance under different learning rates, batch sizes, and text lengths, and then selects the optimal combination of parameters to achieve the best model performance. Figure 5 The experimental results are shown on the Lecture Bank dataset.

[0130] For the learning rate, the candidate values ​​used in this embodiment are { , , , , }; For batch size, the candidate value used is For describing text length, the candidate value used is When testing one parameter, the other parameters remain unchanged. Through experiments, the optimal learning rate was finally determined. The batch size is 16, and the text length is 100.

[0131] 3. Implementation Cases

[0132] To further verify the effectiveness of the contrastive learning phase and the deep mutual learning phase in the LSCPL framework, this embodiment presents a case study comparing the performance of BERT, BERT-S (BERT enhanced by contrastive learning), and LSCPL (a framework that introduces deep mutual learning based on BERT-S) in identifying the dependencies between real concept pairs (A, B).

[0133] First, six pairs of concepts were randomly selected from the three datasets, including three positive samples and three negative samples. Table 4 shows the recognition results of BERT, BERT-S, and LSCPL, where... This indicates that concept B is a dependent condition of concept A. It means no.

[0134] Table 4: Case Study Examples

[0135]

[0136] As shown in Table 4, BERT and BERT-S correctly identified 4 examples of concept dependencies, while LSCPL correctly identified 5. Although BERT and BERT-S identified the same number of correct cases in this specific example, this does not mean that the contrastive learning stage is ineffective. In broader experiments, BERT-S outperformed BERT on several key metrics such as F1 score and AUC, indicating that contrastive learning plays a crucial role in enhancing the model's ability to represent concept dependencies. LSCPL further improves the overall recognition performance by introducing deep mutual learning on top of BERT-S, enhancing collaboration and knowledge sharing between models. The results show that the LSCPL method significantly improves the accuracy of concept pre- and post-relationship recognition without significantly increasing model complexity through the combination of contrastive learning and deep mutual learning. This further validates the effectiveness and superiority of LSCPL in practical applications.

[0137] This embodiment provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the above-described method for identifying concept dependencies in an open knowledge base. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0138] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the above-described method for identifying concept dependencies for open knowledge bases.

[0139] like Figure 6 As shown, the computer device 120 may include: at least one processor 121, such as a central processing unit (CPU), at least one communication interface 123, memory 124, and at least one communication bus 122. The communication bus 122 is used to enable communication between these components. The communication interface 123 may include a display screen and a keyboard; optionally, the communication interface 123 may also include a standard wired interface or a wireless interface. The memory 124 may be high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 124 may also be at least one storage device located remotely from the aforementioned processor 121. The memory 124 stores application programs, and the processor 121 calls the program code stored in the memory 124 to execute any of the aforementioned method steps. The communication bus 122 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 122 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6The term 124 is represented by a single line, but this does not imply a single bus or a single type of bus. The memory 124 may include volatile memory, such as random-access memory (RAM); it may also include non-volatile memory, such as flash memory, hard disk drive (HDD), or solid-state drive (SSD); or it may include combinations of the above types of memory. The processor 121 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. The processor 121 may further include hardware chips. These hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Optionally, the memory 124 is also used to store program instructions. The processor 121 can invoke the program instructions to implement the concept dependency identification method for open knowledge bases as described in this embodiment.

[0140] This embodiment provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for identifying concept dependencies for open knowledge bases.

[0141] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for identifying concept dependencies in open knowledge bases, characterized in that, Includes the following steps: S1: Train the pre-trained language model using a set of concept description texts and a contrastive learning objective function to obtain a concept representation learning model; S2: Based on the concept representation learning model, a concept dependency recognition model is obtained using the deep mutual learning method; S3: Use the concept dependency identification model to identify concept dependencies; The contrastive learning objective function is as follows: ， in, The contrastive learning objective function is the contrastive learning training objective for each training batch. Indicates batch size; Indicates the cosine similarity between two embeddings; and Representing concepts respectively Description text and copying concept Embedding of descriptive text; and Representing concepts respectively Description text and copying concept Embedding of descriptive text, It is the temperature coefficient; Step S2 specifically includes: S21: By replicating the concept representation learning model, we obtain the first isomorphic model and the second isomorphic model; S22: Construct the first total loss function and the second total loss function; S23: Train the first isomorphic model and the second isomorphic model using the first total loss function and the second total loss function to obtain the trained first isomorphic model and the trained second isomorphic model; S24: Test the trained first isomorphic model and the trained second isomorphic model using the validation set, and select the isomorphic model with the best performance to obtain the concept dependency recognition model.

2. The concept dependency identification method for open knowledge bases according to claim 1, characterized in that, The pre-trained language model is the BERT model.

3. The concept dependency identification method for open knowledge bases according to claim 1, characterized in that, Both the first isomorphic model and the second isomorphic model include a concept representation learning model and a fully connected layer connected in sequence.

4. The concept dependency identification method for open knowledge bases according to claim 1, characterized in that, Step S22 specifically includes: constructing the first total loss function and the second total loss function, as shown in the formula: ， ， ， ， ， ， ， in, and These are the first total loss function and the second total loss function, respectively. and These are the objective functions for the first isomorphic model and the second isomorphic model, respectively. and These are the loss functions for mutual learning between the first isomorphic model and the second isomorphic model, respectively. The number of samples; It is a sample Tags; It is an index function, if but ,otherwise ; For the first isomorphic model, for the samples The probability of the output; For the second isomorphic model for samples The probability of the output.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the concept dependency identification method for open knowledge bases as described in any one of claims 1-4.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the concept dependency identification method for open knowledge bases as described in any one of claims 1-4.

7. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the concept dependency identification method for open knowledge bases as described in any one of claims 1-4.

8. An application of the concept dependency identification method for open knowledge bases as described in any one of claims 1-4, characterized in that, It is used in the development of learning systems.

Images