LLM-based management field proposition knowledge graph construction method, system, device and storage medium

Abstract

This invention discloses a method, system, device, and storage medium for constructing a management proposition knowledge graph based on LLM, relating to the fields of knowledge graph and natural language processing technology. It extracts complex propositional relationships through a three-stage framework of "empirical identification – summary generation – thought chain extraction." By introducing a dual-guidance mechanism and few-sample thought chain technology, it overcomes the extraction difficulties of complex nested logics such as mediation and moderating effects, achieving high-precision transformation from unstructured text to structured propositional triples. Then, it employs knowledge distillation and LoRA fine-tuning to train a domain-specific disambiguation model. Using a hybrid strategy of geometric space clustering and logical reasoning verification, it designs a boundary detection and diversion algorithm for polysemy and a dual-view recall mechanism for multiple words with the same meaning. It introduces effect event entities to solve the storage problem of high-order causal logic, realizing the structured reorganization and in-depth mining of management empirical knowledge.

Description

Technical Field

[0001] This invention belongs to the field of knowledge graph and natural language processing technology, and relates to a method, system, device and storage medium for constructing a propositional knowledge graph in the field of management based on LLM. Background Technology

[0002] Academic literature, especially empirical research in management, is an important public data resource. However, this literature often exists in unstructured text form, with its theoretical propositions (such as mediation and moderation effects) deeply embedded in complex empirical logic, leading researchers to face severe "information overload" and knowledge fragmentation dilemmas. Traditional knowledge organization methods and information extraction techniques based on rules or early pre-trained models are insufficient to effectively reveal the deep causal mechanisms between variables, and cannot handle the widespread "polysemy" and "multiple words with the same meaning" problems in management terminology, thus limiting the effective integration of knowledge and innovative discovery.

[0003] With the development of Large Language Models (LLMs), they have demonstrated powerful capabilities in complex semantic understanding and few-shot learning, offering new possibilities for in-depth mining of structured knowledge from specialized literature. However, directly applying general-purpose LLMs to highly specialized and logically nested fields like management still faces challenges such as low extraction accuracy, ambiguous entity referencing, and the difficulty in structurally representing complex effects (such as mediated moderating factors). Therefore, there is an urgent need for an automated and high-precision propositional knowledge graph construction scheme tailored to the characteristics of the management field. Summary of the Invention

[0004] This invention aims to...

[0005] To achieve the above-mentioned technical objectives and effects, the present invention provides the following technical solution:

[0006] A method for constructing a propositional knowledge graph in the field of management based on LLM, the method comprising:

[0007] S1. Construct an empirical literature dataset in the field of management: Using T1-level Chinese journals in high-quality management science journals as the data source, retrieve and obtain a literature dataset that meets the requirements of empirical research, convert unstructured full-text PDFs into Markdown structured texts and perform data cleaning using regular expressions;

[0008] S2. Extraction of management proposition relations based on large language model: Using large language model, a three-stage process including empirical identification, empirical summary generation and fusion summary extraction of thought chain is adopted to extract structured proposition triples containing main effect, mediation effect, moderation effect, chain mediation effect, moderated mediation effect and moderated moderation effect from the literature dataset.

[0009] S3. Named Entity Disambiguation of Management Propositions: Using domain-enhanced data generated based on knowledge distillation and efficient parameter fine-tuning techniques, a domain-specific embedding model and entity discrimination model are trained; based on the embedding model and entity discrimination model, differentiated strategies are adopted for polysemy and homosemantic phenomena to disambiguate and merge entities in the proposition triples to obtain a standardized entity set;

[0010] S4. Construction and Storage of Knowledge Graph for Management Propositions: Design a knowledge graph pattern layer adapted to the complex causal logic of management, introduce effect event entities to solve the problem of graph database storage for high-order nested causal logic, and complete the construction and persistent storage of knowledge graph for management propositions based on standardized entity sets and proposition triples through Neo4j graph database.

[0011] Furthermore, in step S2, the extraction of management proposition relations based on the large language model is specifically implemented through the management proposition relation extraction framework MPGE, and the three-stage process includes:

[0012] S21. Empirical identification stage: Use a large language model to determine whether the literature contains empirical tests of the type described in step S2, in order to filter the data;

[0013] S22. Empirical Abstract Generation Stage: Based on the IMRAD framework, a large language model is used to generate a structured abstract containing the research question, research methods, research hypotheses, and research conclusions;

[0014] S23. Mind chain extraction stage of fused summary: Input the original text of the document and the structured summary into the large language model, and output proposition triples in JSON format through few-sample mind chain prompts that include role settings, task descriptions, extraction steps, examples and output formats.

[0015] Furthermore, in step S2, during the thought chain extraction stage, a dual guidance mechanism is adopted, that is, macro-level guidance and micro-level guidance are provided simultaneously, and a lower temperature parameter is set to optimize the model's extraction performance for complex nested logic.

[0016] Furthermore, in step S3, the disambiguation strategy for polysemy includes: using the domain-specific embedding model, constructing a semantic span tensor based on the semantic vector of the entity definition, locating semantic distribution edge samples through a boundary detection algorithm, performing an early stopping and diversion strategy for monosemous entities after large model discrimination, using the HDBSCAN algorithm and adaptive K-Means strategy to complete semantic segmentation for polysemous entities, and ensuring the logical rigor of the disambiguation results through anchor point consistency verification.

[0017] Furthermore, in step S3, the disambiguation strategy for the phenomenon of multiple words with the same meaning includes: adopting a two-stage "clustering-exploration" strategy, first completing high-confidence semantic clustering and intra-cluster cleaning based on entity name features, then completing cross-vocabulary candidate synonym entity pair recall through approximate nearest neighbor search of entity definition vectors, using the TermJudge model to perform final semantic determination on candidate synonym entity pairs, and merging entities determined to be synonyms.

[0018] Furthermore, in step S4, the schema layer design includes ontology design and relation design. The ontology design includes four sets of mutually exclusive entities: journals, literature, management variables, and effect events. The relation design is divided into two main categories: structural relations and semantic relations. Structural relations are used to construct a literature source index framework, while semantic relations are used to reproduce the empirical logic between variables. The effect event entities enable the complete reproduction of complex theoretical models such as mediated regulation.

[0019] On the other hand, the present invention also provides a system based on the method described above, comprising:

[0020] The data processing module is used in step S1 of the above method;

[0021] The proposition relation extraction module is used in step S2 of the above method;

[0022] The entity disambiguation module is used in step S3 of the above method;

[0023] The map construction and storage module is used to execute step S4 of the above method.

[0024] On the other hand, the present invention also provides an electronic device including at least one processor and a memory, the memory storing a computer program that, when executed by the at least one processor, causes the electronic device to perform the method described above.

[0025] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] 1. By adopting a three-stage pipeline of "empirical identification - summary generation - thought chain extraction" and a dual guidance mechanism, the accuracy and robustness of large language models in extracting complex nested propositional relationships from long texts are significantly improved, overcoming the problems of traditional methods that rely heavily on labeled data and have poor generalization ability.

[0028] 2. Domain knowledge is acquired through knowledge distillation, and dedicated embedding and discriminative models are trained to deeply integrate management semantic features into the disambiguation process. The proposed strategies of "one word with multiple meanings for diffusing disambiguation" and "multiple words with one meaning for dual-view recall" significantly improve the recall rate of long-tail entities while ensuring disambiguation accuracy, successfully reducing the entity size to 29,000 independent constructs.

[0029] 3. By designing the ontology "Effect Event Entity", the problem of storing and representing higher-order logic such as moderating effects and mediated moderating in graph databases is innovatively solved, enabling the constructed knowledge graph to faithfully and structurally reproduce complex theoretical models in empirical management research.

[0030] 4. The final constructed propositional knowledge graph connects massive, fragmented empirical conclusions into a structured network. Through deep path traversal and analysis, it successfully reveals the contradictory conclusions and theoretical mechanisms of issues in different contexts. The constructed graph has significant practical value in assisting academic reviews, discovering theoretical conflicts, and supporting scientific research decision-making. It not only provides a new technical paradigm for knowledge management in the field of management, but also provides a reference solution for the development and utilization of public data elements. Attached Figure Description

[0031] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the specific embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a comparison image of data before and after cleaning in an embodiment of the present invention;

[0033] Figure 2 This is a distribution diagram of the empirical situation of journals in the embodiments of the present invention;

[0034] Figure 3 These are the extraction results at different temperatures in the embodiments of the present invention;

[0035] Figure 4 This describes the entity definition data distribution in this embodiment of the invention.

[0036] Figure 5 This is a disambiguation case for polysemy in an embodiment of the present invention;

[0037] Figure 6 This is a diagram illustrating the effect of multi-word homonymy clustering in an embodiment of the present invention.

[0038] Figure 7 This refers to the stage distribution of the disambiguation entities in the embodiments of the present invention;

[0039] Figure 8 This is the distribution of similarity between multi-word monosemous entity names and definitions in an embodiment of the present invention;

[0040] Figure 9 This is a partial visualization of the knowledge graph of propositions in the field of management in an embodiment of the present invention;

[0041] Figure 10 This is a schematic diagram illustrating the subordinate relationship between variables and titles in an embodiment of the present invention;

[0042] Figure 11 This is a schematic diagram illustrating the influence effect between variables in an embodiment of the present invention;

[0043] Figure 12 This is a schematic diagram illustrating the positive impact of inconsistencies in an embodiment of the present invention.

[0044] Figure 13 This is a schematic diagram illustrating the negative impact of inconsistencies in an embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] The invention will now be further described with reference to the accompanying drawings.

[0047] A method for constructing a propositional knowledge graph in the field of management based on LLM includes the following steps:

[0048] 1. Construct an empirical literature dataset in the field of management.

[0049] This invention selects T1-level Chinese journals from FMS as the benchmark journal source to construct a dataset. A preliminary literature dataset matching the extraction task of this paper is obtained through the professional search function of CNKI.

[0050] Using the CNKI professional search function, the search query shown in Table 1 was used, where LY is the journal name, FT is the full text, TI is the title, and {journal} is filled with the T1 journals in FMS journals. The search time range was set to January 2005 to January 2025, as mentioned by Wen Zhonglin et al. (2005). The retrieved literature is shown in Table 2.

[0051] Table 1. CNKI Search Statements

[0052]

[0053] Table 2. Document Download Status

[0054]

[0055] The retrieved articles were automatically downloaded as PDFs along with their corresponding metadata using Python's Selenium library. Due to data availability issues, some eligible articles could not be downloaded and were skipped here. The final management science literature dataset is shown in the table below. The dataset contains 15,984 journal articles from 26 management science journals. The information units of these articles consist of 13 main parts: title, text, journal, publication year, classification number, keywords, abstract, author, author's affiliation, and publication date. To better understand the overall data and achieve topic identification, statistical analysis and text preprocessing will be performed on this dataset. After obtaining the PDFs, the minerU open-source OCR recognition project was used to convert them into Markdown format, which is compatible with large language models. The Markdown format obtained from minerU, containing references and identified images, was then linked to other local storage paths. Since the insertion method has a fixed format, regular expressions in Python can be used to clean the converted Markdown text.

[0056] Data cleaning results are as follows Figure 1 As shown, the total number of characters in the text before cleaning was 1,190,605,585, and the total number of characters in the text after cleaning was 603,760,838. Specifically, the average number of characters per article per journal decreased from 29,900 to 15,162, a reduction of 49.28%. Converted to the large language model's calculation unit (1 English character ≈ 0.3 tokens, 1 Chinese character ≈ 0.6 tokens), this represents a reduction from 17,940 tokens to 9,097 tokens. This effectively reduced the cost of this extraction task while also improving its quality and efficiency.

[0057] 2. Extraction of management proposition relations based on large language models

[0058] This embodiment employs the management propositional graph extraction (MPGE) framework to execute a three-stage pipeline of "empirical identification—summary generation—thought chain extraction":

[0059] 2.1 Empirical Identification Stage

[0060] To ensure the accuracy and effectiveness of subsequent propositional relation extraction tasks, a high-quality, highly relevant dataset is crucial. Although preliminary screening using keywords and abstracts was conducted during the initial dataset construction phase, this method may introduce some false positives—that is, while the literature mentions terms such as "mediation effect" and "moderation effect" (especially in literature reviews), these empirical tests have not actually been performed in the researchers themselves. To overcome this challenge, this framework designs a dataset filtering module based on a large language model as a precise pre-screening step before entering the core extraction process.

[0061] The core objective of this module is to conduct a secondary, in-depth content review of the initially selected literature collection to confirm whether each article truly implements the empirical analysis methods focused on in this embodiment. Compared to traditional keyword matching methods, this module leverages the powerful semantic understanding and contextual analysis capabilities of LLM. It is no longer limited to isolated words but can read the entire text, especially key sections such as "Research Methods" and "Data Analysis / Results," to determine the existence of empirical testing from the article's practical operational perspective, thereby achieving higher-precision filtering.

[0062] The specific implementation process for this stage is as follows:

[0063] The cleaned full text of the document was input into the first discriminant model (using the DeepSeek-v3.2 large language model), and a second verification of the empirical test was performed using the prompt words shown in Table 3:

[0064] Table 3 Empirical Identification Prompts

[0065]

[0066] Empirical identification results: Through the second test in the first stage, we selected 9601 documents from 15984 articles in the dataset for target empirical analysis, accounting for 55.16%. This indicates that using professional search terms on CNKI (China National Knowledge Infrastructure) still has errors and omissions, and there is still room for further optimization. Figure 2 In the data, we can see the distribution of whether each journal used objective empirical testing methods for empirical testing. Results that conducted objective empirical analysis were dominant, and the overall distribution was balanced.

[0067] 2.2 Empirical Summary Generation Stage

[0068] Since management literature typically contains a wealth of detailed information, directly extracting relations from the entire text not only faces the limitations of a large language model's context window but may also lead to information redundancy, causing the model to pay less attention to the core logic and thus affecting the accuracy of extraction. To address this issue, this framework introduces a structured empirical summary generation stage before the core extraction task, aiming to provide global information support for the subsequent model extraction of propositional relations.

[0069] To ensure that the generated abstract comprehensively reflects the core of the research while possessing internal logical coherence, this invention references the IMRAD (Introduction, Methods, Results, and Discussion) framework, a widely adopted academic writing framework in management and social sciences. The IMRAD framework provides a clear blueprint for accurately and efficiently summarizing the core logic of an empirical study. A standard empirical paper's "Introduction" section presents the research background and questions, naturally leading to the "Methods" section. In the Methods section, the authors elaborate on their research design and the testable research hypotheses derived from it. The subsequent "Results" section directly presents the findings of testing these hypotheses.

[0070] Given that research hypotheses serve as a bridge between theory and empirical evidence and are the direct object of propositional relationship extraction, we emphasized them when applying the IMRAD framework. Our designed abstract generation framework aims to guide LLM to accurately capture the core information across the following four levels: Research Question (corresponding to Introduction): clearly defining the research's starting point and the core problem it aims to solve; Research Methods (corresponding to Methods): summarizing the core empirical testing methods used in the research (such as mediation and moderation effects); Research Hypotheses (embedded in Methods): completely and accurately listing all core research hypotheses proposed in the original text; Research Results (corresponding to Results): summarizing the main findings and conclusions after testing each hypothesis.

[0071] By employing this strategy based on the IMRAD framework and strengthening hypothesis extraction, the generated abstracts can accurately and efficiently summarize the core logical chain of an empirical study, laying a solid foundation for subsequent relation extraction tasks.

[0072] The specific implementation process for this stage is as follows:

[0073] The identified full-text documents are input into the first generation model, which generates structured empirical abstracts based on the IMRAD (Introduction, Methods, Results, and Discussion) academic framework. The prompt word design is shown in Table 4.

[0074] Table 4 Empirical Abstract Keywords

[0075]

[0076] 2.3 Mind Chain Extraction Stage of Integrated Summary

[0077] Few-shot Chain-of-Thought Prompting (Few-shotCoT) combines the advantages of few-shot learning and chain-of-thought reasoning. By providing examples and their reasoning processes within prompts, it guides large language models to progressively complete complex tasks. Specifically, Few-shot CoT not only provides the model with the correspondence between inputs and outputs but also demonstrates the intermediate reasoning steps, thereby helping the model better understand the logical structure of the problem and generate high-quality solutions.

[0078] This invention constructs a multi-stage extraction framework combining few-sample chain-thinking prompts to extract articles from 26 management journals in a dataset (each journal contains nj articles, j=1,2,…,26). The process is divided into four stages: empirical judgment, abstract generation, variable relationship extraction, and judgment evaluation, as follows:

[0079] The first stage is a secondary verification of whether empirical tests have been conducted. Having already undergone the initial screening during dataset construction, which identified literature that has empirically tested mediation and moderating effects, this stage involves a second screening to determine the empirical validity of the articles. The original text is then input into the discriminant model. To determine whether the article used objective empirical testing methods:

[0080]

[0081] in, See the appendix table for the determination of the existence of an empirical test for the objective (i.e.) If so, then proceed to the next stage.

[0082] In the second stage, we will re-enter the original text into... Model, and in conjunction with (See Appendix for details) to generate a structured summary of the empirical process, including the empirical background, empirical analysis methods, and main research conclusions:

[0083]

[0084] In the third stage, after obtaining the abstract, we will... ,summary Few-sample mind chain prompts integrated with the design and global content Enter together The model is used to extract structured triples between research variables:

[0085]

[0086] in, Article The relationships between variables that identified multiple empirically tested effects.

[0087] Table 5 shows the template for designing prompts for propositional relationships in the field of management in a large model, which mainly includes the following four key parts:

[0088] (1) Task description: Define the task objectives and set the roles. The main model for this task is an expert who is proficient in empirical analysis in the field of management.

[0089] (2) Extraction steps: Design a five-step process to guide the model to read the text, identify the relationship between variables, judge the significance, organize the effects and output JSON, and ensure that the task execution logic is clear.

[0090] (3) Case Design: This paper designs a sample to help the large model better understand and perform the task. These cases demonstrate the execution method of the task, enabling the model to learn and imitate the reasoning process.

[0091] (4) Output format description: Specify the output format and content requirements after the task is completed. This clear format requirement helps to maintain the consistency and readability of the output, and facilitates subsequent data analysis and result verification.

[0092] Table 5 incorporates few-sample thought chain prompts from the article abstract.

[0093]

[0094] 2.4 Evaluation of Propositional Relationship Extraction

[0095] A systematic controlled experiment was conducted to evaluate the impact of temperature parameters, guidance mechanisms, and the underlying model on the performance of the structured effects extraction task. All experiments were performed on the same dataset and were conducted in three independent replicates to ensure robustness of the results. Evaluation metrics included effect completeness, variable accuracy, relationship accuracy, significance accuracy, and overall correctness, comprehensively measuring the model's ability to reconstruct semantic structures.

[0096] 2.4.1 Temperature Effect

[0097] Temperature, as a core hyperparameter controlling the randomness of generation, has a significant impact on output quality. The lower the temperature setting, the more stable the output and the more stable the model's adherence to rules. For example... Figure 3 As shown, the experimental results indicate that the lower temperature (0.1) is significantly better than the higher temperature setting. Under the dual-guided mechanism, when the temperature increases from 0.1 to 0.9, the overall score decreases from 0.953 to 0.860 (Δ=-9.8%), the effect completeness decreases from 0.973 to 0.943 (Δ=-3.1%), and the variable accuracy decreases from 0.981 to 0.920 (Δ=-6.2%). This trend is consistent across all basic effect types, suggesting that lower temperatures help the model maintain the determinism and structural integrity of the output.

[0098] Furthermore, the destructive effect of increased temperature on complex effects far exceeds that on basic effects. As shown in Table 6, taking the mediated moderating effect as an example, its overall score is 0.743 at a temperature of 0.1, but drops sharply to 0.481 (Δ=-35.3%) at a temperature of 0.9; the score of the moderated mediating effect also falls from 0.801 to 0.593 (Δ=-26.0%). In contrast, the main effect score only decreases by 9.8%. This phenomenon reveals that the construction of complex semantic structures requires higher determinism in the model, and the random perturbations introduced by high temperatures can severely disrupt the logical dependencies between multi-level variables. Therefore, in structured semantic extraction tasks, a temperature of 0.1 is recommended to maximize the reliability and accuracy of the model.

[0099] Table 6 Evaluation results at different temperatures

[0100]

[0101] 2.4.2 Impact of Guiding Mechanism

[0102] The specific implementation of the dual guidance mechanism is as follows:

[0103] Macro-level guidance: By providing a global research logical framework through structured empirical summaries, the model can grasp the overall context and hypothesis system of the research.

[0104] Micro-level guidance: Through step-by-step instructions and specific examples in the few-sample thinking chain prompts, the model is guided to perform fine-grained variable identification, relationship direction judgment, and significance determination.

[0105] To verify the effectiveness of the dual guidance mechanism, we compared four guidance strategies at a temperature of 0.1: dual guidance, macro guidance only, micro guidance only, and direct prompting.

[0106] As shown in Table 7, the experimental results clearly demonstrate that the dual guidance mechanism significantly outperforms other strategies across all evaluation dimensions. Its overall score (0.953) is 14.3% higher than macro-guidance (0.834), 51.0% higher than micro-guidance (0.631), and 56.2% higher than direct prompting (0.610). In the core indicator of "effect completeness," dual guidance (0.973) also significantly outperforms other strategies, confirming its crucial role in ensuring the integrity of the effect structure.

[0107] Further analysis revealed significant differences in the performance of individual guidance strategies. Macro-level guidance, serving as a robust baseline, achieved the highest overall score (0.834) among the single strategies, particularly demonstrating stability in main effects (0.834) and mediation effects (0.766), indicating that macro-level structural cues effectively constrain the basic framework of the model's output. In stark contrast, micro-level guidance performed extremely poorly in complex effects. The moderated mediation effect achieved an overall score of only 0.088, even lower than the unguided strategy (0.041); the moderated effect score was also only 0.271. This result profoundly reveals that relying solely on micro-level keyword or sentence structure cues is insufficient to support the model's construction of complex, multi-layered semantic dependency networks. Micro-level guidance must synergize with the macro-level framework to realize its fine-grained optimization value.

[0108] The direct prompting strategy performed the worst across the board, with the lowest overall score (0.610) and the lowest effect completeness (0.757), further confirming that clear guiding signals are a necessary condition for ensuring performance in highly structured tasks.

[0109] Table 7 Comparison Results of Methods

[0110]

[0111] 2.4.3 Multi-model comparison

[0112] Under the conditions of a fixed dual-guide mechanism and a temperature of 0.1, we compared four mainstream large language models: GLM-4.5, DeepSeek-v3-0324, Kimi-k2-0711, and Qwen3-235B-23B-2507.

[0113] As shown in Table 8, the experimental results demonstrate a significant divergence in the structured semantic understanding capabilities of different models. The Kimi model performs best in both overall performance and handling of complex effects. Its overall score (0.967) is tied for first place with DeepSeek (0.967), but when dealing with the most complex "mediated mediation effect," Kimi significantly outperforms DeepSeek (0.836) and the GLM baseline (0.762) with a score of 0.907. Similarly, in handling "mediated mediation effect," Kimi's score (0.918) is also significantly higher than other models. This result indicates that Kimi's internal architecture has unique advantages in modeling multi-level, interactive semantic structures.

[0114] The DeepSeek model performed robustly, with overall capabilities on par with Kimi, but its upper limit of capability in extremely complex scenarios (such as "mediated moderation effects") was slightly lower than Kimi's. The Qwen model, on the other hand, exhibited a "strong basic performance, weak complex performance" characteristic. Its overall score (0.956) was similar to the GLM baseline (0.954), and it performed well in main effects (0.956), but its performance in complex effects was comprehensively weaker than GLM. For example, Qwen's score for "mediated moderation effects" (0.714) was lower than GLM's (0.762), and its score for "mediated moderation effects" (0.792) was also lower than GLM's (0.815). This phenomenon has important implications: a model's general capability or parameter size is not directly equivalent to its performance on specific structured tasks. As a powerful general model, Qwen's advantages may be more prominent in free text generation, while its capabilities are not fully realized in tasks requiring strict adherence to a specific logical framework. This also indirectly confirms the targeted activation effect of the dual-guidance mechanism proposed in this embodiment on the model's capabilities.

[0115] Table 8. Results of Multi-Model Comparison

[0116]

[0117] 2.4.4 Consistency Test Results

[0118] To verify the reliability and substitutability of the Automated Evaluation Framework Based on Large Language Models (JudgeLLM) in management proposition extraction tasks, this embodiment manually labeled a benchmark dataset containing 290 samples and used Cohen's Kappa coefficient as the core indicator to conduct a human-machine consistency test. The empirical results (see Table 9) show that the automated evaluation and human expert evaluation exhibit extremely high consistency overall, with an overall weighted Cohen's Kappa coefficient reaching 0.804 (p<0.001) and an accuracy rate as high as 97.32%. According to the criteria of Landis & Koch (1977), this result is in the near-perfect consistency range, which fully demonstrates that JudgeLLM can accurately understand the evaluation rules and has the ability to perform high-quality automated evaluation of large-scale literature.

[0119] Table 9 Consistency Test Results

[0120]

[0121] Further heterogeneity analysis of the evaluation dimensions using a confusion matrix revealed that the model exhibited differentiated discriminative characteristics when handling different types of judgment tasks. Firstly, the model performed best in the dimensions of relational accuracy and significance accuracy, which involve explicit logical inferences, achieving Kappa coefficients of 0.869 and 0.856 respectively, with no false positive errors (FP=0) in either dimension. This indicates that JudgeLLM possesses extremely high accuracy and robustness in handling objective and explicit logical judgments such as positive / negative correlation directions and statistical significance levels, avoiding the hallucination errors common in large models.

[0122] In summary, this embodiment constructs a proposition relation extraction framework (MPGE) for the field of management, aiming to address the challenges of unstructured empirical literature and insufficient annotation. Based on cleaned high-quality FMS journal data, this embodiment proposes a three-stage automated process of "empirical evidence identification—abstract generation—thought chain extraction." Experimental results demonstrate that the dual-guidance mechanism and low-temperature parameter configuration significantly optimize the model's extraction performance for complex nested logic, and the constructed automated evaluation system achieves a high degree of consistency with human experts.

[0123] 3. Named Entity Disambiguation in Management Propositions

[0124] This embodiment employs the management propositional named entity disambiguation framework MP-NED (Management Propositional Named Entity Disambiguation), which specifically includes:

[0125] 3.1 Knowledge Distillation and Data Construction

[0126] Leveraging the powerful contextual understanding capabilities of a large language model (DeepSeek-v3.2 in this embodiment), context-aware definition extraction is implemented. For each entity instance extracted from the corpus... and the paragraph context in which it is located We designed prompt words The model is guided to act as an "expert in the field of management," generating precise and concise semantic definitions. :

[0127]

[0128] Based on the generated entity definitions, this embodiment further utilizes a teacher model to perform continuous semantic scoring on term pairs and definition pairs to obtain fine-grained semantic supervision signals. Let the teacher model be... (This embodiment uses DeepSeek) For any given text pair Mapping it to a continuous semantic space This embodiment guides the process through prompts. Output semantic confidence based on domain knowledge :

[0129]

[0130] in This represents the management context constraints injected. Based on this strategy, we constructed datasets containing term similarity scores. and datasets containing definitions of semantic scores The latter introduces the term name. As an auxiliary context, to eliminate ambiguity in polysemous scenarios, ensuring the model can learn deep semantic distance at the definition level. For the subsequent fine-tuning of the large model used for final disambiguation, this embodiment further constructs an instruction fine-tuning dataset containing explicit inference paths to address the problem of sparse long-tail samples. When dealing with polysemous phenomena, the definitional differences of the same term under different theoretical perspectives are highly hidden, making it difficult to train the model's boundary recognition ability solely based on simple discriminative labels. Therefore, this embodiment introduces an adversarial data generation strategy (Adversarial Synthesis). For real entities... and its definition We utilize Generate adversarial definitions with highly similar syntactic structures but contradictory theoretical implications. The model is required to output a complete thought chain as the training objective. This generation process can be formalized as a conditional probability maximization problem:

[0131]

[0132] Through the above process, this embodiment transforms the implicit domain knowledge contained in the teacher model into structured continuous semantic tags and explicit logical reasoning text, ultimately constructing a model containing... Bar label embedded data and The domain fine-tuning dataset of reasoning enhancement instructions provides a data foundation rich in fine-grained semantic supervision signals for subsequent model fine-tuning.

[0133] 3.2 Training Domain-Specific Models

[0134] To inject fine-grained management semantic knowledge obtained through knowledge distillation into the vector space, this embodiment constructs a dedicated domain embedding model. Unlike general pre-trained language models (PLMs) that only focus on the contextual representation of general corpora, this embodiment aims to construct a vector manifold space with stronger semantic isotropy and more accurately reflecting the distance between concepts in a specific domain. Given the continuous numerical characteristics of the distilled data (semantic confidence from 0.0 to 1.0), this embodiment abandons the traditional discrete classification fine-tuning paradigm and adopts a regression-based continuous semantic supervision strategy to fine-tune all parameters of the base model.

[0135] 3.2.1 Domain Embedding Model Training

[0136] This embodiment selects HFL / Chinese-MacBERT-Base as the backbone network for semantic encoding. This model is based on a multi-layer bidirectional Transformer-Encoder architecture and captures long-distance dependencies and deep syntactic features in text sequences through a multi-head self-attention mechanism. Compared to the native BERT, MacBERT demonstrates superior feature extraction capabilities in Chinese semantic understanding tasks through whole-word masking and N-gram masking strategies.

[0137] Let the input text sequence be ,in The maximum sequence length is set (as per this experiment's setting). The model first uses a tokenizer to segment the data, and then adds special classification tags at the beginning and end. With separator mark Construct the input sequence .

[0138] Input representation of the model It is composed of three embedded vectors added element-wise, aiming to capture text features from different dimensions:

[0139]

[0140] in, Map discrete token indices (Input IDs) to dense word vectors; A position vector is generated for each position index to compensate for the lack of sequence order information in the self-attention mechanism; Paragraph information used to distinguish different sentences.

[0141] Furthermore, given the variability in input text length, this embodiment introduces an attention mask matrix. This is to eliminate the interference of padding tokens on gradient updates. The mask matrix is ​​defined as follows:

[0142]

[0143] During the self-attention calculation process of Transformer, the mask matrix is ​​transformed into a very large negative infinity term (−∞) and superimposed on the Attention Score matrix, thereby ensuring that the model only focuses on effective semantic information.

[0144]

[0145] Subsequently, through The Transformer encoder performs feature extraction in the first layer. The hidden state output H(k) of the layer can be expressed as: , in The hidden layer dimension. This is used to obtain a fixed-length sentence vector representation. This embodiment employs a mean pooling strategy, which calculates the average of all valid token vectors output from the last layer. This has been proven to outperform simple mean pooling in semantic similarity tasks. mark:

[0146]

[0147] In the formula This is a mask indicator function that ensures that padding characters are not included in the calculation.

[0148] Traditional embedding fine-tuning often models the task as a binary classification problem (similar / dissimilar), using InfoNCE or Triplet Loss. However, this discrete labeling loses the confidence information output by the teacher model (e.g., a similarity of 0.6 represents a moderate semantic relationship that is related but not synonymous). Therefore, this embodiment constructs a Cosine Similarity Regression Loss, directly utilizing continuous soft labels generated by knowledge distillation. Guide students in training their models. For a given text pair... The model encodes them into vectors respectively. and We define the predicted semantic similarity. Let the cosine distance between two vectors in the feature space be:

[0149]

[0150] in Let L2 norm be denoted. The optimization objective aims to minimize the predicted similarity. Soft labels generated by the teacher model The mean square error (MSE) between them.

[0151] Overall loss function The definition is as follows:

[0152]

[0153] in For all trainable parameters of the model, The batch size is denoted by . This loss function not only constrains the boundaries between positive and negative samples, but also forces the model to fit the intermediate distribution of semantic similarity, thereby significantly improving the semantic space's ability to distinguish fine-grained differences.

[0154] 3.2.2 Training of the TermJudge Discriminant Model

[0155] This embodiment constructs a dedicated discriminative large language model (LM). Considering the enormous memory overhead and catastrophic forgetting risk associated with fine-tuning large language models with all parameters, this embodiment employs a parameter-efficient fine-tuning strategy based on Low-Rank Adaptation (LoRA). This strategy aims to freeze the backbone parameters of the pre-trained model and adapt it to the entity disambiguation task in management science only by optimizing the low-rank factorization matrix, thereby achieving an optimal balance between computational efficiency and model performance.

[0156] In this embodiment, Qwen3-1.7B is selected as the student model. This model adopts a standard Decoder-only Transformer architecture and has powerful general text generation capabilities. The entity disambiguation task is modeled as a conditional text generation problem.

[0157] Let the input be ,in For task instructions. The output contains the context information of the entity to be disambiguated and its definition; ,in For the reasoning process of the thought chain, This is used to determine the final label. The model's goal is to maximize the output given the input. Generate target sequence Conditional probability:

[0158]

[0159] in For model parameters, For the first A generated token, This represents the total length of the output sequence.

[0160] Based on the inherent dimensionality assumption of large model parameter updates, the LoRA strategy does not directly update the pre-trained weight matrix. Instead, it is achieved by injecting two trainable low-rank matrices. and To simulate parameter update increments Specifically, for the query and value projection matrices in the Transformer layer, we freeze... And introduce the decomposition matrix and ,in It is the rank.

[0161] During forward propagation, the input vector Hidden layer output The calculation is as follows:

[0162]

[0163] In the formula, This is a scaling factor used to balance the contributions of pre-trained features and fine-tuning increments. In this embodiment, r is set to 8 and... =16. Matrix A is initialized with a Gaussian random distribution, matrix... Initialize to a zero matrix to ensure safety during the initial training phase. This preserves the model's pre-training capability. In this way, the number of trainable parameters is reduced to less than 1% of the total number of parameters, significantly reducing memory usage.

[0164] To enable the model to learn the reasoning logic of the teacher model (DeepSeek-v3.2), this embodiment employs an instruction fine-tuning paradigm that includes thought chains. The training data originates from the aforementioned constructed... We format each data entry as shown in Table 10:

[0165] Table 10. TermJudge Fine-tuning Tip Templates

[0166]

[0167] To focus on learning reasoning logic, our model training uses standard autoregressive language modeling loss, calculating cross-entropy loss only for the Response part, while masking the Input and Context parts. The loss function is defined as:

[0168]

[0169] in This introduces a set of trainable parameters for the LoRA module. By minimizing this loss, the student model progressively mimics and internalizes the teacher model's multi-step reasoning ability when dealing with complex polysemous and homosemous samples.

[0170] Specifically, in the supervised fine-tuning (SFT) stage, this embodiment selects 8-bit AdamW as the optimizer to optimize memory efficiency, and uses a Cosine decay strategy combined with linear warm-up for learning rate scheduling. To ensure the reproducibility of the experiment and balance training efficiency and model performance, the specific training environment settings and detailed hyperparameter configurations are shown in Table 11.

[0171] Table 11 Fine-tuning parameters

[0172]

[0173] 3.3 Disambiguation of polysemy

[0174] This embodiment proposes a disambiguation framework that combines an early stopping strategy based on semantic boundary detection with a hybrid clustering strategy. This framework utilizes the geometric edge features of the vector space for rapid semantic consistency verification and incorporates manifold learning to adaptively deconstruct complex semantic structures, thereby significantly reducing resource consumption while ensuring effective capture of complex semantics.

[0175] 3.3.1 Semantic Span Tensor Construction and Boundary Detection

[0176] Let the high-frequency entities extracted from the document collection be... Its set of instances in different contexts is represented as First, the domain-defined embedding model obtained from the above fine-tuning is used. Each instance and its corresponding context definition Mapping to a high-dimensional semantic vector space yields a vector set. To quantify the degree of semantic divergence of an entity in the current corpus, this embodiment introduces the concept of maximum semantic span based on the "semantic convex hull" hypothesis. That is, if the pair of instances with the greatest semantic difference in a set still belong to the same concept, then the set as a whole possesses semantic consistency. The algorithm uses a global scan to select the instances with the lowest cosine similarity (i.e., the greatest geometric distance) in the vector space. For boundary samples, denoted as :

[0177]

[0178] In this embodiment, k=3 is set, focusing on the "boundary pairs" with the greatest geometric distance in the vector space. Based on the convex hull assumption, if sample pairs located at the edge of the semantic distribution are still classified as the same concept, then the samples within them have a high degree of semantic consistency. This strategy minimizes computational redundancy while ensuring disambiguation recall. This step provides a crucial anchor point for subsequent efficient discrimination by locating the geometric edges of the entity semantic distribution.

[0179] 3.3.2 Early Stopping Detection Mechanism Based on Large Language Model

[0180] To minimize computational overhead while maintaining disambiguation accuracy, this embodiment introduces a finely tuned large-scale discriminative model. Semantic consistency verification is performed on boundary samples. This discrimination process is modeled as a binary hypothesis testing problem: null hypothesis... Entity It is a monosemous word; alternative hypothesis It is considered a polysemous word. This is based on the results of the discrimination. The algorithm implements an early stopping and diversion strategy: if the condition is met...

[0181]

[0182] If all boundary sample pairs are determined to have the same semantic meaning, then the null hypothesis cannot be rejected. At this point, the algorithm infers the set. Since there are no significant semantic splits internally, early stopping is implemented, directly mapping all instances to unique entity IDs. This strategy successfully reduces the disambiguation complexity of a large number of high-frequency monosemous words from O(N²) to constant time. This significantly improves processing efficiency. Conversely, if any boundary sample is determined to have a semantic conflict, then it is accepted. Then, initiate the subsequent fine-grained clustering process.

[0183] 3.3.3 Hybrid Clustering Strategy

[0184] This embodiment constructs an adaptive distribution strategy based on the sample size N. For sparse scenarios with extremely small sample sizes (N<6), due to the lack of sufficient density to construct statistically significant clusters, this embodiment abandons unstable unsupervised clustering and instead establishes a semi-supervised anchor point semantic gravity model. This model uses the identified opposing anchor point pairs... Calculate the semantic centroid for the instance to be classified. Cosine similarity to the anchor point. To prevent misclassification of highly heterogeneous new senses, an outlier detection threshold is introduced. (This embodiment is set) Construct the following discriminant function:

[0185]

[0186] In the formula, This represents a potential third meaning independent of known anchor points. The formula indicates that a sample is only assigned to the nearest anchor cluster based on the maximum similarity principle when its similarity to any anchor point exceeds a threshold; otherwise, it is identified as a new semantic branch, thus achieving open set recognition in small-sample environments. For dense scenarios with sufficient samples (N≥6), this embodiment designs a cascaded clustering link of "density detection-structure optimization". First, the density-based HDBSCAN algorithm is used to mine the natural topological structure in the vector manifold space. If the algorithm can identify K≥2 high-density clusters, the result is directly adopted to preserve the natural distribution characteristics of the data. If HDBSCAN fails due to high semantic cohesion (i.e., only a single cluster is identified or all are noise), the adaptive K-Means algorithm is activated. The algorithm traverses the solution space K∈[2,5], optimizing hyperparameters with the silhouette coefficient as the objective function. This step can automatically discover potential ternary and multi-ary semantic structures within concepts.

[0187] 3.3.4 Anchor point consistency verification

[0188] Although cascaded clustering performs well in most cases, unsupervised algorithms can still experience "semantic drift" in extreme cases of extremely high-dimensional semantic entanglement, where logically opposing anchor points determined by LLM are misclassified into the same cluster. To ensure the logical rigor of the disambiguation results, this embodiment designs an anchor point consistency verification mechanism as a final line of defense. Let the opposing anchor points determined in the first step be... and The clustering label output from the second step is mapped as follows: Validation function The definition is as follows:

[0189]

[0190] If the verification result is "Pass," it indicates that the clustering result successfully preserved the core ambiguous logic, and it is output; if it is "Fail," it indicates that the geometric proximity of the vector space masked the logical differences. At this point, a forced anchor projection strategy is triggered. and The opposite poles are redistributed based on the nearest neighbor principle. The corrected label Lcorrected(xi) is calculated as follows:

[0191]

[0192] Through the above process, this embodiment achieves the supervision and error correction of vector space clustering by LLM logical discriminant. Finally, based on the verified cluster partitioning results, this embodiment adopts a structured numerical indexing strategy to assign entity identifiers with unique sequence suffixes (such as Entity_1, Entity_2) to each semantic cluster, ensuring the semantic completeness and academic rigor of the disambiguation results.

[0193] 3.3.5 Experimental Verification

[0194] To verify the effectiveness of the entity disambiguation method proposed in this paper, we conducted experiments on a dataset. In the first stage, we extracted the definitions of entities in the management domain using a large language model.

[0195] Based on the selected empirical research literature in management, this embodiment constructs a comprehensive terminology definition library. After deduplication and frequency statistics, a total of 36,771 variable entities were extracted. Among them, as many as 30,844 isolated entities appearing only in a single document (accounting for 83.9%) were directly retained in this embodiment because there is no cross-document contextual conflict between these entities.

[0196] The study focuses on 5,927 variable entities that appear at least twice, using them as the core candidate set for polysemy disambiguation. The frequency distribution of these entities exhibits a significant long-tail characteristic: 3,011 entities appear twice (50.8%), 1,039 appear three times, while only 464 high-frequency core constructs appear more than 10 times (7.8%). Figure 4 As shown, this data distribution indicates that the disambiguation task must simultaneously consider the processing efficiency of a massive number of low-frequency words and the semantic depth of a small number of high-frequency core words.

[0197] Based on data characteristics, this embodiment adopts a flexible traffic splitting strategy for 5927 polysemous entities to be judged. Figure 5 The Sankey diagram visually illustrates the traffic distribution and final results of these entities under different processing paths.

[0198] First, a boundary detection algorithm was used to check the semantic consistency of all candidate entities. The results showed that the vast majority of entities exhibited significant semantic stability: a total of 5,554 entities (93.7%) met the early stopping condition in the first round of testing and were directly identified as monosemous words and removed from the subsequent calculation process. This strategy successfully concentrated the inference computing resources on the remaining 373 substantial polysemous entities.

[0199] For the remaining 373 ambiguous entities to be disambiguated, the algorithm implemented adaptive routing based on sample density features. Of these, 252 entities (67.6%) were imported into the anchor point binary assignment module due to sample sparsity. In this branch, approximately 76.6% of the samples (193) successfully matched to the preset logical anchor points, while 23.4% of the samples (59) were excluded because their similarity to the anchor points was below a threshold. These were identified as potential new semantic dimensions (semantic dimension expansion), effectively avoiding semantic drift caused by forced classification. On the other hand, 121 dense entities (32.4%) entered the density-based manifold clustering module. The results showed that 70.2% of the clustering results directly passed the LLM logical consistency check, verifying the effectiveness of unsupervised geometric clustering in capturing high-dimensional semantic structures. For the remaining 29.8% of clustering results with logical conflicts, the algorithm triggered a forced anchor projection strategy for semantic logic reconstruction, ensuring the final disambiguation results were rigorous in both geometric distribution and logical definition.

[0200] To further verify the effectiveness of the disambiguation algorithm in capturing fine-grained semantic differences, Table 12 shows some high-frequency terms identified by the algorithm as having multiple meanings and their corresponding heterogeneous definition instances. By calculating the cosine similarity of the definition vectors in different contexts, the semantic changes of the same term under different research paradigms in management can be clearly observed.

[0201] Table 12 Calculation of Similarity of Words with Multiple Meanings

[0202]

[0203] Specifically, terminological ambiguity in empirical research often stems from differences in research level or application context. For example, the similarity scores of "human capital level" and "industrial restructuring" in the vector space are 0.5899 and 0.5762, respectively, lower than the typical level for monosemous terms. This numerical difference reflects the distinction between their definitions from a micro-enterprise perspective and a macro-regional perspective, validating the algorithm's ability to differentiate across cross-level indicators. Furthermore, the similarity score for the term "prosocial behavior" is only 0.4556, reflecting the significant differences in this concept under different perspectives: on the one hand, it represents "individual altruistic behavior" in the traditional viewpoint, and on the other hand, it represents "entrepreneurial political association behavior" in a specific management context.

[0204] Through the aforementioned multi-stage entity disambiguation strategy, this embodiment ultimately deconstructs 373 polysemous entities into 893 fine-grained concepts with independent semantic references. This process eliminates the referential ambiguity of high-frequency core terms, providing a clean set of entities for the accurate extraction of subsequent triple relations, and ensuring the accuracy and logical consistency of the graph entities.

[0205] 3.4 Disambiguation of Multiple Words with One Meaning

[0206] This embodiment proposes and adopts a two-stage "clustering-exploration" strategy.

[0207] 3.4.1 Clustering based on name features

[0208] This embodiment first utilizes the signal-to-noise ratio principle in information theory, leveraging the high signal-to-noise ratio characteristic of entity names as direct symbolic representations of concepts, to construct high-confidence semantic anchors. Let the complete set of entities be... First, a finely tuned domain terminology embedding model is used. Get the name vector representation of all entities:

[0209]

[0210] Considering that the sparsity of high-dimensional vector spaces may weaken clustering results, this embodiment introduces the UMAP algorithm, which aims to map vectors to a low-dimensional manifold space while preserving the local topological structure of the data. This process can be formally described as follows:

[0211]

[0212] in This is the dimensionality-reduced embedding representation. On this low-dimensional manifold, the algorithm performs density-based HDBSCAN clustering to generate an initial set of core synonym clusters:

[0213]

[0214] Because name features have extremely high precision, this step can efficiently merge literal cognate entities such as lexical variants and simple abbreviations, thereby significantly reducing the search space for subsequent calculations.

[0215] 3.4.2 Intra-cluster cleaning based on similarity ranking

[0216] To further eliminate edge noise introduced during clustering and improve fine-grained semantic purity within clusters, this embodiment abandons the coarse centroid comparison strategy and instead implements a refined cleaning mechanism based on local neighborhoods. For the initial clusters any entity in The algorithm first constructs a local search space within the cluster to retrieve results related to... The name vector with the highest similarity Neighbor sequence The algorithm introduces a large model to analyze entity pairs in the sequence. Perform semantic consistency judgment one by one, and denote the judgment result as... To strike a balance between computational efficiency and noise robustness, this embodiment defines a truncated index. The formal expression for the first position that satisfies the following termination condition is as follows:

[0217]

[0218] The above formula incorporates dual filtering logic: where the similarity threshold is the first condition, set... This ensures that only head neighbors highly similar to the target entity are retained, directly truncating tail noise with a sharp drop in similarity; the dynamic tolerance window is the second condition, introducing a tolerance parameter. That is, when the sliding window is continuous When a neighbor is determined to be non-synonymous (y=0), it is inferred that the local semantics have drifted, and an early stop is triggered.

[0219] Ultimately, the cleaned set of valid synonyms consists of the positive samples before the cutoff point, i.e.:

[0220]

[0221] This strategy effectively overcomes the rigidity problem of the single threshold method, and significantly reduces the ineffective reasoning overhead while ensuring semantic consistency and no duplication within the cluster.

[0222] 3.4.3 Definition-Enhanced Synonym Recall

[0223] However, relying solely on surface-level features of names cannot solve the lexical gap problem, i.e., entities that are literally different but semantically identical. To overcome this limitation, this embodiment introduces a definition view as a supplement to deep semantics, performing a global exploration of all entities. For each entity in the dataset... (Regardless of its clustering state), the algorithm constructs a defined vector space. The process performs an approximate nearest neighbor (ANN) search. This step leverages the explanatory text of the definition to overcome literal barriers and retrieves nodes with a definition similarity higher than [previous value]. Top-K neighborhood set To avoid redundant calculations, the algorithm forcibly removes entities that have already been merged during the intra-cluster cleaning phase from the search results, retaining only potential candidates from outside the cluster. To accurately identify synonym relationships among a massive number of candidate pairs, this embodiment employs a dual gating mechanism. As an admission filter for large model discrimination:

[0224]

[0225] in, Used to filter semantic synonyms across words. Only when At that time, entity pair and its context Only then will it be sent to the fine-tuned adjudication model. Perform the final binary semantic reasoning.

[0226] 3.4.4 Semantic Determination and Normalization of Large Models

[0227] Final judgment result Depend on The specific formula is given as follows:

[0228]

[0229] Table 14: Hints for Judging Multiple Words with One Meaning

[0230]

[0231] in, As shown in Table 14, if The system then performs entity normalization at the graph level, mapping the reference relationship between two entities to a globally unique normalized ID. This strategy, through feature decoupling and cascaded filtering, ensures recall while concentrating the expensive computational power of large models on previously difficult-to-determine fuzzy boundaries, achieving an optimal balance between disambiguation accuracy and computational efficiency.

[0232] 3.4.4 Verification Experiment

[0233] To effectively compress the semantic space and identify potential synonyms, this embodiment first performs rigorous deduplication on the 60,849 entities in the entity set after polysemy disambiguation, ultimately retaining 37,291 unique entities for vector space analysis. UMAP dimensionality reduction and HDBSCAN density clustering (parameters set to dimension=32, min_cluster_size=20, min_sample=20) are then used to generate the semantic manifold distribution as follows: Figure 6 As shown in the visualization, a large number of entities form dense semantic clusters in the vector space (colored areas in the figure), indicating that management terms have a significant clustering effect in naming. At the same time, some outliers are still scattered at the edges of high dimensions, which further verifies that a single name clustering is difficult to cover all semantic forms and needs to be combined with subsequent definition views for global completion.

[0234] Through a two-stage disambiguation strategy, this embodiment ultimately identified 7,690 pairs of high-confidence synonymous entity pairs. For example... Figure 7 As shown, the sources of disambiguation exhibit significant complementary characteristics: intra-class disambiguation (first stage) contributed 3,754 pairs (48.8%), mainly covering morphological variants with highly similar names; while the third stage of out-of-class recall contributed 3,936 pairs (51.2%), which is comparable to the intra-class disambiguation stage.

[0235] Further analysis of the name similarity distribution between the two types of entity pairs ( Figure 8 It can be observed that the name similarity of entity pairs captured in the first stage is concentrated in the range of [0.85, 1.0], verifying the clustering ability to aggregate literal variants. In contrast, the name similarity distribution of entity pairs captured in the second stage shifts significantly to the left, with a large number distributed in the low similarity range of [0.3, 0.6]. This difference in distribution intuitively demonstrates the necessity of introducing a definition view in this embodiment—it successfully captures more than 50% of implicit synonyms missed by traditional literal matching methods, effectively bridging the lexical gap between management terms.

[0236] 3.4.5 Micro-semantic analysis of typical cases

[0237] To visually demonstrate the decision-making logic of the disambiguation algorithm at different semantic depths, this embodiment lists typical discrimination examples from two dimensions: intra-cluster merging and inter-cluster merging. Multi-granularity entity disambiguation methods are used to discriminate entities within the same cluster, and merging examples are shown in Table 15:

[0238] Table 15 Entity Merger Cases

[0239]

[0240] In the intra-cluster multi-granularity discrimination stage, Terminology demonstrates precise control over semantic boundaries. Taking "corporate innovation performance" as an example, although it is semantically related to "innovation performance," they completely overlap in their core definitions and measurement dimensions (both pointing to the output results of corporate innovation activities), thus correctly identifying them as synonyms and merging them. Conversely, for "corporate innovation performance" and "corporate financial performance," which have highly similar literal structures, the model keenly captures the essential difference in their constructs: the former focuses on technological and market output, while the latter emphasizes operational financial returns. Although they may be close in the vector space due to high co-occurrence frequency, definition-based logical verification successfully prevents erroneous merging. This result proves that semantic reasoning using the Terminology model effectively avoids misjudgments caused by literal confusion in traditional similarity calculations, ensuring the purity of the graph concepts.

[0241] In the out-of-cluster recall stage (as shown in Table 16), this method uses definition vector retrieval technology to supplement the shortcomings of name matching alone, and identifies entities with large literal differences but the same meaning.

[0242] Table 16 Cases of External Recall Dispute Resolution

[0243]

[0244] First, for synonyms with completely different names, definition vector similarity calculation provides a crucial screening criterion. As shown in Table 16, "employee education investment" and "learning investment" do not overlap in characters literally, but because both definitions emphasize "resource investment by enterprises to improve human capital," they exhibit high similarity in the vector space and are initially screened as candidate pairs. Subsequently, semantic comparison using the Terminology model confirms that their core meanings are consistent, ultimately classifying them as synonyms and merging them. Similarly, although "environmental responsibility" and "green behavior" have different focuses, they are correctly associated through definition vector retrieval and final model confirmation. This demonstrates that definition information can effectively recall terms that are literally unrelated but actually refer to the same concept.

[0245] Secondly, for challenging samples with highly similar definitions but different concepts, the Terminology model played a corrective role. Take "labor mobility" and "degree of labor market integration" as examples. These two terms are interdependent in economic logic, and because both use frequently co-occurring terms such as "factor flow" and "cross-regional allocation efficiency" extensively in their definition generation, their similarity in the definition vector space is extremely high, easily leading to confusion. However, in the final judgment stage, Terminology ultimately determined "no." This result indicates that definition vector retrieval is responsible for expanding the search scope to discover potential synonyms, while introducing the Terminology model for logical judgment is a secondary verification on this basis, avoiding erroneous merging caused by overly similar definition texts.

[0246] Experimental results show that this strategy performs significantly well in semantic dimensionality reduction and concept alignment. Starting from an initial 37,291 candidate entities after deduplication, the algorithm successfully identified and merged 7,690 pairs of synonymous entities. After disambiguation, the entity size of the knowledge graph was finally reduced to 29,601 independent research constructs. This process effectively eliminated redundant noise caused by polysemy, laying a solid foundation for building a high-density management knowledge network.

[0247] 4. Construction and storage of knowledge graphs for management propositions

[0248] 4.1 Data Overview and Scale Statistics

[0249] After extracting propositional relationships and optimizing entities in the field of management, we obtained 66,220 entities (including 29,601 research variable entities, 15,984 document entities, 26 journal entities, and the remainder being moderating event entities) and 245,321 relationships between entities from 15,984 articles in 26 journals. The distribution of entities at a macro scale is shown in Table 14.

[0250] Table 14 Overall Distribution of Knowledge Graph Data

[0251]

[0252] 4.2 Knowledge Graph Pattern Layer Design

[0253] This embodiment uses formal definitions to describe the ontology and relations. Formally, the knowledge graph is represented as a directed multigraph G=(E,R,S), where E is the set of entities, R is the set of relations, and S⊆E×R×E is the set of triples.

[0254] 4.2.1 Ontology Design and Effect Event Modeling

[0255] The entity set E consists of four mutually exclusive subsets:

[0256]

[0257] in, , and These correspond to journals, articles, and variables, respectively. Due to the expression issues of the moderating effect, the moderating effect logically describes a variable ( How to change another variable () ) for dependent variable ( The influence path of an entity is represented by an edge pointing to a relation. However, attribute graph databases typically only support connections between nodes and do not support edges as objects of relations. This embodiment introduces... (Effective event entities) are expressed in a structured manner. To adapt to the storage characteristics of graph databases, this embodiment uses relation instantiation to map the semantic process of "independent variable influencing dependent variable" into independent event nodes. A mapping function is defined.

[0258]

[0259] in The storage structure of the moderating effect is transformed from a single triplet to a nested event structure: First layer (process entityization): the main effect is transformed into an event entity. This entity acts as an intermediate node connecting the independent and dependent variables.

[0260]

[0261] The second layer (regulatory intervention): establishes a relationship between the regulatory variable and the event entity.

[0262]

[0263] By introducing The ontology map is compatible with complex theoretical models, including mediated regulation and regulated mediation. Specific definitions for each ontology type are shown in Table 15.

[0264] Table 15. Body Design Table

[0265]

[0266] 4.2.2 Relationship Type Design

[0267] Based on the above ontology model, this embodiment further defines the semantic connection methods between entities, namely, the relation set R. To accurately characterize the organizational structure and theoretical propositions of management literature, this embodiment divides relations into two main categories: structural relations and semantic relations. Structural relations are mainly used to construct the index framework of the literature, and are formally represented as follows: and This established the subordinate relationship between the article and the journal, as well as the subordinate relationship between the research variable and the source article, thereby achieving knowledge traceability.

[0268] Table 16. Relationship Design Table

[0269]

[0270] Semantic relationships focus on the empirical logic between recurring variables. For conventional causal mechanisms, this embodiment defines linear influence relationships. Nonlinear relationship Directly describe the independent variable For dependent variable The positive, negative, or U-shaped effects. Of particular note is the design of the moderating effect: given that the moderating variable theoretically does not act directly on the dependent variable, but rather exerts its effect by altering the strength of the causal path, this embodiment formally defines it as... In this structure, the regulatory relationship The target of the graph is no longer a single variable, but an effect event entity (eevt) representing the main effect process. This design strictly distinguishes the roles of independent variables and moderating variables in the topological structure, ensuring that the knowledge graph can accurately reflect the deep logic of the "variable moderating process" in management theory when storing complex interaction effects, and avoiding the semantic confusion that may be caused by the traditional triple structure.

[0271] 4.3 Knowledge Graph Construction and Storage Implementation

[0272] Based on the schema layer architecture constructed above, this embodiment selects Neo4j graph database as the physical storage medium for the knowledge graph. Neo4j, with its native graph storage structure and index-free adjacency feature, can efficiently support depth-first traversal and multi-dimensional relational queries on a scale of tens of millions of nodes.

[0273] 4.3.1 Experimental Environment Configuration

[0274] The knowledge graph for management propositions was developed and implemented in Python 3.11, using the py2neo framework as middleware between the application programming interface and the database layer to achieve object-graph mapping and batch data operations. The core storage engine is deployed on Neo4j Community version 4.4.25, and the underlying runtime environment relies on OpenJDK 11. To ensure throughput and real-time computing performance for writing large-scale triplet data, the experimental host machine is configured with 32GB of DDR5 memory and a multi-core high-performance processor, and has been optimized to adapt to the high-density connection characteristics of the management proposition graph.

[0275] 4.3.2 Graph Data Persistence and Complex Logic Storage

[0276] To automate the mapping of massive unstructured triples to graph databases, this embodiment designs a general incremental knowledge graph construction algorithm. This algorithm is no longer limited to specific statistical models but abstracts the construction process into three standardized topology construction stages: first, constructing a source framework centered on journals and articles; second, completing the entityification of research variables; and finally, implementing a differentiated storage strategy based on effect type. Specifically, for simple binary linear relationships, direct edge connections are used, while for complex multivariate higher-order logic such as moderation and mediation, a unified path fragment event-based strategy is adopted for structural reconstruction. By dynamically anchoring the starting and target nodes of the path, this algorithm can flexibly handle moderating effects occurring at any stage of the causal chain. The core construction logic is shown in Table 17.

[0277] Table 17. Pseudocode of the core process of knowledge graph construction

[0278]

[0279] This embodiment, based on the py2neo framework, batch-imported 66,220 entity nodes (covering 29,601 core research variable entities, 15,984 document entities, 26 journal entities, and 20,609 effect event entities) and 245,321 semantic relationship data, which had been cleaned and optimized in previous steps, into the Neo4j graph database, successfully constructing a propositional knowledge graph for the field of management. The topological structure of this graph intuitively presents the multi-dimensional nested relationships between "journals—documents—variables—events," realizing a semantic leap from unstructured text to a structured knowledge network. Figure 9 The diagram showcases a partial visualization of the knowledge network. This highly connected knowledge network not only provides an efficient retrieval portal for theoretical tracing and relationship mining in the field of management, but also lays a data foundation for interdisciplinary pattern discovery and intelligent decision support.

[0280] 4.4 Application of Neo4j Graph Knowledge Base

[0281] 4.4.1 Analysis of Empirical Testing Based on Case Studies

[0282] This embodiment will illustrate how a structured knowledge graph encodes and presents complex theoretical models, using the empirical study "Relational Resource Allocation and Leadership: A Mediated Moderating Model" in the Management Review as an example (e.g.) Figure 10 Orange nodes represent research titles, blue nodes represent core management entities, and green nodes represent defined effects or process events.

[0283] The construction of this knowledge graph begins with the entityification of core variables. In this embodiment, core constructs such as "relational resource allocation," "leadership position," "perceived fairness," and "relationship beneficiary performance" are all defined as management entities. Figure 11 As shown, we can see the basic conclusion of this embodiment, namely the main effect tested in the paper, the significant negative impact of relational resource allocation on leadership, which is represented in the graph as a relationship connecting the independent and dependent variables: (relational resource allocation → significant negative impact → leadership). To clarify the transmission path of this effect, the study examined the mediation effect. This effect in the graph consists of a set of continuous causal relationships, with the structure of (relational resource allocation → significant negative impact → perceived fairness) and (perceived fairness → significant positive impact → leadership). This causal chain clearly reproduces the findings of the paper. Furthermore, to achieve a holistic representation of this multi-step process, the complete path is defined as an independent event entity, whose attributes describe the entire mediation process in a structured language.

[0284] The moderating effect tested in the study is presented in a structured manner through event entities. This interaction is expressed in the knowledge graph as (relationship beneficiary performance → significant positive moderation → event: "Relationship-based resource allocation significantly negatively impacts perceived fairness"). This structure indicates that the moderating variable acts on an identified effect process, directly corresponding to the analytical logic of the interaction terms in the statistical model. Ultimately, the knowledge graph integrates all elements to present the complete model of the study: a tested moderated mediation effect. Its top-level structure is (relationship beneficiary performance → significant positive moderation → event: "Relationship-based resource allocation significantly negatively impacts perceived fairness, thereby significantly positively impacting leadership"). This structure precisely corresponds to the core insight of the study: the moderating variable influences the overall indirect effect of the independent variable on the dependent variable by changing the strength of the mediation process. In summary, by materializing core constructs, typifying causal effects, and eventifying the transmission process, this knowledge graph construction method can transform complex theoretical models deeply embedded in academic texts into logically rigorous, hierarchically distinct, and machine-readable structured knowledge, laying the foundation for large-scale literature analysis and knowledge discovery.

[0285] 4.4.2 Found Inconsistent Conclusions

[0286] This example aims to demonstrate a core analytical capability of knowledge graphs: discovering different, even contradictory, conclusions drawn from different studies regarding the relationship between the same pair of variables. Through searching the entity relationships within the database, the system reveals that there are simultaneously two contradictory relationships between the two management entities, "industrial policy" and "corporate investment efficiency"—one with a significant positive influence and the other with a significant negative influence (e.g., ...). Figure 12 and 13This finding suggests that there is no consensus in academia regarding the true impact of industrial policy on the efficiency of corporate investment, and its effectiveness is likely influenced by different mechanisms and contextual factors.

[0287] The knowledge graph further traces the literature sources and theoretical logic of these two contradictory relationships. The significant positive impact primarily stems from literature such as "A Study on the Impact of Industrial Policy on Corporate Investment Efficiency and Its Mechanism." This study argues that industrial policy, by increasing the availability of bank credit for supported companies, alleviates their funding constraints, thereby promoting more efficient investment activities. Conversely, the knowledge graph also reveals multiple studies supporting a significant negative impact. Studies such as "Industrial Policy, Financing Constraints, and Corporate Investment Efficiency" and "Industrial Policy Tools, Corporate Investment Efficiency, and Stock Price Crash Risk" reach the opposite conclusion: industrial policy support (such as fiscal subsidies and preferential credit) may induce companies to engage in blind or excessive investment, especially in areas encouraged by policy but where companies lack core competitive advantages. This resource misallocation and distortion of investment decisions ultimately leads to a decline in overall corporate investment efficiency.

[0288] Therefore, by juxtaposing these two contradictory causal paths and their underlying theoretical explanations, knowledge graphs clearly outline a core academic point of contention: whether industrial policy in practice promotes the efficient allocation of resources or leads to resource misallocation. Knowledge graphs not only uncover this core conflict but also point to the potential reasons for the differing conclusions—one side emphasizes the positive effects of industrial policy in facilitating financing channels, while the other reveals the potential distortions in resource allocation that it may cause as a form of government intervention. This finding provides a clear entry point for subsequent research, such as exploring under what conditions the positive effects of industrial policy can outweigh its negative effects, and also demonstrates the value of knowledge graphs in systematically organizing and presenting complex academic debates.

[0289] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0290] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for constructing a propositional knowledge graph in the field of management based on LLM, characterized in that, The method includes: S1. Construct an empirical literature dataset in the field of management: Using T1-level Chinese journals in high-quality management science journals as the data source, retrieve and obtain a literature dataset that meets the requirements of empirical research, convert unstructured full-text PDFs into Markdown structured texts and perform data cleaning using regular expressions; S2. Extraction of management proposition relations based on large language model: Using large language model, a three-stage process including empirical identification, empirical summary generation and fusion summary extraction of thought chain is adopted to extract structured proposition triples containing main effect, mediation effect, moderation effect, chain mediation effect, moderated mediation effect and moderated moderation effect from the literature dataset. S3. Named Entity Disambiguation of Management Propositions: Using domain-enhanced data generated based on knowledge distillation and efficient parameter fine-tuning techniques, a domain-specific embedding model and entity discrimination model are trained; based on the embedding model and entity discrimination model, differentiated strategies are adopted for polysemy and homosemantic phenomena to disambiguate and merge entities in the proposition triples to obtain a standardized entity set; S4. Construction and Storage of Knowledge Graph for Management Propositions: Design a knowledge graph pattern layer adapted to the complex causal logic of management, introduce effect event entities to solve the problem of graph database storage for high-order nested causal logic, and complete the construction and persistent storage of knowledge graph for management propositions based on standardized entity sets and proposition triples through Neo4j graph database.

2. The method according to claim 1, characterized in that, In step S2, the extraction of management proposition relations based on a large language model is specifically implemented through the management proposition relation extraction framework MPGE. The three-stage process includes: S21. Empirical identification stage: Use a large language model to determine whether the literature contains empirical tests of the type described in step S2, in order to filter the data; S22. Empirical Abstract Generation Stage: Based on the IMRAD framework, a large language model is used to generate a structured abstract containing the research question, research methods, research hypotheses, and research conclusions; S23. Mind chain extraction stage of fused summary: Input the original text of the document and the structured summary into the large language model, and output proposition triples in JSON format through few-sample mind chain prompts that include role settings, task descriptions, extraction steps, examples and output formats.

3. The method according to claim 2, characterized in that, In step S2, during the thought chain extraction stage, a dual guidance mechanism is adopted, that is, macro guidance and micro guidance are provided simultaneously, and a lower temperature parameter is set to optimize the model's extraction performance for complex nested logic.

4. The method according to claim 1, characterized in that, In step S3, the disambiguation strategy for polysemy includes: using the domain-specific embedding model, constructing a semantic span tensor based on the semantic vector of the entity definition, locating semantic distribution edge samples through a boundary detection algorithm, performing an early stopping diversion strategy for monosemous entities after large model discrimination, using the HDBSCAN algorithm and adaptive K-Means strategy to complete semantic segmentation for polysemous entities, and ensuring the logical rigor of the disambiguation results through anchor point consistency verification.

5. The method according to claim 1, characterized in that, In step S3, the disambiguation strategy for the phenomenon of multiple words with the same meaning includes: adopting a two-stage "clustering-exploration" strategy, first completing high-confidence semantic clustering and intra-cluster cleaning based on entity name features, then completing cross-vocabulary candidate synonym entity pair recall through approximate nearest neighbor search of entity definition vectors, using the TermJudge model to perform final semantic determination on candidate synonym entity pairs, and merging entities determined to be synonyms.

6. The method according to claim 1, characterized in that, In step S4, the schema layer design includes ontology design and relation design. The ontology design includes four sets of mutually exclusive entities: journals, literature, management variables, and effect events. The relation design is divided into two main categories: structural relations and semantic relations. Structural relations are used to construct a literature source index framework, while semantic relations are used to reproduce the empirical logic between variables. The effect event entities are used to achieve a complete reproduction of complex theoretical models such as mediated regulation.

7. A system based on the method of claim 1, characterized in that, include: The data preprocessing module is used to perform step S1 as described in claim 1; The proposition relation extraction module is used to perform step S2 as described in claim 1; An entity disambiguation module is used to perform step S3 as described in claim 1; The map construction and storage module is used to perform step S4 as described in claim 1.

8. An electronic device comprising at least one processor and a memory, said memory storing a computer program, characterized in that, When the computer program is executed by the at least one processor, the electronic device performs the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images