Multi-causal relationship extraction method and system for intelligent evaluation

Abstract

The invention belongs to the field of language processing, particularly relates to a multi-causal relationship extraction method and system for intelligent evaluation, and aims to solve the problems of performance bottleneck, remarkable wooden barrel effect and insufficient adaptability and integration caused by dependence on multiple independent models. The method comprises the steps that a causal analysis prompt template used for generating step-by-step reasoning is constructed to serve as a thinking chain prompt word, a fine adjustment corpus is generated in combination with an original corpus and input into a large language model for training, a low-rank adaptation technology is adopted in the training process to conduct fine adjustment training on the large language model, and a causal relationship extraction model is obtained; inputting corpora of a to-be-extracted causal relationship into the causal relationship extraction model, and outputting a causal relationship extraction result; and generating a comprehensive evaluation result and a decision suggestion. According to the method, the efficiency and the quality of the multi-causal relationship extraction task are improved, the stability and the adaptability of the system are enhanced, meanwhile, the labor cost is reduced, and scientific and reliable evaluation results and decision suggestions are provided.

Description

Background Art

[0002] In the current technical framework, solutions for dealing with complex causal relationships usually rely on a series of step-by-step methods, including but not limited to event element extraction, causal relationship identification, and causal type classification, etc. However, these methods often rely on multiple independent models to complete their respective tasks, which leads to several key problems: Performance bottleneck: Due to the uneven capabilities and accuracies of the independent models, the performance of the entire system is often limited by the weakest link, namely the so-called "wooden barrel effect". This phenomenon greatly affects the performance of the overall task.

[0003] Lack of adaptability: Especially when dealing with scenarios such as the space field with extremely large and complex data volumes, traditional rule-based or single-model methods are inadequate. They are difficult to efficiently process ultra-large datasets and lack the necessary flexibility and adaptability when facing different types of causal relationships.

[0004] Integration challenges: Integrating multiple independent models into an effective system is also a challenge. Issues such as interface design and data format conversion between different models require additional development and maintenance costs, increasing the complexity of the system and potential failure points.

[0005] Based on this, the present invention proposes a multi-causal relationship extraction method and system for intelligent evaluation. Summary of the Invention

[0006] To solve the above problems in the prior art, that is, when dealing with complex causal relationships, due to relying on multiple independent models, there are performance bottlenecks, the "wooden barrel effect" is significant, and there is a lack of adaptability and integration when facing ultra-large-scale data, thus affecting the performance and efficiency of the overall task, the present invention provides a multi-causal relationship extraction method and system for intelligent evaluation.

[0007] In the first aspect of the present invention, there is provided a multi-causal relationship extraction method for intelligent evaluation, the method comprising: Constructing a causal analysis prompt template for generating step-by-step reasoning as a chain-of-thought prompt word, and generating a fine-tuning corpus in combination with the original corpus; wherein, the chain-of-thought prompt word contains multi-stage guiding instructions for text preprocessing, element extraction, and relationship determination; Taking the fine-tuning corpus as input data and inputting it into a large language model, training for causal relationships, and during the training process, using low-rank adaptation technology to fine-tune the large language model to obtain a causal relationship extraction model; Inputting the corpus to be extracted for causal relationships into the causal relationship extraction model, and outputting the causal relationship extraction result; Based on the causal relationship extraction results, combined with the pre-constructed domain event evaluation index system and the construction strategy knowledge base, generate a comprehensive evaluation result and decision-making suggestions.

[0008] Further, the construction process of the chain of thought prompt words is as follows: Step A1: Perform sentence splitting and word segmentation analysis on the original corpus, identify causal marker words in the sentences, and obtain a set of independent sentences with causal markings. Step A2: Locate the causal chain nodes based on the causal marker words, and extract the core elements of the cause part and the result part respectively. The core elements include the action subject, action, object, and spatio-temporal characteristics. Step A3: Determine whether the causal relationship belongs to a direct causal relationship or an indirect causal relationship including intermediate events by analyzing the association path of the causal chain nodes. Step A4: Classify the cause part and the result part into preset event categories respectively. The event categories include at least one of military operations, diplomatic activities, security events, political events, social events, technological developments, economic events, aerospace activities, equipment and armaments. Step A5: Generate structured data in a predetermined format according to the core elements, causal relationship type, and event category. The format includes a causal relationship type field, a cause element set field, and a result element set field. Step A6: Perform integrity verification on the structured data through a multi-round verification mechanism, and dynamically adjust the word segmentation rules and event classification model parameters based on text features to obtain the chain of thought prompt words.

[0009] Further, during the training process, use the low-rank adaptation technology to fine-tune the causal relationship extraction model. The method is as follows: Step B1: Divide the fine-tuning corpus into a training set, a validation set, and a test set. Step B2: Load the pre-trained large language model as the base model of the causal relationship extraction model, and inject a low-rank adaptation matrix into the neural network layer of the base model to form a tunable parameter subspace. Step B3: Perform multi-round iterative training on the base model injected with the low-rank adaptation matrix through the training set, and optimize the hyperparameters in combination with the validation set to obtain a task-based model suitable for multi-causal relationship extraction. Step B4: Evaluate the model performance using the test set, and dynamically adjust the rank parameter of the low-rank matrix and the prompt word embedding strategy according to the evaluation results.

[0010] Further, the method for obtaining the comprehensive evaluation result is as follows: Construct a domain event evaluation index system and perform knowledge vectorization processing; among them, the index system at least includes threat level, impact scope, and risk factors, and the index system is converted into a vector form through knowledge embedding technology and stored in the evaluation knowledge base; According to the causal relationship extraction result, dynamically retrieve and match the evaluation index in the evaluation knowledge base through semantic similarity calculation, and construct an event-specific evaluation system; Based on the complex reasoning large model, fuse the causal relationship extraction result and the event-specific evaluation system for multi-level reasoning, and generate a comprehensive evaluation result including risk factors, impact scope, and priority.

[0011] Furthermore, the method for constructing a domain event evaluation index system is as follows: Obtain the public standard documents, authoritative materials, and industry research results of domain events, and extract qualitative and quantitative evaluation criteria as the domain event evaluation index system.

[0012] Furthermore, the method for obtaining the decision-making suggestions is as follows: Construct a strategy knowledge base, including collecting a set of strategy methods related to domain events. The set of strategy methods includes solution plans, coping strategies, and optimization plans; perform knowledge vectorization processing on the set of strategy methods, convert it into vectorized data that can be called by the large model and store it to form a structured and retrievable strategy knowledge base; Use the comprehensive evaluation result as the input, dynamically retrieve the strategy knowledge base through the knowledge base retriever, and extract the strategy information associated with the current event using semantic matching; Based on the complex reasoning large model, integrate the comprehensive evaluation result and the extracted strategy information, generate decision-making suggestions including specific action plans, priorities, and timeliness, and optimize the feasibility and applicability of the suggestions through multi-round reasoning, and output them in the form of charts or text reports.

[0013] Furthermore, the specific process of generating decision-making suggestions is as follows: Automatically generate short-term action plans and long-term optimization plans according to the event background, risk factors, and strategy information; Optimize the applicability of the suggestions through context adjustment to ensure that they match the risk level of the evaluation result.

[0014] Furthermore, the multi-round reasoning process includes iterative verification of strategy feasibility, resource consumption, and execution timeliness.

[0015] Furthermore, the priority arrangement of the decision-making suggestions is comprehensively determined based on threat urgency, strategy implementation cost, and risk mitigation effect.

[0016] In the second aspect of the present invention, a multi-causal relationship extraction system for intelligent evaluation is proposed, based on a multi-causal relationship extraction method for intelligent evaluation. The system includes: A database construction module configured to construct a causal analysis prompt template for generating step-by-step reasoning as a chain-of-thought prompt word, and generate a fine-tuning corpus in combination with the original corpus; wherein, the chain-of-thought prompt word includes multi-stage guiding instructions for text preprocessing, element extraction, and relationship determination; A training module configured to input the fine-tuning corpus as input data into a large language model, train for causal relationships, and perform fine-tuning training on the large language model using low-rank adaptation technology during the training process to obtain a causal relationship extraction model; A causal relationship extraction module configured to input the corpus for which causal relationships are to be extracted into the causal relationship extraction model and output the causal relationship extraction result; An evaluation and recommendation module configured to generate a comprehensive evaluation result and decision-making recommendations based on the causal relationship extraction result, in combination with a pre-constructed domain event evaluation index system and a construction strategy knowledge base.

[0017] Advantages of the present invention: Improve evaluation efficiency and quality: By using large models and their fine-tuning techniques, especially the low-rank adaptation (LoRA) technique, this method can efficiently handle the multi-causal relationship extraction task in domain events. This not only reduces human participation and the probability of human errors, but also ensures the transparency and interpretability of evaluation results.

[0018] The large model outputs a complete evaluation reasoning process, making each step clearly visible, thereby improving the credibility of the evaluation results.

[0019] Reduce the "barrel effect" and hallucination problems: The chain-of-thought instructions combined with the LoRA fine-tuning technique significantly reduce the hallucination problems of the large model when dealing with complex causal relationships and optimize its instruction-following ability. This makes the entire system more stable and reliable when facing massive data, further improving the accuracy and reliability of task execution.

[0020] By constructing a chain-of-thought prompt word containing guiding instructions for text preprocessing, element extraction, and relationship determination, the problem of a single link becoming a performance bottleneck is effectively avoided, enhancing the overall performance of the system.

[0021] Enhance adaptability and flexibility: Utilize the RAG technique combined with the index system stored in the vector database, adopt a three-dimensional coordinate evaluation method to automatically retrieve the relevant index system, deeply analyze the causal relationship, and complete the risk level assessment of domain events. This combination enables the system to flexibly respond to different types of data and task requirements.

[0022] The introduction of the knowledge base provides reliable auxiliary decision-making suggestions for users, enhancing the practicality and user-friendliness of the system.

[0023] Reduce labor costs: The entire process is highly automated, reducing the dependence on professionals and greatly reducing labor costs. At the same time, through the application of intelligent evaluation methods, work efficiency is improved and the project cycle is shortened.

[0024] Improve the scientific nature of the comprehensive evaluation results and the effectiveness of decision-making suggestions: Combining the pre-constructed domain event evaluation index system and the strategy knowledge base, comprehensive evaluation results and decision-making suggestions are generated, ensuring the scientific nature of the evaluation results and the feasibility of the decision-making suggestions, providing strong support for practical applications. Brief Description of the Drawings

[0025] Other features, objectives, and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings: Figure 1 is a flowchart of a multi-causal relationship extraction method for intelligent evaluation of the present invention; Figure 2 is a flowchart for fine-tuning and training the causal relationship extraction model in a multi-causal relationship extraction method for intelligent evaluation of the present invention; Figure 3 is a flowchart of intelligent evaluation in a multi-causal relationship extraction method for intelligent evaluation of the present invention; Figure 4 is a flowchart of strategy generation in a multi-causal relationship extraction method for intelligent evaluation of the present invention. Detailed Embodiments

[0026] The following further elaborates on the present application with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention and are not intended to limit the invention. Additionally, it should be noted that for ease of description, only parts related to the relevant invention are shown in the drawings.

[0027] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and embodiments.

[0028] The present invention provides a multi-causal relationship extraction method for intelligent evaluation, which includes: Construct a causal analysis prompt template for generating step-by-step reasoning as a thought chain prompt word, and combine it with the original corpus to generate a fine-tuning corpus; wherein, the thought chain prompt word includes multi-stage guiding instructions for text preprocessing, element extraction, and relationship determination; Input the fine-tuning corpus as input data into the large language model, train it for causal relationships, and use the low-rank adaptation technique to fine-tune the large language model during the training process to obtain a causal relationship extraction model; Input the corpus for which causal relationships are to be extracted into the causal relationship extraction model, and output the causal relationship extraction result; Based on the causal relationship extraction result, combine the pre-constructed domain event evaluation index system and the construction strategy knowledge base to generate a comprehensive evaluation result and decision-making suggestions.

[0029] By extracting the causal relationships of domain events, this invention extracts and generates structured causal relationship data, providing high-quality and reliable data foundation support for subsequent intelligent evaluation. Then, combine these structured causal relationships with the index system in the knowledge base, and conduct intelligent evaluation based on the large model to generate comprehensive and accurate evaluation results.

[0030] On this basis, by retrieving the strategy knowledge base, the model will generate reasonable decision-making suggestions, thereby providing efficient and accurate auxiliary decision-making support for users.

[0031] To more clearly illustrate a multi-causal relationship extraction method for intelligent evaluation of this invention, the following combines Figure 1 Expand and describe the embodiments of this invention in detail as follows: See Figure 1 and Figure 2 to construct a causal analysis prompt template for generating step-by-step reasoning as a chain-of-thought prompt word, and combine it with the original corpus to generate a fine-tuning corpus; wherein, the chain-of-thought prompt word contains multi-stage guiding instructions for text preprocessing, element extraction, and relationship determination; In this embodiment, by designing step-by-step prompt words based on logical reasoning, that is, chain-of-thought prompt words, the model is guided to understand and extract causal relationships in a step-by-step reasoning manner, thereby improving the accuracy and logic of causal relationship extraction. Collect the original corpus related to domain events, such as scientific literature, accident reports, technical descriptions, and news reports, etc. These corpora need to cover various scenarios of single causality, multi-causality, and complex causal networks, providing comprehensive and high-quality data support for constructing a causal relationship extraction model.

[0032] In this invention, the construction process of the chain-of-thought prompt word is as follows: Step A1, perform sentence splitting processing and word segmentation analysis on the original corpus, identify the causal marker words in the sentences, and obtain a set of independent sentences with causal markings; Specifically, for sentence splitting processing: split the input text into independent sentences according to punctuation marks for analyzing causal relationships sentence by sentence.

[0033] Word segmentation analysis: Segment each sentence, identify key components such as verbs, nouns, prepositions, conjunctions, etc., and preliminarily judge possible causal structures.

[0034] Identify causal marker words: Detect causal marker words in the text, such as "due to", "result in", "therefore", etc., to indicate the existence of a causal relationship.

[0035] Step A2, based on the causal marker words, locate the causal chain nodes, and respectively extract the core elements of the cause part and the result part. The core elements include the actor, action, object, spatio-temporal characteristics. Specifically, determine the cause and the result: By analyzing the causal marker words, identify the "cause" and "result" parts in the causal chain.

[0036] Decompose the core elements: Cause; Actor: Extract "who" did what; Action: Specific behavior; Object: Relevant object; Time: Time of occurrence; Location: Location of occurrence; Effect: Similarly, extract "who", "did what", "object", "time", "location".

[0037] Step A3, by analyzing the association path of the causal chain nodes, determine whether the causal relationship is a direct causal relationship or an indirect causal relationship involving intermediate events. In the present invention, it is necessary to fully understand the multi-layer causal chains contained in the input text, check the multi-layer causal relationships between events, and understand the complex interaction effects.

[0038] Direct causal relationship: Identify that one event directly triggers another event. For example, event A directly causes event B.

[0039] Indirect causal relationship: Identify the causal relationship formed through intermediate events or conditions. For example, event A affects event B through one or more intermediate events (event C).

[0040] Step A4, classify the cause part and the result part into preset event categories respectively. The event categories include at least one of military operations, diplomatic activities, security incidents, political events, social events, technological developments, economic events, aerospace activities, equipment and armaments. Step A5, generate structured data conforming to a predetermined format according to the core elements, causal relationship type, and event category. The format includes a causal relationship type field, a cause element set field, and a result element set field. Output the extracted "cause" and "result" parts in a preset structured format. The format needs to include the following information: { "causality_type": "Causal relationship type", "cause": { "actor": "Actor", "class": "Event category", "action": "Subject action", "time": "Time information", "location": "Location information", "object": "Object" }, "effect": { "actor": "Actor", "class": "Event category", "action": "Subject action", "time": "Time information", "location": "Location information", "object": "Object."

[0041] Step A6, perform integrity verification on the structured data through a multi-round verification mechanism, and dynamically adjust the word segmentation rules and event classification model parameters based on text features to obtain thought chain prompt words.

[0042] For multiple sentences or paragraphs in the text, repeat the above steps to ensure that no important causal chains are missed.

[0043] Optimization and adjustment: According to the specific text characteristics, adjust the word segmentation rules and event categories to ensure that the output causal chains are complete and accurate.

[0044] The following is the output template: {"causality_list": [{"causality_type": "", "cause": {"actor": "", "class": "", "action": "", "time": "", "location": "", "object": ""}, "effect": {"actor": "", "class": "", "action": "", "time": "", "location": "", "object": ""}}]}.

[0045] Specifically, add the above-mentioned thought chain prompt words with clear logic to the original corpus to guide the model to perform step-by-step reasoning, so as to more accurately understand and extract causal relationships. At the same time, the corpus should cover diverse causal relationship patterns, including direct causal relationships, indirect causal relationships, and complex multi-causal relationship networks, to ensure the generalization ability of the model. To further improve the training quality, manually annotate or semi-automatically annotate the causal relationships in the corpus to generate a structured causal relationship dataset. The annotation content includes causal event pairs (such as "Event A causes Event B") and the type and weight of the causal relationship (such as direct, indirect), to provide detailed training basis.

[0046] Use the fine-tuned corpus as input data and input it into the causal relationship extraction model for training. During the training process, use the low-rank adaptation technique to fine-tune the causal relationship extraction model; Input the corpus whose causal relationship is to be extracted into the causal relationship extraction model, and output the causal relationship extraction result; Among them, the causal relationship extraction model is preferably Qianwen Qwen2.5-14B in this embodiment.

[0047] During the training process, use the low-rank adaptation technique to fine-tune the causal relationship extraction model. The method is as follows: Step B1, divide the fine-tuned corpus into a training set, a validation set, and a test set; Step B2, load the pre-trained large language model as the base model of the causal relationship extraction model, and inject a low-rank adaptation matrix into the neural network layer of the base model to form a fine-tunable parameter subspace; Step B3, perform multiple rounds of iterative training on the base model injected with the low-rank adaptation matrix through the training set, and combine the validation set for hyperparameter optimization to obtain a task-based model suitable for multi-causal relationship extraction; Step B4, use the test set to evaluate the model performance, and dynamically adjust the rank parameter of the low-rank matrix and the prompt word embedding strategy according to the evaluation result.

[0048] Specifically, in the construction of the causality extraction model, the LoRA (Low-Rank Adaptation) technique is used to fine-tune the base large model (Qwen2.5-14B), with a focus on optimizing the inference module in the Qwen model. LoRA significantly reduces the training cost by introducing low-rank matrices, while enhancing the model's adaptability to the causality extraction task. Its advantage lies in achieving efficient optimization under resource-constrained conditions without modifying the original model weights. During this process, the collected corpus is first divided into a training set, a validation set, and a test set to construct a high-quality fine-tuning corpus specifically for optimizing the causality extraction task. We select the Qwen2.5 general pre-trained model as the base model, input the annotated data in the fine-tuning corpus into the model, conduct multiple rounds of training and validation, and continuously adjust the model parameters to ensure its ability to efficiently identify and extract causal relationships.

[0049] During the training process, the model's input not only includes the original corpus data but also incorporates chain-of-thought prompting words as auxiliary information to enhance the model's reasoning ability. Specifically, the chain-of-thought prompting words are embedded in the "system prompt word" part of each input sample, and then the input-output pairs before and after are extracted, corresponding to the "user input" and "model output" respectively. This approach helps the model better capture the logic of causal relationships while understanding the original corpus.

[0050] After fine-tuning, a dedicated large model adapted to the causality extraction task of domain events is obtained. This model has the following characteristics: (1) It can accurately extract causal relationships in the text, including single-cause, multi-cause, and complex network causal relationships.

[0051] (2) It outputs structured data (such as causal chains, causal graphs, etc.), providing standardized input for subsequent intelligent evaluation modules.

[0052] Application example: Input event text: "Country A launches a satellite, resulting in an increase in space debris, which further affects the competition for orbital resources." Model output causal relationship: { "causality_list": { "causality_type": "direct causal relationship", "cause": { "actor": "Country A", "class": "aerospace activities", "action": "launch", "time": "", "location": "", "object": "satellite" }, "effect": { "actor": "", "class": "aerospace activity", "action": "increase", "time": "", "location": "", "object": "space debris"}} { "causality_type": "indirect causality", "cause": { "actor": "", "class": "aerospace activity", "action": "increase", "time": "", "location": "", "object": "space debris" }, "effect": { "actor": "", "class": "aerospace activity", "action": "affect", "time": "", "location": "", "object": "orbital resource competition"}}]}

[0053] Through the above process, this method constructs a domain-adapted causal relation extraction large model, realizing the full-process optimization from corpus preparation to model fine-tuning. This model can efficiently extract the causal relations in domain events, providing a solid data foundation for intelligent evaluation and auxiliary decision-making.

[0054] Based on the causal relation extraction results, combined with the pre-constructed domain event evaluation index system and the construction strategy knowledge base, comprehensive evaluation results and decision-making suggestions are generated.

[0055] The comprehensive evaluation results, and the method for obtaining them is: Construct a domain event evaluation index system and perform knowledge vectorization processing; among them, the index system at least includes threat level, impact scope, and risk factors, and the index system is converted into a vector form through knowledge embedding technology and stored in the evaluation knowledge base; Based on domain-specific expertise (taking space security as an example), first design a comprehensive evaluation index system, including key indicators such as threat level, impact scope, and risk factors. Subsequently, perform knowledge vectorization processing on the designed index system, convert it into a vectorized format that can be understood and called by the large model, and store it in the evaluation knowledge base. Through knowledge vectorization, the model can quickly retrieve high-quality knowledge support, thereby ensuring the comprehensiveness and accuracy of the evaluation process.

[0056] In the process of constructing a domain event evaluation index system and performing knowledge vectorization processing, encode the evaluation index system into a vector state representation, use hierarchical similarity calculation to achieve ultra-high-speed knowledge retrieval, and develop a hybrid retrieval architecture to superimpose vector fast retrieval associations on traditional knowledge graphs.

[0057] Among them, encoding the evaluation index system into a vector state representation specifically includes: Step C1, perform eigenmatrix decomposition on each evaluation index, and extract the principal eigenvector and its corresponding eigenvalue; Step C2, calculate the approximate rank of the eigenmatrix through the matrix rank analysis module, and dynamically allocate the number of storage shards according to the formula Q = ceil(log 2 (r)) + k, where r is the approximate rank, k is the redundancy coefficient based on storage performance; in this embodiment, 0.5 ≤ k ≤ 2.

[0058] Step C3, use the segmented coding technology to map the principal eigenvector to a multi-dimensional vector space, and modulate the vector timeliness through timestamp weighting; Step C4, generate associated metadata tags to form an extensible vector representation unit.

[0059] The similarity calculation method specifically includes: Generate a family of hash functions based on locality-sensitive hashing (LSH), and dynamically set the number of hash buckets M according to the scale of the knowledge base , and map the high-dimensional vector to a low-dimensional hash bucket; Construct a KD-Tree structure in each hash bucket, divide the hyperplane according to the feature variance weight, and the weight w j = feature variance j / ∑ feature variance; Calculate the hash bucket to which the query vector belongs, and extract the candidate set C according to the candidate ratio γ, whereγ is negatively correlated with the number of hash buckets N ; The candidate set is screened in two stages: The first stage: Calculate the cosine similarity S cos , and retain the vectors that satisfy S cos ≥ 0.7θ(t); The second stage: Calculate the weighted Euclidean distance for the retained vectors , and convert it into the final similarity S final = 1 - D w / max( D w ).

[0060] Among them, is the value of the j -th dimension of the query vector. It represents the numerical value of the vector to be retrieved input by the user on a certain feature dimension. For example, in the evaluation of military events, if the evaluation indicators include dimensions such as "threat level" and "time sensitivity", q j is the quantization value of the user query on these specific dimensions.

[0061] v ij is the value of the i -th dimension of the j -th vector in the knowledge base.

[0062] i is the index of the vector in the knowledge base (such as the feature vector of the i -th historical event).

[0063] j : is the index of the feature dimension (such as the j -th evaluation indicator).

[0064] For example, if the knowledge base stores the evaluation vectors of 1000 historical events, and each event has 50 feature dimensions, then v 35,7 represents the 7 -th feature value (such as "diplomatic influence factor") of the 35 -th event.

[0065] The dual - engine index structure includes: a relational database layer and a vector engine layer; Among them, is a database management system based on the relational model (table structure), such as MySQL, PostgreSQL, Oracle, etc. It stores data in the form of rows and columns and performs data operations through SQL (Structured Query Language).

[0066] Clear structured information such as the name, time, classification label, and hierarchical relationship of the causal chain of the event.

[0067] Quickly perform equality queries (such as WHERE event type ='military operation') and range queries (such as time BETWEEN 2020 AND 2023) through SQL.

[0068] Ensure data consistency (such as ACID properties), which is applicable to operations that require high reliability (such as updating event status).

[0069] The vector engine layer is a storage and retrieval system designed specifically for high-dimensional vector data, supporting approximate nearest neighbor search (ANN, Approximate Nearest Neighbor), such as vector extensions of Faiss, Milvus, Elasticsearch, etc.

[0070] The vector engine layer encodes unstructured data such as evaluation metrics and text semantics into vectors (such as a 512-dimensional floating-point number array). It can quickly find the set of vectors that are most similar to the query vector (such as through cosine similarity or Euclidean distance).

[0071] For example, in military events, retrieve historical cases similar to the current battlefield situation.

[0072] Design a dynamic routing strategy. When the query complexity Qc exceeds the threshold Qt start engine retrieval, otherwise use database retrieval. where Qc = λ 1 Causal chain depth + λ 2 Number of time windows; λ 1 、λ 2 is a preset domain adjustment coefficient.

[0073] Among them, during the idle period of the system, preload the high-frequency query mode into the memory cache area, and dynamically maintain data activity through shard heat analysis. The cache refresh interval Δt = τ / log 2 (Ns), τ is the data validity period, Ns is the number of active shards.

[0074] Such as Figure 3 As shown, construct a domain event evaluation index system. The method is as follows: Obtain public standard documents, authoritative materials, and industry research results of domain events, and extract qualitative and quantitative evaluation criteria as the evaluation index system for domain events.

[0075] According to the causal relationship extraction results, dynamically retrieve and match the evaluation criteria in the evaluation knowledge base through semantic similarity calculation to construct an event-specific evaluation system. The causal relationship in the event is the basis for intelligent evaluation, which comes from the structured causal relationship data output by the fine-tuned causal relationship extraction large model, such as causal chains. These data provide key basic information for intelligent evaluation, ensuring that the model can conduct in-depth analysis based on the causal relationship network and generate accurate evaluation results.

[0076] The evaluation knowledge base retrieval and matching are realized through the knowledge base retrieval tool. The system dynamically extracts relevant evaluation criteria from the knowledge base according to the input structured causal relationship data to construct an evaluation system most suitable for the current domain event. For example, when the causal relationship involves "increasing space debris", the system automatically calls relevant criteria such as debris cleaning and orbit adjustment to ensure the pertinence and accuracy of the evaluation.

[0077] Based on the complex reasoning large model, integrate the causal relationship extraction results and the event-specific evaluation system for multi-level reasoning to generate a comprehensive evaluation result including risk factors, influence scope, and priority.

[0078] The evaluation model is based on the complex reasoning large model (Deepseek-R1). It combines causal relationship data and the constructed evaluation index system for reasoning and comprehensive evaluation. The model analyzes the risk factors, influence scope, and priority of the event to generate a comprehensive evaluation result, and adopts a clear step-by-step reasoning chain to ensure the interpretability of the evaluation result. To construct this evaluation system, first, comprehensively summarize by collecting public standard documents, authoritative materials, and industry research results. These materials provide the necessary theoretical and data support for the construction of the evaluation criteria. The specific process includes: collecting and analyzing standardized documents, industry manuals, and academic research results related to the causal relationship extraction task to ensure the authority of the reference materials; refining qualitative evaluation criteria, such as the clarity of the causal relationship and the rationality of the reasoning path, and combining professional knowledge to ensure the comprehensiveness of the evaluation dimensions; establishing quantitative evaluation criteria through historical data and cases, including impact degree, probability distribution, and risk assessment, to ensure the objectivity and repeatability of the evaluation; organically combining qualitative and quantitative evaluation criteria to form a comprehensive evaluation index system to support the accuracy and reliability evaluation of the causal relationship and provide risk management and decision-making support. The reasoning process adopts a clear chain to ensure that the evaluation result has high objectivity, scientificity, and interpretability.

[0079] After reasoning, the evaluation result of the domain event can be obtained. The following takes a domain event as an example for illustration.

[0080] Event description: The increase in orbital debris directly threatens the safety of satellite operations, which in turn restricts military exercises due to the inability to obtain reliable data.

[0081] Causal relationship: { "causality_list": { "causality_type": "direct causal relationship", "cause": { "actor": "", "class": "aerospace activities", "action": "increase", "time": "", "location": "orbit", "object": "debris" }, "effect": { "actor": "", "class": "aerospace activities", "action": "threaten", "time": "", "location": "", "object": "satellite operation safety" } }, { "causality_type": "indirect causal relationship", "cause": { "actor": "", "class": "aerospace activities", "action": "threaten", "time": "", "location": "", "object": "satellite operation safety" }, "effect": { "actor": "", "class": "military operations", "action": "Restricted", "time": "", "location": "", "object": "Military exercise"}}]}。

[0082] Output evaluation result: Risk factors: Orbital debris (external environmental risk), satellite security (technical risk), and restriction of military exercise (strategic risk) Scope of impact: Orbital debris directly threatens satellite security and results in restricted military capabilities.

[0083] Priority: High.

[0084] Through the above steps, the intelligent evaluation system realizes the full-process automation from causal relationship input to evaluation result output. This system combines the causal relationship network with the domain knowledge base and generates comprehensive, accurate, and transparent evaluation results based on the reasoning ability of the large model, providing a solid foundation for subsequent decision-making assistance.

[0085] As Figure 4 shown, the method for obtaining the decision-making suggestions is as follows: Construct a policy knowledge base, including collecting a set of policy methods related to domain events. The set of policy methods includes solutions, coping strategies, and optimization plans; perform knowledge vectorization processing on the set of policy methods, convert it into vectorized data that can be called by the large model, and store it to form a structured and retrievable policy knowledge base; The core of constructing the policy knowledge base lies in sorting out and collecting a set of policy methods related to domain events, including solutions, coping strategies, and optimization plans, such as orbital debris cleaning technology, satellite orbit adjustment plans, and debris monitoring mechanisms. Through knowledge vectorization processing, these policies are converted into vectorized data that can be understood and called by the large model and stored in the policy knowledge base, so as to provide structured and retrievable high-quality information for decision-making assistance and ensure the scientificity and pertinence of decision-making suggestions.

[0086] Take the comprehensive evaluation result as the input, dynamically retrieve the policy knowledge base through the knowledge base retriever, and use semantic matching to extract policy information related to the current event; The retrieval and matching of the knowledge base uses the evaluation results generated by the intelligent evaluation module (such as risk level, impact scope, threat priority, etc.) as input, providing a clear event background and core evaluation indicators for the decision-making module to ensure that the decision-making suggestions are highly relevant to the evaluation results. The knowledge base retriever dynamically retrieves the policy knowledge base according to the input content and extracts the policy information most relevant to the event. For example, when the evaluation result shows that "the increase in orbital debris threatens satellite safety", the retriever will give priority to calling relevant policies for debris cleaning and orbit adjustment. Through semantic matching, the retriever extracts the best policies to solve the current problem from the knowledge base, and this policy information will be used as the key basis for generating decision-making suggestions.

[0087] Based on the large complex reasoning model (Deepseek-R1), integrate the comprehensive evaluation results and the extracted policy information to generate decision-making suggestions including specific action plans, priorities, and timeliness, and optimize the feasibility and applicability of the suggestions through multiple rounds of reasoning, and output them in the form of charts or text reports.

[0088] The specific process of generating decision-making suggestions is as follows: Automatically generate short-term action plans and long-term optimization plans based on the event background, risk factors, and policy information; Optimize the applicability of the suggestions through context adjustment to ensure a match with the risk level of the evaluation results.

[0089] The multiple rounds of reasoning process include iterative verification of policy feasibility, resource consumption, and execution timeliness.

[0090] The priority arrangement of the decision-making suggestions is comprehensively determined based on threat urgency, policy implementation cost, and risk mitigation effect.

[0091] More specifically, the generation of decision-making assistance is achieved through the DeepSeek-R1 base large model. Combining the evaluation results and the retrieved policy information, comprehensively analyze the event background, risk factors, and available policies to generate targeted decision-making suggestions. The model first integrates the input data and automatically generates specific action plans and priority arrangements, and optimizes the feasibility and applicability of the suggestions through multiple rounds of reasoning and context adjustment. For example, when the input evaluation result is "the increase in orbital debris → the safety of satellites is threatened → resulting in restrictions on military exercises; risk level: high", combined with the retrieved "debris cleaning technology" and "satellite orbit adjustment plan", the model outputs decision-making suggestions including short-term actions: initiate debris cleaning tasks, focusing on cleaning debris in the threatened area; long-term plan: optimize satellite orbit design, establish a real-time debris monitoring mechanism, and reduce future risks. The finally generated decision-making assistance suggestions include specific response plans, priorities, and timeliness, and are visually presented in the form of charts or text reports. For example, the suggested directions for "high-risk" events are to implement debris cleaning, adjust orbit design, and establish a monitoring mechanism to improve future response efficiency.

[0092] In summary, based on the intelligent evaluation-assisted decision-making generation process, by combining the evaluation results with the policy knowledge base and supported by the complex reasoning ability of the DeepSeek-R1 large model, the full process automation from data analysis to decision-making recommendations is achieved. This method not only provides scientific and highly targeted decision-making recommendations but also significantly improves the processing efficiency of domain events, providing strong support for the strategic planning of complex tasks.

[0093] In the above embodiments, although the various steps are described in the above sequential order, those skilled in the art can understand that for the purpose of achieving the effects of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reversed order, and these simple changes are all within the protection scope of the present invention.

[0094] A multi-causal relationship extraction system for intelligent evaluation according to the second embodiment of the present invention is based on a multi-causal relationship extraction method for intelligent evaluation. The system includes: A database construction module configured to construct a causal analysis prompt template for generating step-by-step reasoning as a chain-of-thought prompt word and generate a fine-tuning corpus in combination with the original corpus; wherein the chain-of-thought prompt word includes multi-stage guiding instructions for text preprocessing, element extraction, and relationship determination. A training module configured to input the fine-tuning corpus as input data into a large language model, train for causal relationships, and perform fine-tuning training on the large language model using low-rank adaptation technology during the training process to obtain a causal relationship extraction model. A causal relationship extraction module configured to input the corpus for which causal relationships are to be extracted into the causal relationship extraction model and output the causal relationship extraction result. An evaluation and recommendation module configured to generate a comprehensive evaluation result and decision-making recommendations based on the causal relationship extraction result, in combination with a pre-constructed domain event evaluation index system and a construction strategy knowledge base.

[0095] Those skilled in the art of the relevant technical field can clearly understand that for the convenience and conciseness of description, the specific working processes and related descriptions of the above-described system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0096] It should be noted that the multi-causal relationship extraction system for intelligent evaluation provided in the above embodiments is only illustrated by dividing the above functional modules. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be combined into one module, or further split into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing each module or step, and are not regarded as improper limitations of the present invention.

[0097] An electronic device according to a third embodiment of the present invention includes: At least one processor; and A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned multi-causal relationship extraction method for intelligent evaluation.

[0098] A computer-readable storage medium according to a fourth embodiment of the present invention stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned multi-causal relationship extraction method for intelligent evaluation.

[0099] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes and related descriptions of the above-described storage device and processing device can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0100] Those skilled in the art should be able to realize that the modules and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. The programs corresponding to the software modules and method steps can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the art. To clearly illustrate the interchangeability of electronic hardware and software, the components and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in the form of electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0101] The terms "first", "second", etc. are used to distinguish similar objects and are not used to describe or indicate a particular order or sequence.

[0102] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to those processes, methods, articles, or apparatus / devices.

[0103] So far, the technical solution of the present invention has been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

Claims

1. A multi-causal relationship extraction method for intelligent evaluation, characterized in that: The method includes: Constructing a causal analysis prompt template for generating step-by-step reasoning as a thought chain prompt word, and generating a fine-tuning corpus in combination with the original corpus; wherein the thought chain prompt word includes multi-stage guiding instructions for text preprocessing, element extraction and relationship determination; The fine-tuning corpus is input as input data into a large language model, and training is performed on causal relationships. During the training process, the large language model is fine-tuned using a low-rank adaptation technique to obtain a causal relationship extraction model; Input the corpus of causal relationships to be extracted into the causal relationship extraction model, and output the causal relationship extraction results; Based on the causal relationship extraction results, combined with the pre-built domain event evaluation indicator system and the construction strategy knowledge base, comprehensive evaluation results and decision-making recommendations are generated.

2. A method for extracting multiple causal relationships for intelligent evaluation according to claim 1, characterized in that: The construction process of the thought chain prompt words is as follows: Step A1, perform sentence segmentation and word segmentation analysis on the original corpus, identify causal marker words in the sentences, and obtain a set of independent sentences with causal markers; Step A2, locating the causal chain nodes based on the causal marker words, and extracting the core elements of the cause part and the result part respectively, wherein the core elements include the action subject, the behavior action, the object of action, and the spatiotemporal characteristics; Step A3, by analyzing the association paths of the causal chain nodes, determine whether the causal relationship is a direct causal relationship or an indirect causal relationship involving intermediate events; Step A4, classifying the cause part and the result part into preset event categories, respectively, wherein the event category includes at least one of military actions, diplomatic activities, security events, political events, social events, scientific and technological developments, economic events, aerospace activities, equipment and armaments; Step A5, generating structured data conforming to a predetermined format according to the core elements, causal relationship types and event categories, wherein the format includes a causal relationship type field, a cause element set field and a result element set field; Step A6, performing integrity verification on the structured data through a multi-round verification mechanism, and dynamically adjusting the word segmentation rules and event classification model parameters based on text features to obtain thought chain prompt words.

3. The method for extracting multiple causal relationships for intelligent evaluation according to claim 1, characterized in that: During the training process, the low-rank adaptation technology is used to fine-tune the causal relationship extraction model, and the method is as follows: Step B1, dividing the fine-tuning corpus into a training set, a validation set and a test set; Step B2, loading a pre-trained large language model as a base model of the causal relationship extraction model, injecting a low-rank adaptation matrix into the neural network layer of the base model to form a fine-tunable parameter subspace; Step B3, performing multiple rounds of iterative training on the base model injected with the low-rank adaptation matrix through the training set, and performing hyperparameter optimization in combination with the validation set to obtain a task-based model adapted for multi-causal relationship extraction; Step B4, using the test set to evaluate the model performance, and dynamically adjusting the rank parameter of the low-rank matrix and the cue word embedding strategy according to the evaluation results.

4. The method for extracting multiple causal relationships for intelligent evaluation according to claim 1, characterized in that: The comprehensive evaluation results are obtained by: Construct a domain event assessment indicator system and perform knowledge vectorization processing; the indicator system at least includes threat level, impact scope and risk factor, and the indicator system is converted into vector form and stored in the assessment knowledge base through knowledge embedding technology; According to the causal relationship extraction results, the matching evaluation indicators are dynamically retrieved in the evaluation knowledge base through semantic similarity calculation to build an event-specific evaluation system; Based on a large complex reasoning model, the causal relationship extraction results are integrated with the event-specific evaluation system to perform multi-level reasoning and generate a comprehensive evaluation result that includes risk factors, impact scope and priority.

5. A method for extracting multiple causal relationships for intelligent evaluation according to claim 4, characterized in that: The method for constructing a domain event evaluation indicator system is as follows: Obtain public standard documents, authoritative information and industry research results of domain events, extract qualitative and quantitative evaluation standards as the domain event evaluation indicator system.

6. A method for extracting multiple causal relationships for intelligent evaluation according to claim 4, characterized in that: The decision suggestion is obtained by: Constructing a strategy knowledge base, including collecting strategy method sets related to domain events, including solutions, response strategies and optimization plans; performing knowledge vectorization processing on the strategy method sets, converting them into vectorized data that can be called by a large model and storing them, to form a structured and searchable strategy knowledge base; Taking the comprehensive evaluation result as input, dynamically searching the policy knowledge base through a knowledge base retriever, and extracting policy information associated with the current event by using semantic matching; Based on a complex reasoning model, the comprehensive evaluation results and the extracted strategic information are integrated to generate decision recommendations containing specific action plans, priorities and timeliness. The feasibility and applicability of the recommendations are optimized through multiple rounds of reasoning and output in the form of charts or text reports.

7. A method for extracting multiple causal relationships for intelligent evaluation according to claim 6, characterized in that: The specific process of generating decision recommendations is as follows: Automatically generate short-term action plans and long-term optimization plans based on event background, risk factors and strategic information; Adjust the applicability of optimization recommendations through context to ensure they match the risk level of the assessment results.

8. The method for extracting multiple causal relationships for intelligent evaluation according to claim 6, characterized in that: The multi-round reasoning process includes iterative verification of strategy feasibility, resource consumption and execution timeliness.

9. The method for extracting multiple causal relationships for intelligent evaluation according to claim 6, characterized in that: The priority arrangement of the decision recommendations is based on a comprehensive assessment of threat urgency, strategy implementation costs, and risk mitigation effectiveness.

10. A multi-causal relationship extraction system for intelligent evaluation, based on a multi-causal relationship extraction method for intelligent evaluation according to any one of claims 1 to 9, characterized in that: The system includes: A database construction module is configured to construct a causal analysis prompt template for generating step-by-step reasoning as a thought chain prompt word, and generate a fine-tuning corpus in combination with the original corpus; wherein the thought chain prompt word includes multi-stage guiding instructions for text preprocessing, element extraction and relationship determination; A training module, configured to input the fine-tuning corpus as input data into a large language model, perform training on causal relationships, and use a low-rank adaptation technique to fine-tune the large language model during the training process to obtain a causal relationship extraction model; A causal relationship extraction module, which is configured to input the corpus of the causal relationship to be extracted into the causal relationship extraction model and output the causal relationship extraction result; The evaluation and suggestion module is configured to generate comprehensive evaluation results and decision suggestions based on the causal relationship extraction results, combined with a pre-built domain event evaluation indicator system and a construction strategy knowledge base.

Images