Multi-source threat intelligence conflict resolution and contradiction tracing method, system and equipment

Abstract

The invention discloses a multi-source threat intelligence conflict resolution and contradiction tracing method, system and device. The method comprises the steps of obtaining multi-source threat intelligence and extracting multi-dimensional features of threat indexes, and generating initial AI confidence and decision attribution information based on a pre-training model; presenting the information through a preset research and judgment interface, receiving a correction operation of a user on the confidence coefficient and a structured correction reason, and generating a training sample; continuously optimizing the model through incremental learning based on a training sample; and processing new information by using the optimized model, and generating a final confidence coefficient and a traceability report fusing AI attribution and manual correction history. According to the method, a hybrid enhanced intelligent closed loop of AI preliminary evaluation, artificial study and judgment feedback and model continuous optimization is constructed, so that the problems of blind context, unmatched service and model drift of a pure AI model in threat intelligence evaluation are solved, and the accuracy, credibility and automation level of threat intelligence evaluation are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security and artificial intelligence, in particular to a multi-source threat intelligence conflict resolution and contradiction tracing method, system and device. BACKGROUND

[0002] In the current network security operation, the fusion of multi-source threat intelligence has become a standard practice for discovering advanced threats. However, conflicts between heterogeneous intelligence sources seriously restrict the reliability of automated response (SOAR). Existing research attempts to assess the credibility of intelligence through machine learning models, but these purely algorithmic models have the following inherent defects: Lack of domain knowledge: algorithmic models cannot understand internal, non-public business logic. For example, an IP marked as malicious by a public intelligence source may be a third-party service provider authorized by the enterprise, which is actually benign in the business context.

[0003] Poor scene adaptability: a general AI model is difficult to adapt to the changing network architecture, business focus and risk tolerance of different enterprises.

[0004] "Black box" decision-making is difficult to trust: the confidence given by purely deep learning models lacks explainability, and security analysts cannot understand the basis of their decision-making, so they are unwilling to trust and adopt their results, leading to the abandonment of the system.

[0005] Lack of effective human feedback loop: existing systems, even if they allow human correction, are only used as one-time override operations, and the corrected knowledge cannot be absorbed by the system for optimizing future automatic decision-making, leading to the repetition of the same errors.

[0006] Expert knowledge cannot be applied on a large scale: the decision-making of security experts is valuable knowledge assets, but in the traditional way, this knowledge cannot be deposited into automated systems, and cannot realize the accumulation and reuse of knowledge.

[0007] Model drift cannot be corrected: as the threat situation and intelligence source quality change, the performance of AI models will appear "drift". Without continuous human feedback injection, the system cannot correct this performance degradation.

[0008] Therefore, it is a technical problem that needs to be solved by those skilled in the art to provide a multi-source threat intelligence conflict resolution and contradiction tracing method, system and device for solving the above problems. SUMMARY

[0009] To solve the above technical problems, the purpose of the present application is a multi-source threat intelligence conflict resolution and contradiction tracing method, which has clear logic and simple operation. Through the mixed enhanced intelligent closed loop of "AI evaluation-artificial feedback-model optimization", expert knowledge is systematically integrated into the automatic process, improving the accuracy, reliability and automation level of threat intelligence evaluation.

[0010] The technical solutions provided by the present application are as follows: A multi-source threat intelligence conflict resolution and contradiction tracing method, comprising the following steps: Obtain first multi-source threat intelligence, extract multi-dimensional features of threat indicators, and generate initial AI confidence and decision attribution information of the threat indicators based on a pre-trained baseline confidence generation model; Present the initial AI confidence and the decision attribution information on a preset research interface, receive user correction operations and correction reasons for the initial AI confidence, and generate training samples containing feature vectors, corrected confidence and the correction reasons; Optimize the baseline confidence generation model based on the training samples to obtain an optimized confidence generation model; Process second multi-source threat intelligence based on the optimized confidence generation model to generate final AI confidence and a tracing report that integrates the decision attribution information and artificial correction history.

[0011] Preferably, the optimization of the baseline confidence generation model based on the training samples to obtain an optimized confidence generation model comprises the following steps: Store the training samples in a preset sample pool; Periodically or when the number of samples in the preset sample pool reaches a threshold, extract a batch of training samples from the preset sample pool for incremental learning or fine-tuning of the baseline confidence generation model to obtain an optimized confidence generation model.

[0012] Preferably, after obtaining the optimized confidence generation model, the method further comprises the following steps: Performance evaluation and comparison of the optimized confidence generation model and the confidence generation model currently used online; Based on the comparison result, it is determined whether to update the optimized confidence generation model as a new online version.

[0013] Preferably, the multi-dimensional features include but are not limited to at least one of intelligence source dynamic reputation, intelligence timeliness, intelligence content richness, external coordination or conflict network features of threat indicators, geographical features, behavior features, correlation features and time features.

[0014] Preferably, the presenting on the preset judgment interface comprises the following steps: prioritizing the threat indicators requiring human judgment based on at least one of the following factors: uncertainty of the initial AI confidence, conflict severity of multi-source intelligence, or criticality of the asset associated with the threat indicator; generating and presenting an intelligent judgment work list according to the prioritization result, the intelligent judgment work list being used to guide the user to prioritize processing of the threat indicators with high priority.

[0015] Preferably, the correction reason comprises a label selected from a predefined label set and / or a user inputted text description. The predefined label set comprises labels for characterizing false positives, business relevance, and attribution.

[0016] Preferably, the optimizing the baseline confidence generation model based on the training sample further comprises the following steps: constructing or updating an expert rule base based on one or more of the correction reasons; introducing rules in the expert rule base as regularization constraints into an incremental learning or fine-tuning training process of the baseline confidence generation model.

[0017] Preferably, the provenance report presents a logical relationship between the initial AI confidence of the threat indicator, the user-performed correction operation, the correction reason, and the final confidence in a visual manner.

[0018] A multi-source threat intelligence conflict resolution and contradiction provenance system comprises: An AI confidence module is configured to obtain first multi-source threat intelligence, extract multi-dimensional features of a threat indicator, and generate an initial AI confidence of the threat indicator and decision attribution information based on a pre-trained baseline confidence generation model. A training sample generation module is configured to present the initial AI confidence and the decision attribution information on a preset judgment interface, receive a user's correction operation on the initial AI confidence and a correction reason, and generate a training sample comprising a feature vector, a corrected confidence, and the correction reason. A model optimization module is configured to optimize the baseline confidence generation model based on the training sample to obtain an optimized confidence generation model. An intelligence processing and provenance module is configured to process second multi-source threat intelligence based on the optimized confidence generation model, generate a final AI confidence, and generate a provenance report that fuses the decision attribution information and a human correction history.

[0019] An electronic device comprising at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of the preceding claims.

[0020] The application discloses a multi-source threat intelligence conflict resolution and contradiction tracing method, which comprises the following steps: acquiring multi-source threat intelligence, extracting multi-dimensional features of threat indicators, generating initial AI confidence and decision attribution based on a pre-trained model; presenting the above information on a research and judgment interface and receiving user correction operations and reasons for the confidence, generating training samples; optimizing the model based on the training samples; processing new intelligence using the optimized model to generate a final confidence and a tracing report that combines AI attribution and artificial correction history.

[0021] The application integrates expert knowledge into the automatic process through the mixed enhanced intelligent closed loop of "AI evaluation-artificial feedback-model optimization", improves the accuracy, reliability and automation level of threat intelligence evaluation, and realizes the technical effects in multiple levels and cooperation, which are embodied in the following aspects: The "context blindness" problem of the AI model is solved: by introducing artificial feedback, the business context knowledge is systematically injected into the AI model, so that the model output is more suitable for the actual business scenario.

[0022] A continuous optimization learning closed loop is constructed: the artificial correction results are converted into high-quality training samples for incremental learning of the model, so that the AI evaluation capability continuously approaches the expert level, and the system realizes self-evolution.

[0023] The transparency and credibility of the decision are improved: by providing decision attribution information and tracing reports that combine artificial correction history, the AI decision-making process is explainable and traceable, which enhances the trust of security analysts in the system.

[0024] The scale application of expert knowledge is realized: the research and judgment experience and rules of experts are deposited into the automatic system to form reusable knowledge assets, which improves the overall efficiency and maturity of security operation.

[0025] The model drift is effectively alleviated: the continuous feedback learning mechanism enables the model to adapt to changes in threat situation and intelligence source quality, maintaining high performance.

[0026] The application also provides a multi-source threat intelligence conflict resolution and contradiction tracing system and an electronic device, which have the same beneficial effects as the method because they belong to the same technical concept and solve the same technical problems, and will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only represent some embodiments described in the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0028] Figure 1 A flowchart of a multi-source threat intelligence conflict resolution and contradiction tracing method provided in an embodiment of the present application; Figure 2 A flowchart of step S3 provided in an embodiment of the present application; Figure 3 A flowchart after step S2 provided in an embodiment of the present application; Figure 4 A flowchart of training a baseline confidence generation model provided in an embodiment of the present application; Figure 5 A structural schematic diagram of a multi-source threat intelligence conflict resolution and contradiction tracing system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only represent some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0030] Embodiments of the present application are written in a progressive manner.

[0031] Embodiments of the present application provide a multi-source threat intelligence conflict resolution and contradiction tracing method, system and device. The technical problem of the prior art that the context blindness of the pure AI model and the business do not fit is mainly solved.

[0032] As shown in Figure 1 A multi-source threat intelligence conflict resolution and contradiction tracing method, comprising the following steps: S1. Obtain a first multi-source threat intelligence, extract multi-dimensional features of threat indicators, and based on a pre-trained baseline confidence generation model, generate initial AI confidence of the threat indicators and decision attribution information; S2. Present the initial AI confidence and the decision attribution information on a preset research interface, receive a user's correction operation on the initial AI confidence and a correction reason, and generate a training sample containing a feature vector, a corrected confidence and a correction reason; S3. Optimize the baseline confidence generation model based on the training samples to obtain the optimized confidence generation model; S4. Based on the optimized confidence generation model, process the second multi-source threat intelligence to generate the final AI confidence score and a source tracing report that integrates decision attribution information and manually corrected history.

[0033] The multi-source threat intelligence in step S1 refers to evidential information about cybersecurity threats collected, integrated, and analyzed from multiple and different types of external sources. Threat indicators, also known as intrusion indicators or attack indicators, refer to specific, observable pieces of evidence observed in network activity, systems, or files that indicate potential malicious intent or past security vulnerabilities. Initial AI confidence refers to a preliminary, quantitative assessment value given by a pre-trained baseline confidence generation model in the invention, before manual correction, of the probability that a certain threat indicator (such as IP, domain name, file hash, etc.) is malicious. Decision attribution information refers to auxiliary information generated by the AI ​​model to explain and illustrate why it gives a specific "initial AI confidence." It reveals the basis, key factors, and contributions of the model in making the decision. The preset judgment interface in step S2 is a specially designed and developed software operation interface. Its core purpose is to efficiently guide security analysts (users) to review and correct the initial judgment (initial AI confidence level) of AI, and to collect their feedback information in a structured manner. The manual correction history in step S4 refers to the complete set of records of human experts intervening, reviewing, and modifying the initial results generated by automated systems (such as AI models) within a specific system or process. Specifically, it refers to the collection of all operation records, reasons, and contextual information of security analysts correcting the initial AI confidence levels of all threat indicators on the "preset assessment interface." The source tracing report is a comprehensive document detailing the final judgment result of a specific threat indicator (such as IP address or domain name) and its formation process. It does not merely provide a "yes" or "no" conclusion, but clearly traces and presents the complete logical chain and decision-making basis for arriving at that conclusion.

[0034] Steps S1 to S4 detail the implementation of the multi-source threat intelligence conflict resolution and contradiction tracing method. By automatically generating initial AI confidence levels through a pre-trained model, rapid and preliminary screening of massive amounts of multi-source intelligence is achieved, avoiding the inefficiency of relying entirely on manual judgment. "Decision attribution information" is provided, transforming the AI's "black box" decision-making into understandable key factor analysis. This establishes analysts' initial trust in the AI ​​and provides a focus and entry point for subsequent manual judgment, reducing the cognitive load on analysts. Through manual correction, internal business logic, context, and expert experience, which are imperceptible to the machine, are directly injected into the system. For example, correcting an IP labeled as malicious by public intelligence to an "internal business IP" directly corrects the AI's misjudgment due to a lack of domain knowledge. The correction operations are structured and recorded as "training samples" (feature vector, corrected confidence level, and reason for correction), which is equivalent to continuously producing accurate, business-labeled "nutrients" for the AI ​​model. These data are far more valuable than raw data obtained from the public internet because they closely align with the organization's actual environment. Incremental learning of the model using human feedback data allows the AI's assessment capabilities to continuously approach and learn from the decision-making level of security experts, forming a virtuous cycle where assessment accuracy continuously improves with the accumulation of feedback data. As the cyber threat landscape changes, model performance can degrade. This method, through continuous human feedback input, enables the model to adapt to new threat patterns and changes in intelligence source quality, maintaining the advanced nature and reliability of its assessment capabilities. Incorporating expert judgment (reflected in the reasons for corrections and correction values) into the AI ​​model allows the wisdom of one expert to be replicated and extended throughout the automated system, solving the problem of scarce expert resources. Using a "calibrated" and optimized model to process new intelligence results in a "final AI confidence score" that better reflects actual business scenarios, significantly reducing false positives and false negatives. The generated "attribution report" integrates AI attribution and human correction history, making the judgment results of each threat indicator traceable, explainable, and auditable. This greatly enhances the security operations team's trust in the automated system and meets compliance requirements.

[0035] The above solution combines the advantages of humans and machines through a series of closely linked steps, ultimately achieving a qualitative leap at the system level from "static and untrustworthy automation" to "dynamic, trustworthy, and continuously evolving intelligent enhancement," with significant and complete technical effects.

[0036] In one embodiment, threat indicators include, but are not limited to, at least one of: IP address, domain name, and file hash. The pre-trained baseline confidence generation model architecture is as follows: It uses XGBoost for structured features, GraphSAGE for relational features, and BERT-Based Transformer for textual features such as domain names as the base models, constructing a meta-learning layer, StackingClassifier. XGBoost is an advanced machine learning algorithm, short for eXtreme Gradient Boosting. Simply put, it's a decision tree-based model built using ensemble learning. GraphSAGE is a pioneering inductive graph neural network algorithm used to generate low-dimensional vector representations of nodes in a graph. Its core idea is to learn the embedding representation of a node by sampling and aggregating features from its local neighbors. Transformer (the underlying architecture) and BERT (a specific model based on this architecture) are also included. Simply put, BERT is a pre-trained model built entirely on a Transformer encoder stack. StackingClassifier, an ensemble learning technique, uses the predictions of multiple different base classifiers as new features to train a "meta-classifier" for final prediction. Simply put, it doesn't involve a simple vote or average of the predictions from the base models, but rather it allows another model to learn how to best combine the predictions from these base models.

[0037] like Figure 2 As shown, preferably, the baseline confidence generation model is optimized based on training samples to obtain the optimized confidence generation model, including the following steps: A1. Store the training samples into the preset sample pool; A2. Periodically or when the number of samples in the preset sample pool reaches a threshold, a batch of training samples is extracted from the preset sample pool to perform incremental learning or fine-tuning training on the baseline confidence generation model, so as to obtain the optimized confidence generation model.

[0038] Incremental learning in step A2 refers to the ability of a machine learning model, after initial training, to continuously learn new knowledge and adjust its parameters from newly arriving data without forgetting existing knowledge. Steps A1 to A2 are the specific implementation details of step S3. By introducing a "preset sample pool" and a "batch processing triggering mechanism," the scattered and random manual feedback is transformed into a structured and industrialized model optimization process. This resolves the potential risks of directly using real-time feedback for model updates, ensuring the stability and reliability of system evolution.

[0039] Specifically, based on step A1, scattered, single-time correction operations (training samples) are first uniformly stored in a pre-defined "sample pool" instead of being immediately used to update the model. This acts as a knowledge base or data buffer, avoiding the instability and fluctuations that may result from directly updating the model online using a single sample. Individual samples may contain noise or randomness, and immediate learning could lead to model bias. The sample pool aggregates correction records from different analysts, targeting different types of threats, and at different time points. When batch samples are drawn from the pool, a batch of samples is more statistically representative in terms of features and labels, reducing variance during model training and helping to learn more general patterns. Compared to learning one sample at a time, batch learning is more stable and has better convergence.

[0040] Based on step A2, flexible update strategies are provided through two trigger conditions: "periodic" or "sample quantity reaching a threshold." Periodic updates (e.g., daily / weekly) are suitable for scenarios with stable traffic and less stringent real-time requirements, facilitating resource planning and system maintenance. Threshold-triggered updates are suitable for scenarios with variable traffic and a desire for rapid knowledge absorption. Once sufficient new knowledge is accumulated (reaching the threshold), learning is immediately triggered, ensuring timely model updates. This design allows the system to adapt to the operational rhythms and computational resource constraints of different enterprises. Step A2 explicitly mentions "incremental learning or fine-tuning training," covering two mainstream continuous learning techniques: Incremental learning / online learning: This typically refers to the model rapidly updating its parameters in a lightweight manner after receiving new data batches, not entirely dependent on the initial training data. It has low computational overhead and is suitable for frequent updates. Fine-tuning training: This typically refers to using data from the sample pool (or combining some initial data) as the training set to perform a new, complete training round on the model. This may have higher computational overhead, but the results may be more thorough. This statement expands the scope of patent protection. Regardless of the specific optimization algorithm used, as long as the "pooling-batch processing" process is followed, it falls within the protection scope of this invention.

[0041] These two steps translate an idealized concept of "continuous learning" into a robust and controllable engineering implementation plan: from "real-time streaming" to "batch processing": transforming continuous data streams into discrete batch tasks, facilitating management, monitoring, and fault recovery; from "unstable" to "stable": accumulating batch samples smooths out noise that may be introduced by a single sample, making the model update direction more stable; from "uncontrollable" to "schedulable": update triggering conditions become explicit, allowing system administrators to clearly know when the model will be updated, facilitating performance evaluation and resource allocation.

[0042] The above scheme ensures that the "benchmark confidence generation model" can evolve gradually in a safe, efficient and reliable manner, thereby robustly achieving the core technical effect pursued by the patent: "making the confidence assessment results continuously approach the decision-making level of security experts".

[0043] like Figure 3 As shown, preferably, after obtaining the optimized confidence generation model, the following steps are also included: B1. Evaluate and compare the performance of the optimized confidence generation model with the current online confidence generation model; B2. Based on the comparison results, decide whether to update the optimized confidence generation model to the new online version.

[0044] Steps B1 to B2 are the specific implementation details following step S2, together forming a robust model deployment and version management process. They ensure that only validated, higher-performing models can be used in the actual production environment, thus transforming the high-risk operation of "model optimization" from an "adventure" into a "controlled experiment," greatly improving the reliability, stability, and trustworthiness of the entire system.

[0045] Specifically, based on step A1, comparing the performance of the new and old models on a unified test set (typically including historical data and recently manually corrected samples) yields quantitative metrics such as accuracy, recall, F1 score, and AUC. This avoids the bias of judging a model's quality based on intuition or a single case. It eliminates the subjectivity and uncertainty in the model optimization process. Without this step, it's impossible to know whether the so-called "optimization" is a genuine improvement or merely overfitting noise. Performance evaluation not only examines overall accuracy but also allows for in-depth analysis of model performance differences across different types of threats and asset criticality, helping to uncover potential flaws or regressions in the new model.

[0046] Based on step B2 and the objective comparison results from step B1, if the new model's performance does not meet the preset standards (e.g., insignificant accuracy improvement or severe regression), the system decides not to update the online version. This avoids security risks (e.g., missed real threats) and operational chaos (e.g., a surge in false positives) caused by model degradation. It directly addresses the potential for "catastrophic forgetting" or performance degradation due to improper optimization in incremental learning, ensuring the SLA of online services. Even if the new model's performance meets the standards, step B2 represents a conscious and controllable deployment action. This allows the operations team to release the model within a controllable time window and can combine strategies such as blue-green deployment or canary deployment to allow a small portion of traffic to use the new model first, further observing its real-world online performance to ensure everything goes smoothly and minimize the risk of model updates. This achieves a smooth upgrade with little or no business awareness.

[0047] The above solutions achieve the following: a shift from "blind updates" to "scientific decision-making": system evolution is no longer a black-box process, but a scientific decision-making process based on objective data and clear standards. A "safety first" evolutionary perspective is established: "No degradation" is recognized as a more important principle than "rapid evolution." While pursuing performance improvements, the stability and reliability of existing services are paramount.

[0048] Enhancing operational confidence and automation: It is precisely because of this rigorous quality gate that developers and operators dare to allow the system to conduct more frequent automated model training and evaluation, thereby truly achieving safe, efficient, and continuous learning and delivery.

[0049] A complete technological closed loop is formed: B1 and B2, together with the sample pool and incremental learning of A1 / A2, form a perfect closed loop: collecting feedback -> batch training -> evaluation and verification -> secure deployment. This closed loop is the lifeline for the system's continuous and healthy evolution.

[0050] Preferably, the multidimensional features include, but are not limited to, at least one of the following: dynamic reputation of intelligence sources, intelligence timeliness, richness of intelligence content, external coordination or conflict network characteristics of threat indicators, geographical features, behavioral features, correlation features, and time features.

[0051] Among them, the dynamic reputation of the intelligence source refers to the real-time or near-real-time, continuously updated quantitative assessment of the reliability and accuracy of a threat intelligence provider (i.e., the "intelligence source"); intelligence timeliness refers to the length of time from the generation and release of a threat intelligence to its acquisition and processing by the system, as well as the effective time range of the threat described by the intelligence. It measures the "freshness" and "effective period" of the intelligence; the external coordination or conflict network characteristics of threat indicators refer to a type of characteristic that infers the potential threat level by analyzing the connection patterns, positions, and structural relationships of the indicator in a large network composed of many interconnected indicators, rather than viewing a single threat indicator (such as an IP) in isolation. In practical applications, these feature dimensions do not work in isolation; they generate a powerful synergistic effect: Cross-validation: For example, if an IP that is marked as malicious (intelligence source reputation) has a high degree of confidence if its geographical features show that it comes from a high-incidence area of ​​attacks and its network features show that it is closely connected to known malicious nodes, then its malicious probability is confirmed by multiple factors.

[0052] Resolving contradictions: For example, an IP address might be flagged as malicious by two sources, but labeled as benign by another high-reputation source. In this case, analyzing its correlation characteristics might reveal that it is a corporate VPN server (business-related), and combined with network characteristics (not associated with malicious nodes), the system could tend to consider this a false alarm.

[0053] Rich attribution: Multidimensional features provide rich material for "decision attribution information", enabling the model to clearly point out while giving confidence: "The main reason for judging that the IP is malicious is that it is associated with the resolution of 5 known malicious domain names (network features), and the activity time is between 2-5 am (time features), which matches the known attack pattern (behavioral features)." This optimized set of multidimensional features forms the input foundation for a deep, three-dimensional, and adaptive threat assessment model. It greatly improves the basic quality of the "initial AI confidence level," laying a solid data foundation for subsequent human-machine collaboration and model optimization. It is one of the core prerequisites for the entire patented solution to achieve the expected technical effect.

[0054] Preferably, the analysis is presented on a preset analysis interface, including the following steps: Based on at least one of the following factors—the uncertainty of the initial AI confidence level, the severity of the conflict of multi-source intelligence, or the criticality of the assets associated with the threat indicator—threat indicators that require manual assessment are prioritized. Based on the priority ranking results, an intelligent assessment task list is generated and presented. This intelligent assessment task list is used to guide users to prioritize and address high-priority threat indicators.

[0055] The above steps are the specific implementation details of obtaining the intelligent assessment task breakdown and presenting this content on the assessment interface in step S2. Through an intelligent priority sorting algorithm, the originally potentially chaotic queue of manual assessment tasks is transformed into a structured and guided intelligent task list. Its core effect is to optimize the allocation of limited human resources in the face of unlimited security threats, ensuring that the experts' attention is always guided to the most valuable and urgent tasks.

[0056] Among these approaches, prioritizing cases based on the "uncertainty of initial AI confidence" focuses on improving the AI ​​model itself. Experts are given priority to handle cases with high model uncertainty, as the feedback from these cases provides the greatest value for model learning and optimization, most efficiently filling the AI's cognitive blind spots. This avoids experts wasting time on simple cases where the AI ​​is already quite confident (regardless of right or wrong), ensuring that each manual correction becomes a high-quality "teaching" experience, directly accelerating the system's evolution. Based on the severity of conflicts among multi-source intelligence, the focus is on resolving core contradictions in intelligence fusion. Indicators where different sources provide diametrically opposed judgments represent a "deadlock" in automated decision-making and are the highest-risk points most prone to misjudgment and underreporting. Forcibly and prioritizing the resolution of these conflicts can most quickly and directly reduce the overall decision-making risk of the system, preventing response paralysis or erroneous actions caused by internal intelligence contradictions. Based on the priority of assets associated with threat indicators, the focus is on business impact and risk mitigation. Adhering to the first principle of "protecting the most important assets," the protection of core business operations is ensured to remain undiminished even when resources are deemed scarce. Risk-driven resource allocation is implemented to ensure that a threat to a regular office computer is not prioritized over a threat to a core database server, thereby maximizing the return on security investments. The "Intelligent Analysis Task List" provides a clear, prioritized action plan that greatly reduces the cognitive load and decision fatigue of analysts, allowing them to immediately enter a state of in-depth analysis without having to spend time sifting through a massive number of alerts. It also avoids uneven task distribution, where some analysts are overloaded while others are idle, thereby improving the overall operational efficiency of the entire team.

[0057] By using the aforementioned solution as the "scheduling hub" for human-computer interaction, and through intelligent sorting and list presentation, it ensures that the valuable cognitive resources of human experts are used for tasks that best enhance system intelligence, resolve core conflicts, and guarantee business security. It is not only a tool to improve the efficiency of individual assessments, but also a catalyst driving the efficient operation of the entire "hybrid augmented intelligence" closed loop, directly contributing to the patent's ultimate goal of "improving the maturity of security operations, the level of reliable automation, and operational efficiency."

[0058] Preferably, the reasons for the modification include: tags selected from a predefined set of tags and / or text descriptions entered by the user; The predefined set of tags includes tags used to characterize false alarms, business relevance, and attribution.

[0059] The above content is the specific implementation details of the correction reasons in step S2. By using a hybrid input mode of "structured tags + unstructured text", the feedback information is ensured to be machine readable and learnable, while taking into account flexibility and detailed supplementation. This transforms the scattered manual correction actions into a high-quality knowledge source for the sustainable evolution of the system.

[0060] Specifically, it achieves knowledge standardization and structuring by selecting tags from a predefined set of tags, greatly improves human-computer interaction efficiency, and directly generates machine-understandable features; it preserves detailed information and contextual details through user-input text descriptions, and provides a data source for future system optimization; The three labels in the predefined label set were not chosen randomly, but precisely targeted the three most critical blind spots of the AI ​​model: "False alarms": These directly correct the accuracy of the model's judgments and are the most direct data for optimizing the model's core performance.

[0061] "Business-related": Directly injecting external business context knowledge is the key to solving the "context-blind" problem.

[0062] "Attribution Relationship" (e.g., "True Positive, but attribution incorrect"): Corrects the model's attribution accuracy, helping the model understand the source and context of the attack, rather than just malice; The above approach ensures that the information flow from "human judgment" to "model optimization" is high-quality, low-loss, and scalable. This means that every human correction not only solves the current problem but also becomes a drop of "nutrient solution" nourishing the AI ​​model, making it smarter. It is one of the core components driving the efficient operation of the entire hybrid intelligent closed loop.

[0063] like Figure 4 As shown, preferably, optimizing the baseline confidence generation model based on training samples further includes the following steps: C1. Based on one or more reasons for modification, construct or update the expert rule base; C2. Introduce the rules in the expert rule base as regularization constraints into the incremental learning or fine-tuning training process of the baseline confidence generation model.

[0064] The expert rule base in step C1 refers to a structured collection of knowledge in the system that can be invoked by the machine learning process. Each rule in the system represents a clear logic or experience that security experts follow when making threat assessments in a specific context. Regularization in step C2 is a technique used in machine learning to prevent overfitting. Its core idea is to add an extra penalty term to the model's loss function to constrain the model's complexity and make it tend to learn simpler and more general patterns.

[0065] Using an "expert rule base as a regularization constraint" means incorporating the knowledge of human experts, in a computable mathematical form, as a penalty term into the model's objective function. Its purpose is no longer to prevent "mathematical complexity," but rather to prevent patterns learned by the model from "violating known domain knowledge and logic." Steps C1 to C2 are the specific implementation details of training the baseline confidence generation model in step A2. By transforming the logical rules of human experts into mathematical constraints in the model training process, a deep integration of symbolic AI (rule-based) and connectionist AI (data-driven) is achieved. Its core effect is to proactively and directionally shape the model's evolutionary path, rather than passively learning from data, thereby ensuring that the model's learning outcomes are not only accurate but also consistent with business logic and domain common sense. Specifically, based on step C1, the system automatically extracts universally applicable rules from a large number of scattered "correction reasons." For example, when an analyst repeatedly marks a threat from a specific IP segment as "business-related" and corrects it to benign, the system can automatically generate a rule: "If the IP belongs to network segment A, then its malicious confidence should be significantly reduced." This elevates the specific experience of experts to reusable organizational knowledge assets, avoiding the inefficiency of models needing to learn similar cases countless times to master a simple rule. The expert rule base is not static but continuously updated as new correction reasons are added. New rules are added, and outdated rules are weakened or eliminated. This allows the knowledge base to adapt to changes in the business environment and threat landscape, becoming a "living" knowledge system. Based on step C2, traditional model optimization only adheres to the data. This approach adds a higher principle to "adherence to data": "adherence to human-defined business logic." By transforming rules into regularization terms and incorporating them into the loss function, the model, when searching for the optimal solution, not only requires the predicted results to closely approximate the true labels but also demands that its parameter updates do not violate expert rules. This proactively prevents the model from learning erroneous or illogical associations. For example, without constraints, the model might discover a one-sided association such as "all IPs active at midnight are malicious." However, expert rules (such as "business-related IPs can be benign at any time") act as constraints, preventing the model from heading in this erroneous direction. Some business rules may only be supported by a few amendments (small data volume). If driven solely by data, the model will find it difficult to learn and be convinced of this pattern. However, by using it as a strong constraint, the model can be forced to quickly grasp and abide by the rule even with limited or no counterexample data, greatly accelerating the learning process. The model's decisions are driven not only by statistical patterns in the data but also by explicit rules that humans can understand. This makes the model's decision-making process closer to human thinking patterns, making it easier for analysts to understand and trust. When a source tracing report can indicate that "this judgment conforms to the company's internal business rule #XX", its persuasiveness far exceeds that of a simple numerical confidence level.

[0066] The above solution elevates security experts from passive "data labelers" to proactive "model architects." They not only provide "fuel" (data) by correcting behavior but also define "traffic regulations" (constraints) through rules, systematically and reliably "infusing" their wisdom and experience into the AI ​​model. This ensures that the system's evolutionary direction remains within the controllable range of human experts, ultimately outputting an AI model that possesses both powerful data-driven capabilities and a deep understanding of business operations, truly aligning with business logic. This is the most solid technical guarantee for achieving the patent's ultimate goal—"outputting intelligent threat intelligence that combines automated efficiency with expert-level accuracy."

[0067] Preferably, the source tracing report presents the logical relationship between the initial AI confidence level of the threat indicator, the corrective actions performed by the user, the reasons for the corrections, and the final confidence level in a visual manner.

[0068] The above solution presents the specific real-time details of the traceability report in step S4. By visualizing and structuring the entire process of human-machine collaborative decision-making, a logically clear traceability report is generated. Its core effect is to build a trust bridge from "black box" automation to "white box" intelligent enhancement and create a continuous value feedback loop. The above solution offers the following benefits across four dimensions: Regarding "decision transparency": it enables end-to-end traceability, significantly enhancing analysts' trust in and willingness to adopt the system's output. They know the system's conclusions have undergone both human and machine verification, and the process is auditable. Regarding "operations and auditing": it empowers efficient operations and compliance, providing the Security Operations Center (SOC) with standardized "operation logs" and "decision-making basis." Regarding "knowledge transfer and training": it solidifies expert experience, allowing newly hired security analysts to quickly understand the company's unique business environment, common false alarm types, and the analytical approaches of senior experts by studying these reports. This enables expert experience to transcend individuals, achieving effective transfer and standardization within the organization. Regarding "system optimization": it provides higher-order feedback loops, offering higher-dimensional guidance for systemic optimization beyond individual data samples. For example, if a large number of corrections involve "business-related" tags, it may mean that a round of reinforcement training of the model specifically for internal business IP data is needed, or feature engineering needs to be improved.

[0069] This visualized source tracing report is not merely an output of this patented method; it is a value amplifier and the starting point for a new round of optimization. By establishing trust, it ensures the healthy operation of the human-machine collaboration model; by empowering operations and auditing, it enhances the overall maturity of the security team; by solidifying knowledge, it accelerates the growth of team capabilities; and ultimately, by providing macro-level insights, it guides the next evolutionary direction of the entire hybrid intelligent system.

[0070] In one embodiment, the attribution report not only includes AI's attribution analysis but also explicitly states: "The final confidence level of this indicator was initially assessed as X by AI, and subsequently adjusted to Y by a security analyst based on [reason for correction]." This makes the attribution process transparent and credible.

[0071] like Figure 5 As shown, a multi-source threat intelligence conflict resolution and contradiction tracing system includes: The AI ​​confidence module is used to acquire first multi-source threat intelligence, extract multi-dimensional features of threat indicators, and generate initial AI confidence and decision attribution information of threat indicators based on a pre-trained benchmark confidence generation model. The training sample generation module is used to present the initial AI confidence and decision attribution information on the preset judgment interface, receive the user's correction operation on the initial AI confidence and the reason for the correction, and generate training samples containing feature vectors, corrected confidence and the reason for the correction. The model optimization module is used to optimize the baseline confidence generation model based on training samples to obtain the optimized confidence generation model. The intelligence processing and attribution module is used to process the second multi-source threat intelligence based on the optimized confidence generation model, generate the final AI confidence score and an attribution report that integrates decision attribution information and human-corrected history.

[0072] This invention also discloses a multi-source threat intelligence conflict resolution and contradiction tracing system, aiming to achieve multi-source threat intelligence conflict resolution and contradiction tracing in a modular manner. The system acquires first multi-source threat intelligence through an AI confidence module, extracts multi-dimensional features of threat indicators, and generates initial AI confidence and decision attribution information for threat indicators based on a pre-trained baseline confidence generation model. The initial AI confidence and decision attribution information are presented on a preset judgment interface using training samples. The system receives user correction operations and reasons for the initial AI confidence, generating training samples containing feature vectors, corrected confidence, and correction reasons. A model optimization module optimizes the baseline confidence generation model based on the training samples to obtain an optimized confidence generation model. Finally, an intelligence processing and tracing module processes second multi-source threat intelligence based on the optimized confidence generation model, generating a final AI confidence and a tracing report that integrates decision attribution information and manual correction history.

[0073] An electronic device includes at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method as described above.

[0074] The present invention also provides an electronic device in which a processor can realize a method for resolving conflicts and tracing the source of multi-source threat intelligence.

[0075] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.

[0076] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, they perform the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0077] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0078] The foregoing has provided a detailed description of a method, system, and device for resolving multi-source threat intelligence conflicts and tracing the source of contradictions. The above description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for resolving conflicts and tracing the root causes of multi-source threat intelligence, characterized in that, Includes the following steps: Acquire first multi-source threat intelligence, extract multi-dimensional features of threat indicators, and generate initial AI confidence and decision attribution information of the threat indicators based on a pre-trained baseline confidence generation model; The initial AI confidence level and the decision attribution information are presented on the preset judgment interface. The user's correction operation on the initial AI confidence level and the reason for the correction are received, and training samples containing feature vectors, corrected confidence levels and the reason for the correction are generated. The baseline confidence generation model is optimized based on the training samples to obtain the optimized confidence generation model; The second multi-source threat intelligence is processed based on the optimized confidence generation model to generate the final AI confidence score and a source tracing report that integrates the decision attribution information and the manually corrected history.

2. The method for resolving multi-source threat intelligence conflicts and tracing the source of contradictions as described in claim 1, characterized in that, The optimization of the baseline confidence generation model based on the training samples to obtain the optimized confidence generation model includes the following steps: The training samples are stored in a preset sample pool; Periodically or when the number of samples in the preset sample pool reaches a threshold, a batch of training samples is extracted from the preset sample pool to perform incremental learning or fine-tuning training on the baseline confidence generation model, thereby obtaining an optimized confidence generation model.

3. The method for resolving multi-source threat intelligence conflicts and tracing the source of contradictions as described in claim 2, characterized in that, After obtaining the optimized confidence generation model, the following steps are also included: The performance of the optimized confidence generation model is evaluated and compared with the confidence generation model currently used online. Based on the comparison results, a decision will be made on whether to update the optimized confidence generation model to a new online version.

4. The method for resolving multi-source threat intelligence conflicts and tracing the source of contradictions as described in claim 1, characterized in that, The multidimensional features include, but are not limited to, at least one of the following: dynamic reputation of intelligence sources, intelligence timeliness, richness of intelligence content, external coordination or conflict network characteristics of threat indicators, geographical characteristics, behavioral characteristics, correlation characteristics, and time characteristics.

5. The method for resolving multi-source threat intelligence conflicts and tracing the source of contradictions as described in claim 1, characterized in that, The presentation on the preset analysis interface includes the following steps: Based on at least one of the following factors—the uncertainty of the initial AI confidence level, the severity of the conflict of multi-source intelligence, or the criticality of the assets associated with the threat indicators—the threat indicators that require manual assessment are prioritized. Based on the priority ranking results, an intelligent assessment task list is generated and presented. This intelligent assessment task list is used to guide users to prioritize the handling of high-priority threat indicators.

6. The method for resolving multi-source threat intelligence conflicts and tracing the source of contradictions as described in claim 1 or 2, characterized in that, The reasons for the correction include: tags selected from a predefined set of tags and / or text descriptions entered by the user; The predefined tag set includes tags used to characterize false alarms, business relevance, and attribution.

7. The method for resolving multi-source threat intelligence conflicts and tracing the source of contradictions as described in claim 1 or 2, characterized in that, The optimization of the baseline confidence generation model based on the training samples further includes the following steps: Based on one or more of the aforementioned reasons for correction, construct or update the expert rule base; The rules in the expert rule base are used as regularization constraints and introduced into the incremental learning or fine-tuning training process of the baseline confidence generation model.

8. The method for resolving multi-source threat intelligence conflicts and tracing the source of contradictions as described in claim 1, characterized in that, The source tracing report presents the logical relationship between the initial AI confidence level of the threat indicator, the corrective actions performed by the user, the reasons for the corrections, and the final confidence level in a visual manner.

9. A multi-source threat intelligence conflict resolution and contradiction tracing system, characterized in that, include: The AI ​​confidence module is used to acquire first multi-source threat intelligence, extract multi-dimensional features of threat indicators, and generate initial AI confidence and decision attribution information of the threat indicators based on a pre-trained benchmark confidence generation model. The training sample generation module is used to present the initial AI confidence and the decision attribution information on a preset judgment interface, receive the user's correction operation on the initial AI confidence and the reason for the correction, and generate training samples containing feature vectors, corrected confidence and the reason for the correction. The model optimization module is used to optimize the baseline confidence generation model based on the training samples to obtain the optimized confidence generation model. The intelligence processing and tracing module is used to process the second multi-source threat intelligence based on the optimized confidence generation model, generate the final AI confidence score and a tracing report that integrates the decision attribution information and the manual correction history.

10. An electronic device comprising at least one processor and a memory communicatively connected to the at least one processor; the memory storing instructions executable by the at least one processor to enable the at least one processor to perform the method as claimed in any one of claims 1 to 8.

Images