Cognitive domain deception attack detection method and system based on generative AI

Abstract

This application provides a cognitive deception attack detection method and system based on generative AI, belonging to the field of artificial intelligence technology. The method includes: acquiring multimodal data of the target object; calculating a pre-judgment confidence score based on text data; extracting multi-dimensional features from multimodal data with a pre-judgment confidence score below a threshold; calculating anomaly scores, including generative AI traces, factual consistency, and emotion-induced anomalies; performing sensitivity analysis on the application scenario to which the target object belongs, and obtaining a sensitivity matching score; aggregating anomalies based on each anomaly score, sensitivity matching score, and the contribution of each score to obtain a comprehensive cognitive deception index; determining the attack type based on the anomaly score; generating a structured anomaly tracing graph based on the contribution and attack type; and outputting the comprehensive cognitive deception index and its level, attack type, anomaly tracing graph, and handling operation identifier. This solution reduces the false negative rate and false positive rate, and improves detection accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a cognitive domain deception attack detection method and system based on generative AI. Background Technology

[0002] With the continuous development and large-scale application of artificial intelligence technology, the ways people obtain information and make decisions are undergoing profound changes. In various scenarios such as finance, government affairs, and e-commerce, a large amount of information, investment advice, and policy interpretations are disseminated through social media platforms, financial information platforms, investor interaction platforms, and other channels, becoming an important reference for the public to obtain information, form judgments, and make decisions.

[0003] Among this information, some content is generated with the assistance of artificial intelligence models or is completely automated. This type of content plays a positive role in improving efficiency and reducing production costs, and is considered non-malicious AI-assisted content.

[0004] However, at the same time, an attack method that utilizes generative artificial intelligence technology to create content with cognitive manipulation intent is rapidly evolving. This type of attack is not a traditional virus, Trojan, or exploit attack; instead, it targets people's perception, cognition, judgment, and decision-making processes. Through highly realistic and cognitively manipulative false or misleading content, it induces the target to make incorrect judgments and thus act in accordance with the attacker's expectations. Because generative AI can mass-produce "partially true, partially false" content—that is, content where background information or general descriptions are consistent with credible sources, but key semantic blocks of information (such as titles, concluding sentences, yield rates, financial indicators, risk levels, trading recommendations, etc.) are altered—this type of attack is extremely deceptive and has a high degree of dissemination.

[0005] Existing content security detection technologies mainly target traditional content risks, spam, and malware propagation. However, they have limited detection dimensions, fail to consider the abnormal combination relationship between generative AI traces and factual consistency, struggle to effectively identify "half-true, half-false" tampering attacks, and cannot dynamically adjust the participation weight of each dimension based on the hit rate of sensitive elements in the application scenario.

[0006] Therefore, there is a need for a cognitive domain deception attack detection method and system that can integrate multi-dimensional abnormal features, identify abnormal combination relationships, and adapt to the hit degree of sensitive items. Summary of the Invention

[0007] In view of this, the embodiments of this application provide a cognitive domain deception attack detection method and system based on generative AI, which at least partially solves the problems of existing technologies, such as single detection dimension, failure to consider abnormal combination relationship between generative AI traces and factual consistency, inability to adapt to dynamic adjustment of detection weights for sensitive elements in different application scenarios, and difficulty in effectively identifying "half-true, half-false" cognitive deception attacks.

[0008] In a first aspect, embodiments of this application provide a cognitive domain deception attack detection method based on generative AI, including:

[0009] Acquire multimodal data of the object to be detected, including text data, image data, audio data, and video data; The predicted confidence level is calculated based on the text data. The predicted confidence level is obtained by calculating the explicit keyword matching degree, basic scene matching degree, and statistical distribution deviation of the text data. Multi-dimensional feature extraction is performed on multi-modal data with a predicted confidence level lower than the first preset threshold. Anomaly scores corresponding to each dimension feature are calculated, and sensitivity analysis of the application scenario to which the object to be detected belongs is performed to obtain sensitivity matching scores. The anomaly scores corresponding to each dimension feature include generative AI trace anomaly scores, factual consistency anomaly scores, and emotion-induced anomaly scores. Anomalies are aggregated based on the anomaly scores, sensitive item matching scores, and contributions of each score corresponding to the features of each dimension to obtain a cognitive deception comprehensive index. The contribution of each score is dynamically adjusted according to the sensitive item hit results in the application scenario of the object to be detected. The cognitive deception comprehensive index level is determined based on the cognitive deception comprehensive index, and the attack type is determined based on the anomaly scores corresponding to each dimension of features. A structured anomaly attribution map is generated based on the contribution level and attack type corresponding to each score; Output cognitive deception comprehensive index, attack type, cognitive deception comprehensive index level, structured anomaly tracing map, and handling operation identifier based on pre-judgment confidence level.

[0010] According to a specific implementation of an embodiment of this application, the expression for the predicted confidence level is: , Where P is the prediction confidence level, S_keyword is the explicit keyword matching degree, S_stat is the statistical distribution deviation, S_match_base is the basic scene matching degree, and α, β and γ are the weight coefficients corresponding to the explicit keyword matching degree, statistical distribution deviation and basic scene matching degree, respectively.

[0011] According to a specific implementation of an embodiment of this application, the step of extracting multi-dimensional features from multimodal data with a predicted confidence level lower than a first preset threshold includes: A second preset threshold is set for the predicted confidence level, and the second preset threshold is less than the first preset threshold. Multi-dimensional feature extraction is performed on multi-modal data with a predicted confidence level lower than the first preset threshold but greater than or equal to the second preset threshold, and on multi-modal data with a predicted confidence level lower than the second preset threshold. The priority of multi-dimensional feature extraction for multi-modal data with a predicted confidence level lower than the second preset threshold is greater than that for multi-modal data with a predicted confidence level lower than the first preset threshold but greater than or equal to the second preset threshold.

[0012] According to a specific implementation of an embodiment of this application, the calculation process of the generative AI trace anomaly score includes: Extract text perplexity anomaly features and text statistical anomaly features from the text data of the object to be detected; extract image anomaly features from the image data of the object to be detected; extract audio anomaly features from the audio data of the object to be detected; extract video anomaly features from the video data of the object to be detected. The outliers of text perplexity anomaly features, text statistical anomaly features, image anomaly features, audio anomaly features, and video anomaly features are weighted and aggregated to obtain a generative AI trace anomaly score. The calculation of outliers in the text perplexity anomaly feature includes: The perplexity of the text data to be detected is calculated word by word under the reference language model to obtain the perplexity value sequence of each word position. The perplexity value sequences of all word positions are summarized to obtain the perplexity distribution of the text to be detected. The degree of deviation between the perplexity distribution of the text to be detected and the perplexity distribution of the reference human text is calculated based on the distribution difference metric function. The degree of deviation is used as the outlier value of the text perplexity abnormal feature.

[0013] According to a specific implementation of an embodiment of this application, the calculation process of the factual consistency anomaly score includes: Extract entity sets and relation triples from the text data of the object to be detected; Based on entity sets and relation triples, a search is performed in a pre-built trusted source library to obtain a set of candidate trusted texts corresponding to the text data of the object to be detected; The text data of the object to be detected and the candidate credible text set are divided into multiple semantic blocks. The fact difference degree is calculated for each semantic block. The fact difference degree is the semantic difference between the semantic block of the text data of the object to be detected and the corresponding semantic block of the candidate credible text set. Based on the fact difference degree of each semantic block, the fact credibility deviation degree is calculated. Cross-modal semantic consistency detection is performed on the image data and video data of the object to be detected and the corresponding text data to obtain cross-modal consistency scores. Cross-modal anomaly scores are then calculated based on the cross-modal consistency scores. We fused the factual credibility deviation and cross-modal anomaly scores with weights to obtain the factual consistency anomaly score.

[0014] According to a specific implementation of an embodiment of this application, the calculation process of the emotion-induced abnormality score includes: The comprehensive sentiment score of the text data of the object to be detected is calculated by using a sentiment analysis model. Based on the comprehensive sentiment score and the baseline sentiment range corresponding to the content type, polarity outliers are calculated. Construct an emotion manipulation keyword library containing words of urgency, words of extreme emotions, and words of repeated inducement. Calculate the word frequency density of words that hit the emotion manipulation keyword library in the text data of the target object. At the same time, detect the number of times the same keyword appears repeatedly within a preset distance. Calculate the outlier value of the emotion manipulation pattern based on the word frequency density and the number of times the same keyword appears repeatedly within the preset distance. Construct a call-to-action phrase library, count the frequency of occurrence of call-to-action phrases in the text data of the target object, and detect sentence structure features. Based on the frequency of occurrence of call-to-action phrases in the library and sentence structure features, calculate call-to-action outliers. Normalization and maximum value aggregation were performed on polarity outliers, emotion manipulation pattern outliers, and call to action outliers to obtain an emotion-induced anomaly score.

[0015] According to a specific implementation of this application, the anomaly scores corresponding to each dimension feature further include a propagation coordination anomaly score, and the calculation process of the propagation coordination anomaly score includes: Calculate outliers related to release time synchronization, semantic skeleton similarity, common links, and account association. Based on anomalies related to release time synchronization, semantic skeleton similarity, common links, and account association, a propagation collaboration anomaly score is obtained. The expression for the propagation collaboration anomaly score is: C = 100×max(C_time×C_sem, C_link, C_account), Where C is the propagation collaboration anomaly score, max is the maximum value function, C_time is the release time synchronization anomaly, C_sem is the semantic skeleton similarity anomaly, C_link is the co-link anomaly, and C_account is the account association anomaly.

[0016] According to a specific implementation of an embodiment of this application, the expression for the cognitive deception comprehensive index is: CDI = F(A, B, E, C, S_match, R_combo), Wherein, CDI is the cognitive deception comprehensive index, A is the generative AI trace anomaly score, B is the factual consistency anomaly score, E is the emotion-induced anomaly score, S_match is the sensitive item matching score, R_combo is the abnormal combination relationship function, and F is the nonlinear aggregation function.

[0017] According to a specific implementation of an embodiment of this application, the step of determining the attack type based on the anomaly scores corresponding to each dimension feature includes: If the abnormal score of generative AI traces is ≥ 60, the abnormal score of factual consistency is < 40, and the abnormal score of emotion induction is < 40, then the attack type is determined to be pure AI generation. If the factual consistency anomaly score is ≥60 and the factual difference of the semantic block is ≥0.7, then the attack type is determined to be a half-true, half-false tampering type. If the emotional inducement anomaly score is ≥55, or any one of the polarity anomaly, emotional manipulation pattern anomaly, or call to action anomaly is ≥70, then the attack type is determined to be emotional inducement manipulation. If the anomaly scores corresponding to multiple dimensional features simultaneously exceed the corresponding attention thresholds, the attack type is determined to be composite.

[0018] Secondly, embodiments of this application also provide a cognitive domain deception attack detection system based on generative AI, used to implement the cognitive domain deception attack detection method based on generative AI as described in any embodiment of the first aspect, the system comprising: The data acquisition module is used to acquire multimodal data of the object to be detected, including text data, image data, audio data, and video data. The rapid prediction module is used to calculate the prediction confidence based on text data. The prediction confidence is calculated based on the explicit keyword matching degree, basic scene matching degree, and statistical distribution deviation of the text data. The feature extraction module is used to extract multi-dimensional features from multimodal data with a predicted confidence level lower than the first preset threshold, calculate the anomaly score corresponding to each dimension feature, and perform sensitivity analysis on the application scenario to which the object to be detected belongs, and obtain the sensitivity matching score. The anomaly scores corresponding to each dimension feature include generative AI trace anomaly score, factual consistency anomaly score, and emotion-induced anomaly score. The anomaly aggregation and judgment module is used to aggregate anomalies based on the anomaly scores, sensitive item matching scores, and contribution values ​​of each score corresponding to the features of each dimension, to obtain a cognitive deception comprehensive index, and dynamically adjust the contribution values ​​of each score according to the sensitive item hit results in the application scenario to which the object to be detected belongs; determine the cognitive deception comprehensive index level based on the cognitive deception comprehensive index, and determine the attack type based on the anomaly scores corresponding to each dimension feature. The anomaly attribution graph generation module is used to generate a structured anomaly attribution graph based on the contribution of each score and the attack type. The results output module is used to output the cognitive deception comprehensive index, attack type, cognitive deception comprehensive index level, structured anomaly tracing map, and handling operation identifier based on pre-judgment confidence level.

[0019] Beneficial effects: The cognitive domain deception attack detection method and system based on generative AI in this application embodiment (1) overcomes the limitation of existing technologies that judge from only a single dimension by simultaneously detecting three necessary dimensions: generative AI traces, factual consistency and emotional induction, and can further combine two enhanced dimensions: propagation collaboration and the degree of sensitivity hit. Because multi-dimensional comprehensive detection can cover a variety of attack types such as pure AI generation, semi-true and semi-false tampering, emotional induction manipulation, and collaborative propagation amplification, it can effectively distinguish between "non-malicious AI-assisted generated content" and "deceptive content with cognitive manipulation intent", thereby reducing the false negative rate and false positive rate of single-dimensional detection.

[0020] (2) By comparing facts at the semantic block level, higher weights are given to key semantic blocks such as titles, conclusion sentences, yield rates, financial indicators, and investment recommendations. Because the weight q_i of key semantic blocks is higher than that of ordinary background description sentences (e.g., title q=0.95 compared to background description q=0.30), it can identify half-true and half-false attacks where the background information is consistent with the credible source but the key information semantic blocks are tampered with, thus solving the problem of insufficient sensitivity of existing full-text comparison methods to local tampering of key positions.

[0021] (3) By using the dynamic adjustment mechanism of dimensional participation, the contribution of factual consistency and sensitive item matching degree can be improved in highly sensitive scenarios such as finance and government affairs. Because the confirmation of factual anomalies has the highest priority in financial scenarios, followed by generative AI trace anomalies and emotion-induced anomalies, the participation coefficients of each dimension can be dynamically adjusted according to the characteristics of the application scenario, thereby improving the detection accuracy in highly sensitive scenarios.

[0022] (4) Through the serial hierarchical judgment architecture, the rapid prediction layer completes the initial judgment of anomalies within 500ms, and the multi-dimensional fine judgment layer performs full-modal anomaly verification on samples above the attention threshold. Because the rapid prediction layer only uses text keywords and surface anomaly pattern detection, it can meet the real-time requirements. Because the multi-dimensional fine judgment layer calls full-modal anomaly trace detection, semantic-level fact credibility assessment, propagation collaborative analysis and anomaly source map generation, it can ensure detection accuracy. Thus, it simultaneously takes into account the technical requirements of low latency and high accuracy.

[0023] (5) Adversarial training targeting cognitive domain deception features generates adversarial samples that maintain the attack labels and deception intent during model training or update. Because adversarial samples cover a variety of perturbation types such as text synonym replacement, sentence transformation, perturbation of non-core fact expression, and small variations of key fact values, they can improve the robustness of the model to slightly modified adversarial cognitive domain deception attack content, thereby reducing the risk of the model being bypassed.

[0024] (6) Through the anomaly tracing map generation mechanism, each detection result contains five dimensions of anomaly tracing map data (map header, anomaly map summary, multidimensional anomaly tracing record, cognitive comprehensive index derivation path, and handling guidelines); because the anomaly tracing map organizes multidimensional evidence according to the hierarchical structure of "score → location → content → basis", it can bring the following beneficial effects: Structured assembly of multi-dimensional evidence: It is not a simple listing of features, but an organization according to a hierarchical structure, which makes it easier for people to quickly locate anomalies and significantly reduces the cost of manual review; Precise location of anomalies: It not only tells you "there is an anomaly", but also "where the anomaly is" (text paragraph index, image region coordinates, audio time period, video frame interval), which shortens the manual review time from "reading through the whole text" to "focusing on reviewing the anomaly location"; Interpretability of the derivation path of the Cognitive Comprehensive Index: The calculation process of CDI (formula, contribution of each dimension) is clearly given, which meets the compliance requirements of "interpretable AI" in scenarios such as finance and government affairs, and makes the test results auditable and traceable; Traceability of anomaly tracing graph: Through content hash and graph ID, it supports tracing and querying of historical detection records, which can be used in scenarios such as model performance auditing, regulatory inspection, and false alarm / false alarm analysis; Cross-dimensional evidence correlation: Explain the correlation between evidence in each dimension in the derivation path of the cognitive comprehensive index (for example, if "abnormality of factual consistency" and "abnormality of emotional induction" occur at the same time and the abnormal content points to the same entity, it indicates that this is a "compound attack of purposeful factual alteration + emotional manipulation"), provide an explanation of the combined risk, and improve the persuasiveness of the detection results; This improves the interpretability, traceability, and supportability of test results for decision-making. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a cognitive domain deception attack detection method based on generative AI according to an embodiment of the present invention; Figure 2 This is a diagram of a serial hierarchical analysis architecture according to an embodiment of the present invention; Figure 3 This is a flowchart of the fact consistency anomaly score calculation according to an embodiment of the present invention; Figure 4 This is a flowchart of a multi-dimensional feature extraction process according to an embodiment of the present invention; Figure 5 This is a flowchart of adversarial training and model update according to an embodiment of the present invention. Detailed Implementation

[0027] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0028] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0030] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The illustrations only show the components related to this application and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0031] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0032] The abbreviations and key terms in this application are defined as follows: Cognitive domain deception attack: refers to the use of generative artificial intelligence technology to create content with cognitive manipulation intent, and to manipulate the target object into making incorrect judgments about risks through identity spoofing, fact mismatch, emotional inducement, collaborative propagation or action guidance, etc. This invention does not directly detect the target object's psychological state or cognitive process, but detects observable technical features related to cognitive domain deception attack, including content anomalies, fact anomalies, emotional inducement anomalies, propagation collaboration anomalies, and the degree of sensitivity hit.

[0033] Generative AI Trace Characteristics: When generative AI models generate content, they leave detectable technical traces in text, images, audio, or video. For example, AI-generated text differs from naturally written text in terms of perplexity and statistical distribution; images exhibit characteristics in frequency domain noise and local texture consistency; audio and video also display their own anomalous behaviors. While these traces may not be visually identifiable, they can be detected through technical means and serve as a basis for determining whether content was generated by AI.

[0034] Factual consistency feature: This measure is used to assess the degree of consistency between factual information stated in the content to be detected and corresponding information in a trusted source database. In practice, key information is first extracted through entities and relations, then multi-granularity fact comparison is performed, employing a weighted comparison strategy for different semantic blocks (with key semantic blocks having higher weights). Finally, cross-verification is conducted through cross-modal consistency validation. These steps yield a quantified consistency feature.

[0035] Emotionally manipulative characteristics: Some content employs emotionally manipulative language patterns that exceed the normal range of expression, such as abnormal polarity (excessively positive or negative), emotional manipulation patterns (deliberately sensationalizing), and abnormal sense of urgency and calls to action (e.g., expressions like "take action now" or "if you miss this opportunity, you'll have to wait another ten years"). It should be noted that this invention detects the observable expression patterns within the content itself, without considering the actual emotional state of the target audience (which is not required by this invention and is difficult to accurately obtain through technical means).

[0036] Propagation Coordination Characteristics: In normal propagation, content posted by different users will naturally differ in terms of posting time, wording style, and resources used. However, if an organized, coordinated propagation attack exists, some abnormal coordination phenomena will appear, such as highly synchronized posting times, highly similar semantic skeletons, use of the same links or resources, and relationships between accounts. Propagation coordination characteristics identify and quantify these abnormal coordination phenomena to determine whether an organized, coordinated propagation attack exists.

[0037] Sensitive item hit degree: refers to the degree of matching between the content to be detected and the predefined sensitive elements in its application scenario. The higher the matching degree, the greater the potential harm when the content is used as a carrier of cognitive domain deception attack in that scenario.

[0038] Serial hierarchical analysis refers to an analysis architecture that divides the detection process into two serial layers. The first layer is a rapid prediction layer, which outputs prediction confidence based on text features, keywords, the hit rate of sensitive items, and a lightweight model within a preset time threshold. The second layer is a multi-dimensional precision analysis layer, which calls multi-modal consistency detection, trusted source comparison, propagation collaborative analysis, and anomaly tracing map generation modules for samples above the attention threshold.

[0039] Anomaly combination relationship: refers to the synergistic relationship of multiple detection features that simultaneously exhibit anomalies, rather than the independent judgment results of each dimension feature; for example, when generative AI trace anomalies and factual consistency anomalies occur simultaneously, their comprehensive anomaly confidence is higher than the linear sum of the independent scores of the two dimensions. This invention uses this as the core basis for anomaly aggregation to drive the calculation of the cognitive deception comprehensive index CDI.

[0040] The applicant's research found that existing methods for detecting cognitive domain deception attacks or AI-generated content can be mainly categorized as follows: The first category: Methods based on single-modal AI-generated trace detection. This method targets a single modality in text, images, audio, or video to detect whether it was generated by AI. For text, perplexity or certain statistical features (such as sentence length distribution, lexical richness, N-gram repetition rate, etc.) are typically used to determine this. For images, frequency domain features, noise patterns, or local texture consistency are usually detected. For audio, acoustic feature anomalies and temporal consistency are detected. For video, inter-frame consistency and lip-sync features are detected. The workflow of this type of method is as follows: Step 1: Obtain the single-modal content to be detected; Step 2: Extract the AI-generated trace features corresponding to this modality; Step 3: Input the extracted features into the pre-trained binary classification model and output the judgment result of "AI generated" or "human generated"; The second category: Detection methods based on factual consistency. This method compares the factual information stated in the content to be detected with credible sources to determine its authenticity. The process is as follows: Step 1: Perform entity recognition and relation extraction on the text to be detected; Step 2: Compare the extracted entity relationships with the results returned by the knowledge base or search engine; Step 3: Output the authenticity score based on the comparison differences.

[0041] The third category: Detection methods based on propagation behavior. This method identifies organized, coordinated propagation attacks by monitoring the propagation behavior characteristics of the same content across multiple channels and accounts. The process is as follows: Step 1: Collect the dissemination records of the same content on multiple platforms; Step 2: Calculate the propagation synergy features such as release time synchronization, semantic similarity, and account association; Step 3: Determine whether a coordinated propagation attack exists based on the coordinated feature score.

[0042] The above methods rely on a single dimension for labeling and anomaly detection, including statistical feature extraction of single-modal AI-generated traces (text, images, audio, and video, each independent), comparison of credible sources for content factual consistency, and analysis of collaborative patterns in dissemination behavior. They construct detection results using a single-stage discriminant or scoring model, and therefore have the following drawbacks: 1. Because existing technologies mostly adopt single-modal and single-dimensional detection methods, they do not comprehensively evaluate generative AI trace features, factual consistency features, emotion-inducing features, and propagation synergy features, nor do they consider abnormal combination relationships between multiple detection features; therefore, existing technologies cannot effectively identify "half-true, half-false" attacks—that is, attacks in which background information is consistent with trusted sources but key information semantic blocks are tampered with, resulting in a high false negative rate for such attacks.

[0043] 2. Because existing technologies mostly use full-text comparison to detect factual consistency without setting differentiated weights for different semantic blocks, irrelevant differences in ordinary background descriptions and substantial tampering in key information semantic blocks are treated equally, resulting in insufficient sensitivity of the system to local tampering in key locations.

[0044] 3. Because existing technologies do not introduce an adaptive weight adjustment mechanism for the degree of sensitivity hits, the same detection strategy and judgment threshold are used for all scenarios. Therefore, in highly sensitive scenarios such as finance and government affairs, the system cannot dynamically adjust the weight ratio of each dimension feature according to the potential harm of the content, resulting in a high false negative rate in high-risk scenarios.

[0045] 4. Because existing technologies mostly adopt a single-stage detection process, a fixed trade-off must be made between detection accuracy and detection latency. Therefore, either detection speed is sacrificed (comprehensive multimodal analysis, with high latency) or detection accuracy is sacrificed (only light detection, with a high false negative rate). It is impossible to simultaneously achieve low latency and high accuracy.

[0046] 5. Because existing technologies do not perform specialized adversarial training for the characteristics of cognitive domain deception attacks during the model training phase, nor do they design adversarial sample generation strategies that keep the attack labels and deception intentions unchanged, the models are not robust enough to slightly modified adversarial cognitive domain deception attack content and are easily bypassed.

[0047] 6. Because existing technologies do not have a structured evidence chain output mechanism, they only output a single detection conclusion or classification result; therefore, the interpretability of the detection results is insufficient, and in scenarios such as finance and government affairs that require manual review and traceability analysis, it is difficult to support subsequent disposal decisions and evidence collection needs.

[0048] In response to the above situation, this application proposes a cognitive domain deception attack detection method and system based on generative AI.

[0049] To address the first drawback (single-dimensional detection, unable to identify abnormal combination relationships and half-true / half-false attacks), this application solves the problem of current detection methods being single-dimensional and unable to identify multi-feature collaborative anomalies by multi-dimensional feature fusion and abnormal combination relationship identification; To address the second shortcoming (insufficient sensitivity to tampering at key locations in full-text comparison), this application solves the problem of insufficient ability of existing technologies to identify local tampering of key semantic blocks by using semantic block-level weighted fact comparison. To address the third drawback (lack of scene adaptation and high false negative rate in highly sensitive scenes), this application solves the problem that existing technologies cannot dynamically adjust the detection strategy according to the application scenario by adaptively adjusting the weight of the sensitivity hit degree. To address drawback 4 (single-stage detection, where latency and accuracy are difficult to balance), this application solves the problem that existing technologies cannot simultaneously achieve low latency and high accuracy by using a serial hierarchical judgment architecture. To address the shortcomings of point 5 (lack of dedicated adversarial training and insufficient model robustness), this application solves the problems of insufficient robustness and susceptibility to adversarial rewriting and bypassing of existing models by adopting an adversarial training strategy oriented towards cognitive domain deception features. To address the sixth drawback (lack of structured evidence chain and insufficient interpretability), this application solves the problem of insufficient interpretability of existing detection results and difficulty in supporting subsequent disposal decisions through an anomaly tracing map generation mechanism.

[0050] The cognitive domain deception attack detection method and system based on generative AI of this application will be described in detail below with reference to the accompanying drawings.

[0051] In a first aspect, embodiments of this application provide a cognitive domain deception attack detection method based on generative AI, including: Step 101: Obtain multimodal data of the object to be detected. Multimodal data includes text data, image data, audio data, and video data. Step 102: Calculate the prediction confidence score based on the text data. The prediction confidence score is calculated based on the explicit keyword matching degree, basic scene matching degree, and statistical distribution deviation of the text data. Step 103: Extract multi-dimensional features from multimodal data whose predicted confidence level is lower than the first preset threshold, calculate the anomaly score corresponding to each dimension feature, and perform sensitivity analysis on the application scenario to which the object to be detected belongs to obtain the sensitivity matching score. The anomaly scores corresponding to each dimension feature include generative AI trace anomaly score, factual consistency anomaly score, and emotion-induced anomaly score. Step 104: Based on the anomaly scores, sensitive item matching scores, and contribution values ​​corresponding to each dimension of features, perform anomaly aggregation to obtain the cognitive deception comprehensive index. Dynamically adjust the contribution values ​​corresponding to each score according to the sensitive item hit results in the application scenario to which the object to be detected belongs. Determine the cognitive deception comprehensive index level based on the cognitive deception comprehensive index and determine the attack type based on the anomaly scores corresponding to each dimension of features. Step 105: Generate a structured anomaly attribution map based on the contribution and attack type corresponding to each score; Step 106: Output the cognitive deception comprehensive index, attack type, cognitive deception comprehensive index level, structured anomaly tracing map, and handling operation identifier based on the predicted confidence level.

[0052] In this embodiment, by simultaneously detecting three essential dimensions—generative AI traces, factual consistency, and emotional manipulation—as well as an enhanced dimension of sensitivity hit rate, the limitations of existing technologies that rely solely on a single dimension are overcome. Multi-dimensional comprehensive detection can cover various attack types, including purely AI-generated content, semi-fake content, emotional manipulation, and collaborative propagation amplification. Therefore, it can effectively distinguish between "malicious AI-assisted generated content" and "deceptive content with cognitive manipulation intent," thereby reducing the false negative and false positive rates of single-dimensional detection. Through a dynamic adjustment mechanism for the contribution of each feature dimension, the participation coefficients of each dimension can be dynamically adjusted according to the characteristics of the application scenario, thus improving detection accuracy in highly sensitive scenarios. By setting the pre-judgment confidence level in step 2, rapid analysis is performed, followed by multi-dimensional fine-tuning of the data entering step 3. This achieves a serial hierarchical analysis architecture. Rapid pre-judgment uses only text keywords and surface-level anomaly pattern detection, meeting real-time requirements. The multi-dimensional fine-tuning process invokes full-modal anomaly trace detection, semantic-level factual credibility assessment, and anomaly source mapping generation, ensuring detection accuracy. This simultaneously addresses the technical requirements of low latency and high accuracy. The anomaly source mapping generation mechanism improves the interpretability, traceability, and supportability of the detection results for decision-making.

[0053] In practice, the acquisition, storage, and processing of multimodal data in step 1 strictly comply with relevant laws and regulations. Data sources include, but are not limited to, the following legal channels: (a) Publicly accessible content through public channels, including official website announcements, public news reports, and publicly released social media content, will not collect any unauthorized non-public personal information or private communications. (b) Data for which the data controller has obtained legal authorization or consent from the relevant parties, including but not limited to content submitted by users and data collected in accordance with the law within the scope of the enterprise's own business; (c) Data obtained with the legal authorization of the data provider and after signing a data processing agreement.

[0054] For data processing and storage, the following conditions must be met: (1) The data acquisition module performs format standardization and noise reduction preprocessing on the multimodal data. During the preprocessing process, the data involving personal information is de-identified or anonymized to ensure that the purpose of "identifying a specific natural person" cannot be met, and a standardized multimodal dataset is obtained and transmitted to the feature extraction module. (2) The purpose of data processing is clearly limited to the detection of cognitive domain deception attacks, and it shall not be used for purposes other than this purpose; (3) Data storage and transmission shall be protected by security measures that comply with relevant laws and regulations; (4) For content submitted by users, it shall be processed within the framework of a legal and valid user agreement and privacy policy, and users shall be ensured to enjoy the rights of viewing, correcting and deleting as stipulated by law.

[0055] Furthermore, the expression for the predicted confidence level is: , Where P is the pre-judgment confidence level, S_keyword is the explicit keyword matching degree (value 0-1, measuring the proportion of preset keywords hit), S_stat is the statistical distribution deviation degree (value 0-1, measuring the degree of deviation of text statistical features from normal content), and S_match_base is the basic scene matching degree (value 0-1, measuring the density of sensitive elements hit); α, β, and γ are the weight coefficients corresponding to the explicit keyword matching degree, statistical distribution deviation degree, and basic scene matching degree, respectively, α+β+γ=1. In the financial scenario example, the values ​​are: α=0.4, β=0.3, γ=0.3, that is, the explicit keyword matching has the highest contribution, followed by the statistical distribution deviation degree and the basic scene matching degree; the above values ​​can be adjusted according to the actual scenario.

[0056] Furthermore, the step of extracting multi-dimensional features from multimodal data with a predicted confidence level lower than a first preset threshold includes: A second preset threshold is set for the predicted confidence level, and the second preset threshold is less than the first preset threshold. Multi-dimensional feature extraction is performed on multi-modal data with a predicted confidence level lower than the first preset threshold but greater than or equal to the second preset threshold, and on multi-modal data with a predicted confidence level lower than the second preset threshold. The priority of multi-dimensional feature extraction for multi-modal data with a predicted confidence level lower than the second preset threshold is greater than that for multi-modal data with a predicted confidence level lower than the first preset threshold but greater than or equal to the second preset threshold.

[0057] In specific implementation, this embodiment adopts a serial hierarchical analysis architecture, referring to... Figure 2 Specifically, it includes: First layer: Fast prediction layer (target latency < 500ms / target) First judgment node: The rapid pre-judgment module receives the content to be detected. The acquisition of the content to be detected strictly complies with the requirements of relevant laws, regulations and platform user agreements, and only processes public content that can be obtained according to law or content that has obtained legal authorization; the rapid pre-judgment module only performs surface abnormal pattern detection on text data (does not involve in-depth analysis of images, audios or videos), including explicit keyword matching, basic scene element comparison and statistical distribution deviation calculation, calls a lightweight rule model or statistical model, and outputs a pre-judgment confidence P. The explicit keyword matching, basic scene element comparison, statistical distribution deviation and the extracted multi-dimensional features are in a progressive relationship of "coarse-grained pre-judgment → fine-grained precise judgment": the explicit keyword matching corresponds to the preliminary screening of the subsequent fact consistency dimension, the statistical distribution deviation corresponds to the rough estimation of the generative AI trace dimension, and the basic scene element comparison corresponds to the simplified calculation of the sensitive item matching degree dimension. The rapid pre-judgment layer quickly filters out high-trust normal content at extremely low computational cost, and only performs the full-modal deep feature extraction in step 3 on medium and low-trust content.

[0058] The shunt decision unit is used to perform three-level filtering and shunting according to the pre-judgment confidence P: The pre-judgment confidence P is mapped to three credibility intervals based on a first preset threshold Th1 and a second preset threshold Th2: P≥Th1 indicates high credibility (no precise judgment required), Th2≤P<Th1 indicates suspiciousness (standard precise judgment required), and P<Th2 indicates high suspiciousness (priority precise judgment required). Example values for financial scenarios: Th1 can be in the range of 0.7 to 0.8, and Th2 can be in the range of 0.3 to 0.4, and the specific values are adjusted according to the actual deployment scenario.

[0059] If P≥Th1 (high credibility pass threshold), the content is determined to be normal content, the processing identifier "release" is output, and the process ends; If Th2≤P<Th1 (medium credibility to be confirmed), the content is determined to be suspicious content, the processing identifier "mark" is output, and the content is sent to the second-layer multi-dimensional precise judgment queue; If P<Th2 (low credibility abnormal), the content is determined to be highly suspicious content, and is sent to the second-layer multi-dimensional precise judgment queue after the priority is marked.

[0060] Second layer: Multi-dimensional precise judgment layer Second judgment node: The multi-dimensional precise judgment module performs full-modal generative AI trace anomaly detection, semantic-level fact consistency anomaly evaluation, and emotion induction pattern recognition on the content entering the precise judgment queue; if propagation collaboration detection is configured, propagation collaboration anomaly features are further extracted.

[0061] The dimensional participation modifier dynamically generates the contribution of each feature dimension (rather than fixed weights) based on the hit combination of sensitive elements in the application scenario to which the content to be detected belongs. In financial application scenarios, the contribution allocation of each feature dimension in the Cognitive Deception Index (CDI) is determined according to the financial sensitive items hit in the content. The financial sensitive items include one or more of the following: involving specific securities codes or names, containing specific investment operation suggestions, containing specific yield figures, lacking risk warnings, citing regulatory agency names, and containing preset authoritative endorsement terms. Among them, the contribution of the factual consistency dimension and the sensitive item matching degree dimension are dynamically increased according to the hit results of the financial sensitive items. In other application scenarios (such as government affairs, e-commerce, public opinion monitoring, etc.), the same dynamic adjustment logic framework of dimensional participation is adopted as in the financial scenario, that is: defining the sensitive element system of the scenario, detecting the sensitive elements hit by the content, and calculating the contribution of each feature dimension according to the hit results. Taking the government affairs scenario as an example, its sensitive elements may include: involving specific policy statements, citing document numbers, and containing major sensitive words, etc., and the contribution allocation logic of each feature dimension is consistent with that of the financial scenario. The differences between different scenarios are mainly reflected in the list of sensitive elements and the baseline value of contribution, but the logical framework for dynamic adjustment of dimensional participation remains consistent.

[0062] The anomaly aggregation and judgment module performs nonlinear evidence aggregation calculations based on the contribution of each feature dimension and the abnormal combination relationships between multiple feature dimensions to obtain the Cognitive Deception Composite Index (CDI). The calculation of CDI comprehensively considers the anomaly scores of each feature dimension and their abnormal combination relationships. Its general form is CDI = F(A, B, E, C, S_match, R_combo), where F is a nonlinear aggregation function. An exemplary implementation is as follows: Where: σ is the normalization function, S_i is the anomaly score corresponding to each feature dimension, i.e., S_i ∈ {A, B, E, C, S_match} (A is the generative AI trace anomaly score, B is the factual consistency anomaly score, E is the sentiment-induced anomaly score, and S_match is the sensitivity item matching score); w_i is the basic weight coefficient corresponding to each feature dimension, i.e., w_A corresponds to dimension A, w_B corresponds to dimension B, w_E corresponds to dimension E, w_C corresponds to dimension C, and w_S corresponds to dimension S_match. The value range of each basic weight coefficient is 0-1, and it is finally normalized by σ; R_combo_ij is the anomaly combination enhancement coefficient between dimension i and dimension j, i.e., the extra score coefficient when any two dimensions are simultaneously anomaly. When the anomaly scores of the two dimensions simultaneously exceed their respective preset attention thresholds (e.g., A≥50 and B≥50), R_combo_ij>1, otherwise R_combo_ij = 0.

[0063] Taking a financial scenario as an example: if a piece of financial news has A=60 (obvious traces of generative AI) and B=70 (highly inconsistent with the facts), then R_combo_AB is activated. In addition to the base score, the CDI (Combined Risk Indicator) adds an extra enhancement score of R_combo_AB×60×70, making the final CDI value higher than the weighted sum of A and B alone. This is the mathematical manifestation of "abnormal combination relationships": when multiple dimensions are simultaneously abnormal, their combined risk is non-linearly enhanced. F can also use other non-linear aggregation functions to achieve the same effect of "the more abnormal dimensions and the higher the degree of abnormality, the non-linearly enhanced CDI value." The meanings of the abnormal scores for each feature dimension are as follows: A (Generative AI Trace Anomaly Score, 0) 100): Measures the degree to which the content to be detected deviates from the distribution of normal human content in terms of text perplexity, statistical distribution, image frequency domain noise, audio acoustic features, and video frame consistency. The higher the A value, the more obvious the traces of AI generation in the content. For example, A≥60 indicates that the text perplexity distribution is highly consistent with the typical distribution of AI generation, while A<30 is close to the distribution characteristics of natural human content.

[0064] B (Factual Consistency Exception Score, 0) 100): Measures the degree of consistency between the factual information stated in the content to be detected and the corresponding information in the trusted source database. The higher the B value, the more serious the factual deviation. For example, B≥60 indicates that there is a substantial difference between the key semantic blocks (such as the title, concluding sentence, and yield value) and the trusted source, while B<30 indicates that it is basically consistent with the trusted source.

[0065] E (Emotion-Induced Abnormality Score, 0) 100): Measures the salience of emotionally manipulative language patterns in the content. The higher the E value, the stronger the emotional induction. For example, E≥55 indicates the presence of obvious urgency words, extreme emotional expressions, or high-frequency calls to action (such as "buy now" or "if you miss it, you'll have to wait another ten years"). E<30 is within the range of normal objective expression.

[0066] C (Propagation Synergistic Anomaly Score, 0) 100 (optional): Measures the degree of abnormality in the dissemination of the same content across multiple channels and accounts. The higher the C value, the more significant the characteristics of coordinated dissemination. For example, C≥60 indicates that the release time is highly synchronized and the semantic skeleton is highly similar, suggesting organized coordinated dissemination.

[0067] S_match (Sensitive Item Matching Score, 0-100): S_match = min(100, S_match_base +Σ(weight_k × hit_k) × S_match_inc), where: S_match_base is the base matching score (a preset starting value, defaulting to 0); hit_k is the hit flag for the k-th sensitive item (1 for a hit, 0 for a miss); weight_k is the weight coefficient for the k-th sensitive item (reflecting the importance of the sensitive item, such as "involving specific securities codes" having a higher weight than "general financial terms"); S_match_inc is the scenario enhancement coefficient (the amplification factor of each hit sensitive item in the same scenario to the final score, such as 20 for financial scenarios). For example, if a financial news item simultaneously hits three sensitive financial criteria: "involves specific securities codes," "contains investment advice," and "lacks risk warnings," assuming their weights are 0.4, 0.3, and 0.3 respectively, then S_match = min(100, 0 + (0.4+0.3+0.3) × 20) = min(100, 20) = 20. If more sensitive criteria are hit or their weights are higher, the S_match value increases accordingly. A higher S_match value indicates more or more critical sensitive criteria are hit, and a greater potential harm in the current context.

[0068] R_combo (anomaly combination function): This function operates on a subset of dimensions that exhibit anomalies simultaneously. When the scores of any two or more dimensions simultaneously exceed their respective preset attention thresholds (e.g., A≥50 and B≥50), R_combo produces a non-linear enhancement effect, causing the CDI value to be higher than the linear sum of the scores of each dimension. The more anomalous dimensions and the higher the degree of anomalousness, the more significant the enhancement effect. For example, when A=60 and B=70, the enhancement effect of R_combo causes the calculated CDI value to be higher than the linear expectation of (60+70) / 2=65.

[0069] When multiple feature dimensions do not include the propagation collaboration feature dimension, C does not participate in the aggregation calculation; the abnormal aggregation result is input into the judgment model, and the final attack type, cognitive deception comprehensive index CDI and its corresponding level are output.

[0070] The anomaly tracing graph generation module generates an anomaly tracing graph based on the contribution analysis of each feature dimension of the judgment model (the contribution of each feature dimension is obtained based on the sensitivity analysis of the anomaly scores of each dimension to the final CDI value during the CDI calculation process). The anomaly tracing graph specifically includes: graph header (unique identifier, detection timestamp, content hash value, content summary), anomaly graph summary (attack type, cognitive deception comprehensive index CDI, anomaly scores of each dimension and main contributing dimensions), multi-dimensional anomaly tracing records (anomaly scores of each dimension, anomaly location, anomaly content fragments and judgment basis), cognitive comprehensive index derivation path (CDI calculation process, participation coefficients of each dimension, activation status of the anomaly combination relationship function R_combo and judgment threshold description), and handling guidelines (handling operation identifier, review priority and suggested review points). The multi-dimensional anomaly tracing records include at least the following dimensions of tracing records: generative AI trace anomaly dimension, factual consistency anomaly dimension, emotion-induced anomaly dimension, and optional propagation collaboration anomaly dimension and sensitive item matching degree dimension. The anomaly tracing map generation module is triggered after the multidimensional precision judgment layer detection is completed and before the final result is output. For samples that are directly allowed to pass with a prediction confidence level P ≥ Th1 (high confidence pass threshold), a simplified anomaly tracing map can be selectively recorded, containing only the detection records of the fast prediction layer, for use in subsequent model optimization.

[0071] The results output module outputs the final detection results, including attack type, cognitive deception index (CDI) and corresponding level, structured anomaly tracing map data, and handling operation identifier; the handling operation identifier is a structured output field for external systems to call, and this invention does not limit the specific handling process of external systems.

[0072] Furthermore, a three-level or multi-level analysis architecture can be adopted. For example, an intermediate complexity analysis layer can be added between rapid prediction and multi-dimensional fine analysis to conduct further transitional analysis on content whose prediction confidence is near the attention threshold. Alternatively, a parallel analysis architecture can be adopted, with multiple dimensions of fine analysis modules processing in parallel, and finally the output of each dimension being summarized through an anomaly aggregation module.

[0073] The following section provides a detailed description of each feature extraction process and the corresponding anomaly score calculation process.

[0074] In one embodiment, the calculation process of the generative AI trace anomaly score includes: Extract text perplexity anomaly features and text statistical anomaly features from the text data of the object to be detected; extract image anomaly features from the image data of the object to be detected; extract audio anomaly features from the audio data of the object to be detected; extract video anomaly features from the video data of the object to be detected. The outliers of text perplexity anomaly features, text statistical anomaly features, image anomaly features, audio anomaly features, and video anomaly features are weighted and aggregated to obtain a generative AI trace anomaly score. The calculation of outliers in the text perplexity anomaly feature includes: The perplexity of the text data to be detected is calculated word by word under the reference language model to obtain the perplexity value sequence of each word position. The perplexity value sequences of all word positions are summarized to obtain the perplexity distribution of the text to be detected. The degree of deviation between the perplexity distribution of the text to be detected and the perplexity distribution of the reference human text is calculated based on the distribution difference metric function. The degree of deviation is used as the outlier value of the text perplexity abnormal feature.

[0075] In practice, the generative AI trace anomaly score A (0-100) is obtained by normalizing and aggregating the sub-features of the following five modalities: a) Text Perplexity Anomalies: Perplexity (PPL) is a metric that measures the "prediction difficulty" of a language model for a given text. For the text data to be detected, the perplexity of each word under the reference language model is calculated, resulting in a sequence of PPL values ​​for each word position. AI-generated text typically exhibits a generally low PPL value (relatively "safe" word choice and strong predictability) and relatively small fluctuations within sentences; reference human text, on the other hand, has a relatively high PPL value and greater fluctuations (more random word choice). The PPL values ​​for all word positions are summarized into a perplexity distribution D(PPL_sample) for the text to be detected (i.e., a histogram of the frequency of each PPL value range). This distribution is then compared with the perplexity distribution D(PPL_human_ref) for the reference human text (pre-statistically obtained from a large number of manually written, non-AI-generated texts). The degree of deviation between the two distributions, PPL_abn, is calculated using a distribution difference measure function (such as KL divergence). PPL_abn = Distance(D(PPL_sample), D(PPL_human_ref)), The value of PPL_abn is normalized to 0-1. The higher the value, the greater the difference between the text to be detected and the reference human text in terms of perplexity features, and the greater the suspicion of AI generation.

[0076] b) Textual statistical anomalies: Statistical indicators such as sentence length distribution uniformity, lexical richness (e.g., type ratio), punctuation usage patterns, and N-gram repetition rate of the text to be tested are statistically analyzed, and the degree of deviation between these indicators and the baseline distribution of reference human text is calculated. The greater the deviation, the higher the anomaly value.

[0077] c) Image anomaly features: Extract image frequency domain features (such as spectral energy distribution), noise pattern features (such as sensor noise consistency), and local texture consistency features (such as the consistency of local binary pattern LBP) to comprehensively determine whether the image was generated by an AI generation model (such as diffusion model, GAN). If the image exhibits typical AI generation traces in the above features (such as regular artifacts in the frequency domain, excessively uniform local texture), then the anomaly value for this item is high.

[0078] d) Audio anomaly features: Detect audio acoustic feature anomalies (such as spectral feature distribution, fundamental frequency fluctuation pattern) and temporal consistency anomalies (such as whether the acoustic feature transition between adjacent frames is natural) to determine whether the audio was generated by an AI speech synthesis model.

[0079] e) Video anomaly features: Detect the inter-frame consistency (whether the content transition between adjacent frames is natural), lip-sync features (whether the audio matches the lip movements), and spatiotemporal texture features (spatiotemporal consistency of the video frame sequence) to determine whether the video is generated by AI or deepfake.

[0080] The outliers of each modality sub-feature in a) to e) above are normalized to the 0-1 range, and then the final A score (0-100) is obtained by weighted aggregation or maximum value aggregation. The specific aggregation method is not limited to the one listed in this embodiment.

[0081] In one embodiment, the calculation process for the factual consistency anomaly score includes: Extract entity sets and relation triples from the text data of the object to be detected; Based on entity sets and relation triples, a search is performed in a pre-built trusted source library to obtain a set of candidate trusted texts corresponding to the text data of the object to be detected; The text data of the object to be detected and the candidate credible text set are divided into multiple semantic blocks. The fact difference degree is calculated for each semantic block. The fact difference degree is the semantic difference between the semantic block of the text data of the object to be detected and the corresponding semantic block of the candidate credible text set. Based on the fact difference degree of each semantic block, the fact credibility deviation degree is calculated. Cross-modal semantic consistency detection is performed on the image data and video data of the object to be detected and the corresponding text data to obtain cross-modal consistency scores. Cross-modal anomaly scores are then calculated based on the cross-modal consistency scores. We fused the factual credibility deviation and cross-modal anomaly scores with weights to obtain the factual consistency anomaly score.

[0082] In specific implementation, refer to Figure 3 The factual consistency anomaly score B is obtained by fusing the textual factual deviation and cross-modal consistency verification results, specifically including: Step 201: Entity and Relation Extraction. Perform named entity recognition on the text to be detected, and extract entity sets (people, organizations, products, events, times, places, financial indicators, etc.) and relation triples (subject-relation-object); send the extraction results to the trusted source retrieval submodule.

[0083] Step 202: Trusted Source Retrieval. The trusted source retrieval submodule searches a pre-built trusted source database based on the extracted entities and relationships. This database includes publicly disclosed information from regulatory agencies, stock exchanges, and industry associations. The retrieved candidate trusted text set is then transmitted to the multi-granularity fact comparison submodule. It should be noted that the candidate trusted text is not extracted from the text to be tested, but rather authoritative information retrieved from an externally pre-built trusted source database based on the entities and relationships extracted in Step 201. The text to be tested is unknown content requiring analysis, while the candidate trusted text is publicly verifiable authoritative evidence (such as regulatory agency announcements, official disclosures from listed companies, etc.), and their sources are completely different. Dividing the two types of text into semantic blocks aims to establish a one-to-one correspondence between "block to be tested" and "trusted block." After calculating the factual difference δ_i for each block, it is possible to locate which semantic blocks in the text to be tested do not match the authoritative information.

[0084] Step 203: Multi-granularity fact comparison (semantic block differential evaluation). The multi-granularity fact comparison submodule divides the text to be detected and the candidate credible text into multiple semantic blocks, including: title, introduction, core conclusion sentence, rate of return / financial indicators, investment advice, risk disclosure, and background description; and calculates the fact difference δ_i for each semantic block (δ_i represents the first semantic block in the text to be detected). The semantic difference between each semantic block and the corresponding semantic block of the candidate credible text (with a value range of 0-1); and the factual credibility deviation S_fact_abn is calculated based on the preset semantic block importance weight q_i. S_fact_abn = 100 ×Σ(q_i ×δ_i) / Σq_i, Among them, the title, concluding sentence, rate of return, and investment recommendations It is superior to ordinary background description statements; it transmits the factual credibility deviation to the cross-modal consistency verification submodule.

[0085] Typical semantic block weight values: title semantic block q=0.95, lead semantic block q=0.85, core conclusion sentence semantic block q=0.95, rate of return / financial indicators semantic block q=0.90, investment advice semantic block q=0.95, risk disclosure semantic block q=0.80, background description semantic block q=0.30.

[0086] Step 204: Cross-modal consistency verification. The cross-modal consistency verification submodule performs cross-modal semantic consistency detection between the image / video keyframes and their corresponding text descriptions in the content to be detected (e.g., extracting visual feature vectors from the image / video keyframes using a visual encoder, extracting text feature vectors from the corresponding text using a semantic encoder, and calculating the cosine similarity or semantic matching score between the two), obtaining a cross-modal consistency score, Consistency_Score (ranging from 0 to 1, with higher scores indicating greater semantic consistency between the image and text). An exemplary implementation is as follows: extracting visual feature vectors from the image / video keyframes using a visual encoder. The corresponding text will be processed by a semantic encoder to extract text feature vectors. The cosine similarity between the two is calculated as Consistency_Score = Cos(V_img, V_text). Consistency_Score can also be calculated using other cross-modal semantic matching models. This embodiment does not limit the specific consistency calculation method.

[0087] The cross-modal anomaly score S_cross_abn = 100 × (1 - Consistency_Score) is calculated based on the Consistency_Score, meaning that the higher the degree of inconsistency between the text and the image, the higher the S_cross_abn value. Then, the factual credibility deviation S_fact_abn obtained in step 3 is weighted and fused with the S_cross_abn obtained in this step to obtain the factual consistency anomaly score B = α_B × S_fact_abn + (1 - α_B) × S_cross_abn, where α_B is the fusion weight of the textual factual deviation in the factual consistency judgment, typically α_B = 0.7 (text as the primary factor, cross-modal verification as a secondary factor). The data is then sent to the anomaly aggregation and judgment module.

[0088] In one embodiment, the calculation process of the emotion-induced abnormality score includes: The comprehensive sentiment score of the text data of the object to be detected is calculated by using a sentiment analysis model. Based on the comprehensive sentiment score and the baseline sentiment range corresponding to the content type, polarity outliers are calculated. Construct an emotion manipulation keyword library containing words of urgency, words of extreme emotions, and words of repeated inducement. Calculate the word frequency density of words that hit the emotion manipulation keyword library in the text data of the target object. At the same time, detect the number of times the same keyword appears repeatedly within a preset distance. Calculate the outlier value of the emotion manipulation pattern based on the word frequency density and the number of times the same keyword appears repeatedly within the preset distance. Construct a call-to-action phrase library, count the frequency of occurrence of call-to-action phrases in the text data of the target object, and detect sentence structure features. Based on the frequency of occurrence of call-to-action phrases in the library and sentence structure features, calculate call-to-action outliers. Normalization and maximum value aggregation were performed on polarity outliers, emotion manipulation pattern outliers, and call to action outliers to obtain an emotion-induced anomaly score.

[0089] The specific implementation includes the following steps: a) Polarity outlier E_polarity (normalized to 0-1): Analyzes the degree of match between the sentiment polarity (positive / negative rating) of the text and the expected objective expression of the content. The overall sentiment score Sentiment_Score (ranging from -1 to +1, where -1 is extremely negative and +1 is extremely positive) is calculated using a sentiment analysis model. Combined with the baseline sentiment range [L, H] corresponding to the content type (e.g., the baseline sentiment for factual statements should be close to 0, where L is the lower limit and H is the upper limit), E_polarity is calculated as max(0, |Sentiment_Score - (L+H) / 2|-(HL) / 2), representing the degree to which the actual sentiment deviates from the expected baseline range. The closer E_polarity is to 1, the less the sentiment expression conforms to the normal expectations of this type of content, and the more likely it is to be a deliberate manipulation of the reader's emotions.

[0090] b) Outlier Emotional Manipulation Pattern E_emotion_pattern (normalized to 0-1): Detects whether there are emotionally manipulative language patterns in the text that exceed the needs of normal information delivery. An emotional manipulation keyword database is constructed (including urgency words such as "immediately," "right away," "last chance," extreme emotion words such as "amazing," "bombshell," "horrifying," and repetitive leading words such as "crazy," "absolutely," "certainly"). The word frequency density (Density) of the keyword database hits in the text is calculated as the number of hits / total number of words. Simultaneously, the number of times the same keyword appears within a short distance (e.g., within 50 words) is detected (Repeat_Count). The overall calculation is E_emotion_pattern = min(1, α × Density + β × Repeat_Count), where α and β are weighting coefficients, set empirically. The closer E_emotion_pattern is to 1, the more intensive the use of emotionally manipulative language in the text.

[0091] c) Call-to-Action Outliers E_manip (normalized to 0-1): Detects whether there are directive calls to action in the text that exceed the scope of everyday information dissemination. An action call phrase library is constructed (e.g., "Buy Now," "Forward Now," "Take Action Immediately," "Scan to Buy," "Miss it and wait another ten years"), and the frequency of occurrence of phrases matching the library in the text is counted (Call-to-Action_Count). Simultaneously, sentence structure features (Imp_Flag, whether an imperative sentence structure or exclamation mark is used) are detected. The combined calculation is E_manip = min(1, Call_to-Action_Count / Max_Threshold + Imp_Flag), where Max_Threshold is a preset frequency upper limit, and Imp_Flag takes a value of 0 or 0.3 (Imp_Flag is a binary indicator variable: 0.3 when an imperative sentence structure or exclamation mark is detected, otherwise 0). The closer E_manip is to 1, the more frequent or stronger the call to action in the text. After normalizing the outliers of the above three sub-dimensions, the E score is calculated by aggregating the maximum values. E = 100×max(E_polarity, E_emotion_pattern, E_manip) That is, the E value is determined by the one with the highest degree of abnormality among the three sub-dimensions; the specific aggregation method is not limited to this, and other aggregation methods such as weighted average can also be used.

[0092] In one embodiment, refer to Figure 4 The paper presents a multi-dimensional feature extraction process. For data exhibiting propagation behavior, the features in each dimension also include propagation coordination features. The anomaly scores corresponding to each dimension feature also include propagation coordination anomaly scores. The calculation process for the propagation coordination anomaly scores includes: Calculate outliers related to release time synchronization, semantic skeleton similarity, common links, and account association. Based on anomalies related to release time synchronization, semantic skeleton similarity, common links, and account association, a propagation collaboration anomaly score is obtained. The expression for the propagation collaboration anomaly score is: C = 100×max(C_time×C_sem, C_link, C_account), Where C is the propagation collaboration anomaly score, max is the maximum value function, C_time is the release time synchronization anomaly, C_sem is the semantic skeleton similarity anomaly, C_link is the co-link anomaly, and C_account is the account association anomaly.

[0093] In practice, the calculation process for the propagation coordination anomaly score includes: a) Anomaly Value C_time (normalized to 0-1): Detects abnormal synchronization of content posted by multiple accounts within a short period. Using the time point when the content is first detected as the baseline, the time difference Δt_i between other accounts posting highly similar content is calculated. If the Δt_i of a large number of accounts is concentrated within a narrow time window (e.g., posted within a few minutes), it indicates organized synchronized posting behavior. C_time = 1-e^(-k / n), where n is the number of accounts posting simultaneously, and k is the time window concentration coefficient. The more accounts synchronizing and the more concentrated the time, the closer C_time is to 1. A higher C_time indicates a greater probability of coordinated manipulation and dissemination of the content.

[0094] b) Semantic skeleton similarity outlier C_sem (normalized to 0-1): Detects the degree of similarity between multiple texts in their core semantic skeleton. Extract the main semantic structure of each text (subject-verb-object skeleton after removing modifiers), calculate the cosine similarity or edit distance between each pair of texts, and count the proportion of text pairs with similarity exceeding a preset threshold (e.g., similarity ≥ 0.85). C_sem = similar text pair ratio, ranging from 0 to 1. A higher C_sem indicates that although multiple texts differ at the lexical level (avoiding simple deduplication), their core semantics are completely consistent—this is a typical characteristic of batch generation of the same semantic template in "half-true, half-false" attacks. c) C_link outlier (normalized to 0-1): Detects whether multiple pieces of content contain the same or highly similar links, images, videos, or attachments. It calculates the proportion of content in a collection that shares the same external resource (same URL, same image hash, same attachment fingerprint). C_link = Number of pieces sharing the same resource / Total number of pieces of content. A higher C_link indicates that multiple pieces of content point to the same external resource, further supporting the claim that this content originates from the same source of manipulation.

[0095] d) Account Association Anomaly Value C_account (normalized to 0-1): Detects whether there are abnormal associations between multiple accounts that publish content, including: whether the registration time is concentrated (e.g., a large number of accounts register within a week), whether the IP addresses overlap (whether they are published from the same IP range), and whether the device fingerprints are similar (e.g., the same User-Agent, browser fingerprint). The number of anomalies in the above association dimensions is counted, and C_account = number of abnormal associations / total number of detected items. The higher the C_account, the more likely these accounts are fake accounts created in batches by the same operator, rather than real independent users.

[0096] The calculation method for the propagation coordination anomaly score C is as follows: C = 100 × max(C_time×C_sem, C_link, C_account) C is calculated as the maximum of the three factors: the joint anomaly of time synchronization and semantic skeleton (both must be significantly high), common link anomaly, and account association anomaly. C_time × C_sem reflects the combined synergistic effect of "synchronized release time and similar semantic skeleton." An occasional high value in a single dimension will not lead to a high C; only when both dimensions are high will the product term significantly increase. C_link and C_account are independent auxiliary indicators; a high value in either one is sufficient to trigger a high C. Compared to linear weighted summation, this method better reflects the actual characteristics of propagation-based collaborative attacks: simple time synchronization with different semantics (high C_time but low C_sem) may be coincidental; only time synchronization with identical semantics (high C_time and high C_sem) constitutes a substantial anomaly. Typical values ​​(financial scenario): time synchronization threshold of 30 minutes, semantic similarity threshold of 0.85, and independent trigger thresholds for C_link and C_account of 0.6. Specific values ​​are adjusted according to the actual deployment scenario.

[0097] In one embodiment, the expression for the cognitive deception comprehensive index is: CDI = F(A, B, E, C, S_match, R_combo), Wherein, CDI is the cognitive deception comprehensive index, A is the generative AI trace anomaly score, B is the factual consistency anomaly score, E is the emotion-induced anomaly score, S_match is the sensitive item matching score, R_combo is the abnormal combination relationship function, and F is the nonlinear aggregation function.

[0098] In one embodiment, the attack type determination based on the anomaly scores corresponding to each dimension of features includes: If the abnormal score of generative AI traces is ≥ 60, the abnormal score of factual consistency is < 40, and the abnormal score of emotion induction is < 40, then the attack type is determined to be pure AI generation. If the factual consistency anomaly score is ≥60 and the factual difference of the semantic block is ≥0.7, then the attack type is determined to be a half-true, half-false tampering type. If the emotional inducement anomaly score is ≥55, or any one of the polarity anomaly, emotional manipulation pattern anomaly, or call to action anomaly is ≥70, then the attack type is determined to be emotional inducement manipulation. If the anomaly scores corresponding to multiple dimensional features simultaneously exceed the corresponding attention thresholds, the attack type is determined to be composite.

[0099] Furthermore, the method of determining the attack type based on the anomaly scores corresponding to each dimension of features also includes: if C≥60, then the attack type is determined to be a cooperative propagation amplification attack.

[0100] Furthermore, the Cognitive Deception Index (CDI) level determination rules are as follows: A CDI < 30 indicates a reliable value. A CDI of 30 ≤ CDI < 50 requires attention; A value of 50 ≤ CDI < 75 is considered suspicious. A CDI of 75 or higher indicates a high degree of suspicion.

[0101] In one embodiment, refer to Figure 5 The method also includes model training and continuous updating. During the model training or model update phase, an adversarial training strategy is adopted: adversarial samples are generated based on the original training samples. During the generation process (text perturbation, image perturbation, and cross-modal perturbation), the attack labels and deceptive intent of the samples remain unchanged. The perturbation types include one or more of the following: text-level synonym replacement, sentence transformation, perturbation of non-core fact expressions, minor variations of key fact values, image compression, or local editing. The original samples and adversarial samples are mixed for training to minimize the joint loss function on the original samples and adversarial samples, thereby achieving model training / model update.

[0102] During the actual deployment phase, the cognitive domain deception attack identification model is periodically retrained or incrementally updated by collecting false alarm samples and missed alarm samples from manual review feedback.

[0103] In one embodiment, the identification model may be a rule model, a statistical model, a machine learning model, a neural network model, or a combination thereof; the calculation of propagation collaborative features may be a weighted fusion formula, graph neural network modeling, or statistical threshold judgment; the trusted source library may be official public information, a dynamic trusted source library, a third-party authoritative data source API, or multi-source cross-validation; the semantic block segmentation may be a preset seven types of semantic blocks, an automatic semantic block segmentation algorithm based on text structure analysis, an automatic semantic block identification method based on attention mechanism, or a user-defined semantic block template.

[0104] In one embodiment, the present application's solution is used to detect cognitive domain deception attacks on the financial content to be detected, and the following exemplary detection results are obtained (the specific values ​​do not constitute a limitation on the scope of protection of the present invention): Input content: "[Breaking News] It is rumored that the China Securities Regulatory Commission has just approved XX Technology (stock code: 688XXX) to cooperate with an AI giant. Net profit in Q1 2025 is expected to increase by 300% year-on-year. It is recommended to buy immediately, or you will have to wait another ten years!" Test results: Attack type: Semi-realistic alteration (complex emotional manipulation). Cognitive Deception Composite Index (CDI): High risk (CDI=69.8); Anomaly scores for each dimension: A=45, B=88, E=92, C=30, S_match=95; Anomaly source tracing map: Fact Anomaly: The projected 300% year-on-year increase in net profit lacks authoritative source verification, and the difference δ=0.88 (key semantic block) is compared with a reliable source database. Abnormal emotions: Includes high-frequency emotionally suggestive words such as "blockbuster," "buy now," and "if you miss it, you'll have to wait another ten years"; Sensitive Item Hit Level: Involves specific securities codes, contains specific yield figures, contains specific investment operation suggestions, and lacks risk warnings, hitting 4 sensitive financial items; Generative AI traces: The text perplexity is within the distribution range of AI-generated text, but the deviation is not high; Operation label: Review.

[0105] Secondly, embodiments of this application also provide a cognitive domain deception attack detection system based on generative AI, used to implement the cognitive domain deception attack detection method based on generative AI as described in any embodiment of the first aspect, the system comprising: The data acquisition module is used to acquire multimodal data of the object to be detected, including text data, image data, audio data, and video data. The rapid prediction module is used to calculate the prediction confidence based on text data. The prediction confidence is calculated based on the explicit keyword matching degree, basic scene matching degree, and statistical distribution deviation of the text data. The feature extraction module is used to extract multi-dimensional features from multimodal data with a predicted confidence level lower than the first preset threshold, calculate the anomaly score corresponding to each dimension feature, and perform sensitivity analysis on the application scenario to which the object to be detected belongs, and obtain the sensitivity matching score. The anomaly scores corresponding to each dimension feature include generative AI trace anomaly score, factual consistency anomaly score, and emotion-induced anomaly score. The anomaly aggregation and judgment module is used to aggregate anomalies based on the anomaly scores, sensitive item matching scores, and contribution values ​​of each score corresponding to the features of each dimension, to obtain a cognitive deception comprehensive index, and dynamically adjust the contribution values ​​of each score according to the sensitive item hit results in the application scenario to which the object to be detected belongs; determine the cognitive deception comprehensive index level based on the cognitive deception comprehensive index, and determine the attack type based on the anomaly scores corresponding to each dimension feature. The anomaly attribution graph generation module is used to generate a structured anomaly attribution graph based on the contribution of each score and the attack type. The results output module is used to output the cognitive deception comprehensive index, attack type, cognitive deception comprehensive index level, structured anomaly tracing map, and handling operation identifier based on pre-judgment confidence level.

[0106] In practice, the data acquisition module receives multimodal data of the object to be detected, and the feature extraction module receives the multimodal data. It performs generative AI trace feature extraction, factual consistency feature extraction, and emotion-inducing feature extraction on the text data, and performs corresponding modal AI trace feature extraction on the image, audio, and video data. If the propagation collaboration detection function has been configured and the content to be detected has propagation behavior data, then the propagation collaboration features are further extracted. The feature extraction module transmits the extracted multidimensional feature set to the anomaly aggregation module. The above multidimensional features are used for in-depth analysis of the second multidimensional precision judgment layer.

[0107] The anomaly aggregation module receives a multi-dimensional set of abnormal features from the feature extraction module, performs comprehensive analysis based on the abnormal combination relationship between multiple detection features, and obtains a cognitive deception anomaly aggregation feature vector. The abnormal combination relationship refers to the collaborative judgment relationship in which multiple detection features are abnormal at the same time, rather than the simple addition of the independent scores of each dimension feature. The anomaly aggregation module transmits the cognitive deception anomaly aggregation feature vector to the anomaly judgment model module.

[0108] The anomaly detection model module receives the anomaly aggregation feature vector from the anomaly aggregation module and inputs it into the cognitive domain deception anomaly detection model. The anomaly detection model includes a rule model, a statistical model, a machine learning model, a neural network model, or a combination thereof. The anomaly detection model module outputs the attack type of the object to be detected (including pure AI generation type, semi-real and semi-fake tampering type, emotion-induced manipulation type, collaborative propagation amplification type, and composite type) and the cognitive deception comprehensive index (CDI) and its corresponding level, and transmits them to the result output module.

[0109] The results output module receives the attack type, Cognitive Deception Index (CDI), and contribution analysis results of each dimension generated internally by the anomaly detection model module, and assembles them into structured detection results. The detection results include the attack type, CDI, anomaly scores of each feature dimension, and evidence chain data supporting the CDI, and are output to external systems or manual review interfaces.

[0110] The embodiments provided by this invention include: acquiring multimodal data, extracting multi-dimensional abnormal features (including at least generative AI trace anomalies, factual anomalies, and emotion-induced anomalies), aggregating anomalies based on anomaly combination relationships, determining the attack type and cognitive deception comprehensive index (CDI) through a judgment model, and outputting detection results containing an anomaly source map. The technical means employed in this method mainly include the following aspects: 1) Multi-dimensional anomaly aggregation processing technology, including: Anomaly combination relationship fusion mechanism: When multiple detection dimensions are abnormal at the same time, the overall anomaly confidence is higher than the linear sum of the independent scores of each dimension; Scenario-adaptive weight dynamic adjustment mechanism: dynamically adjust the weight parameters (contribution) of each dimension feature according to the hit results of sensitive items in the application scenario.

[0111] 2) Semantic block-level weighted fact comparison method: By setting differentiated importance weights q_i for different semantic blocks, the fact credibility deviation S_fact_abn = 100×Σ(q_i ×δ_i) / Σq_i is calculated to identify half-true and half-false attacks where the background information is consistent but the key semantic blocks are tampered with.

[0112] 3) A serial, hierarchical analysis architecture is adopted, including: The first layer is a fast prediction layer (target latency <500ms / text): it performs surface-level abnormal pattern detection on the text, calls a lightweight rule model or statistical model, outputs the prediction confidence P, and performs three-level filtering based on P (high confidence pass, medium confidence pending confirmation, low confidence highly suspicious). The second layer, the multidimensional precision judgment layer, calls upon full-modal anomaly trace detection, multi-granularity fact credibility assessment, emotion-induced pattern recognition, optional propagation collaborative anomaly detection, anomaly aggregation and judgment, and anomaly source map generation for samples above the attention threshold.

[0113] 4) Anomaly source mapping generation technology, including the following steps: Triggering conditions: After the multi-dimensional precision judgment layer detection is completed and the cognitive deception comprehensive index (CDI) is determined, the anomaly tracing map generation module is triggered; Source tracing data collection: Collect anomaly scores for each dimension, intermediate results generated during the detection process (anomaly location, anomaly content fragments, anomaly details), and contribution analysis results for each dimension from the anomaly aggregation module and the anomaly judgment model module respectively; The graph structure assembly is structured according to a five-dimensional structure (graph header, abnormal graph summary, multi-dimensional abnormal tracing record, cognitive comprehensive index derivation path, and handling guidelines). The abnormal tracing record of each dimension includes four elements: abnormal score, abnormal location, abnormal content fragment, and judgment basis. Graph storage and retrieval: Calculate the graph ID and content hash value, establish an index of content hash-graph ID, and store it in the anomaly tracing graph database to support subsequent tracing queries and rapid matching of duplicate detection of the same content.

[0114] 5) An adversarial training strategy for cognitive domain deception features is proposed, including adversarial sample generation, hybrid training, and continuous updating.

[0115] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting cognitive domain spoofing attacks based on generative AI, characterized in that, include: Acquire multimodal data of the object to be detected, including text data, image data, audio data, and video data; The predicted confidence level is calculated based on the text data. The predicted confidence level is obtained by calculating the explicit keyword matching degree, basic scene matching degree, and statistical distribution deviation of the text data. Multi-dimensional feature extraction is performed on multi-modal data with a predicted confidence level lower than the first preset threshold. Anomaly scores corresponding to each dimension feature are calculated, and sensitivity analysis of the application scenario to which the object to be detected belongs is performed to obtain sensitivity matching scores. The anomaly scores corresponding to each dimension feature include generative AI trace anomaly scores, factual consistency anomaly scores, and emotion-induced anomaly scores. Anomalies are aggregated based on the anomaly scores, sensitive item matching scores, and contributions of each score corresponding to the features of each dimension to obtain a cognitive deception comprehensive index. The contribution of each score is dynamically adjusted according to the sensitive item hit results in the application scenario of the object to be detected. The cognitive deception comprehensive index level is determined based on the cognitive deception comprehensive index, and the attack type is determined based on the anomaly scores corresponding to each dimension of features. A structured anomaly attribution map is generated based on the contribution and attack type corresponding to each score. Output cognitive deception comprehensive index, attack type, cognitive deception comprehensive index level, structured anomaly tracing map, and handling operation identifier based on prediction confidence level; The step of extracting multi-dimensional features from multimodal data with a predicted confidence level lower than a first preset threshold includes: A second preset threshold is set for the predicted confidence level, and the second preset threshold is less than the first preset threshold. Multi-dimensional feature extraction is performed on multi-modal data with a predicted confidence level lower than the first preset threshold but greater than or equal to the second preset threshold, and on multi-modal data with a predicted confidence level lower than the second preset threshold. The priority of multi-dimensional feature extraction for multi-modal data with a predicted confidence level lower than the second preset threshold is greater than that for multi-modal data with a predicted confidence level lower than the first preset threshold but greater than or equal to the second preset threshold. The anomaly scores corresponding to each dimension of features also include propagation coordination anomaly scores, the calculation process of which includes: Calculate outliers related to release time synchronization, semantic skeleton similarity, common links, and account association. Based on anomalies related to release time synchronization, semantic skeleton similarity, common links, and account association, a propagation collaboration anomaly score is obtained. The expression for the propagation collaboration anomaly score is: C = 100×max(C_time×C_sem, C_link, C_account), Where C is the score for abnormal propagation collaboration, max is the function to take the maximum value, C_time is the abnormal value for release time synchronization, C_sem is the abnormal value for semantic skeleton similarity, C_link is the abnormal value for common link, and C_account is the abnormal value for account association. The expression for the cognitive deception comprehensive index is: CDI = F(A, B, E, C, S_match, R_combo), Among them, CDI is the cognitive deception comprehensive index, A is the generative AI trace anomaly score, B is the factual consistency anomaly score, E is the emotion-induced anomaly score, S_match is the sensitivity item matching score, R_combo is the anomaly combination relationship function, and F is the non-linear aggregation function. The non-linear aggregation function indicates that when the anomaly scores of two dimensions exceed their respective preset attention thresholds, the combined risk is non-linearly enhanced.

2. The generative AI-based cognitive domain deception attack detection method of claim 1, wherein, The expression for the predicted confidence level is: ， Where P is the prediction confidence level, S_keyword is the explicit keyword matching degree, S_stat is the statistical distribution deviation, S_match_base is the basic scene matching degree, and α, β and γ are the weight coefficients corresponding to the explicit keyword matching degree, statistical distribution deviation and basic scene matching degree, respectively.

3. The generative-AI-based cognitive domain deception attack detection method of claim 1, wherein, The calculation process for the generative AI trace anomaly score includes: Extract text perplexity anomaly features and text statistical anomaly features from the text data of the object to be detected; extract image anomaly features from the image data of the object to be detected; extract audio anomaly features from the audio data of the object to be detected; extract video anomaly features from the video data of the object to be detected. The outliers of text perplexity anomaly features, text statistical anomaly features, image anomaly features, audio anomaly features, and video anomaly features are weighted and aggregated to obtain a generative AI trace anomaly score. The calculation of outliers in the text perplexity anomaly feature includes: The perplexity of the text data to be detected is calculated word by word under the reference language model to obtain the perplexity value sequence of each word position. The perplexity value sequences of all word positions are summarized to obtain the perplexity distribution of the text to be detected. The degree of deviation between the perplexity distribution of the text to be detected and the perplexity distribution of the reference human text is calculated based on the distribution difference metric function. The degree of deviation is used as the outlier value of the text perplexity abnormal feature.

4. The generative AI-based cognitive domain deception attack detection method of claim 1, wherein, The calculation process for the factual consistency anomaly score includes: Extract entity sets and relation triples from the text data of the object to be detected; Based on entity sets and relation triples, a search is performed in a pre-built trusted source library to obtain a set of candidate trusted texts corresponding to the text data of the object to be detected; The text data of the object to be detected and the candidate credible text set are divided into multiple semantic blocks. The fact difference degree is calculated for each semantic block. The fact difference degree is the semantic difference between the semantic block of the text data of the object to be detected and the corresponding semantic block of the candidate credible text set. Based on the fact difference degree of each semantic block, the fact credibility deviation degree is calculated. Cross-modal semantic consistency detection is performed on the image data and video data of the object to be detected and the corresponding text data to obtain cross-modal consistency scores. Cross-modal anomaly scores are then calculated based on the cross-modal consistency scores. We fused the factual credibility deviation and cross-modal anomaly scores with weights to obtain the factual consistency anomaly score.

5. The generative AI-based cognitive domain deception attack detection method of claim 4, wherein, The calculation process for the emotion-induced abnormality score includes: The comprehensive sentiment score of the text data of the object to be detected is calculated by using a sentiment analysis model. Based on the comprehensive sentiment score and the baseline sentiment range corresponding to the content type, polarity outliers are calculated. Construct an emotion manipulation keyword library containing words of urgency, words of extreme emotions, and words of repeated inducement. Calculate the word frequency density of words that hit the emotion manipulation keyword library in the text data of the target object. At the same time, detect the number of times the same keyword appears repeatedly within a preset distance. Calculate the outlier value of the emotion manipulation pattern based on the word frequency density and the number of times the same keyword appears repeatedly within the preset distance. Construct a call-to-action phrase library, count the frequency of occurrence of call-to-action phrases in the text data of the target object, and detect sentence structure features. Based on the frequency of occurrence of call-to-action phrases in the library and sentence structure features, calculate call-to-action outliers. Normalization and maximum value aggregation were performed on polarity outliers, emotion manipulation pattern outliers, and call to action outliers to obtain an emotion-induced anomaly score.

6. The generative AI-based cognitive domain deception attack detection method of claim 5, wherein, The attack type determination based on the anomaly scores corresponding to each dimension of features includes: If the abnormal score of generative AI traces is ≥ 60, the abnormal score of factual consistency is < 40, and the abnormal score of emotion induction is < 40, then the attack type is determined to be pure AI generation. If the factual consistency anomaly score is ≥60 and the factual difference degree of the semantic block is ≥0.7, then the attack type is determined to be a half-true, half-false tampering type. If the emotional inducement anomaly score is ≥55, or any one of the polarity anomaly, emotional manipulation pattern anomaly, or call to action anomaly is ≥70, then the attack type is determined to be emotional inducement manipulation. If the anomaly scores corresponding to multiple dimensional features simultaneously exceed the corresponding attention thresholds, the attack type is determined to be composite.

7. A generative AI based cognitive domain deception attack detection system for implementing the generative AI based cognitive domain deception attack detection method according to any one of claims 1-6, characterized in that, The system includes: The data acquisition module is used to acquire multimodal data of the object to be detected, including text data, image data, audio data, and video data. The rapid prediction module is used to calculate the prediction confidence based on text data. The prediction confidence is calculated based on the explicit keyword matching degree, basic scene matching degree, and statistical distribution deviation of the text data. The feature extraction module is used to extract multi-dimensional features from multimodal data with a predicted confidence level lower than the first preset threshold, calculate the anomaly score corresponding to each dimension feature, and perform sensitivity analysis on the application scenario to which the object to be detected belongs, and obtain the sensitivity matching score. The anomaly scores corresponding to each dimension feature include generative AI trace anomaly score, factual consistency anomaly score, and emotion-induced anomaly score. The anomaly aggregation and judgment module is used to aggregate anomalies based on the anomaly scores, sensitive item matching scores, and contribution values ​​of each score corresponding to the features of each dimension, to obtain a cognitive deception comprehensive index, and dynamically adjust the contribution values ​​of each score according to the sensitive item hit results in the application scenario to which the object to be detected belongs; determine the cognitive deception comprehensive index level based on the cognitive deception comprehensive index, and determine the attack type based on the anomaly scores corresponding to each dimension feature. The anomaly attribution graph generation module is used to generate a structured anomaly attribution graph based on the contribution of each score and the attack type. The results output module is used to output the cognitive deception comprehensive index, attack type, cognitive deception comprehensive index level, structured anomaly tracing map, and handling operation identifier based on pre-judgment confidence level.

Images