Risk event anomaly detection method and system based on double anti-fact decoding

Abstract

The invention relates to a risk event anomaly detection method and system based on double anti-fact decoding, and relates to the technical field of artificial intelligence safety and anomaly detection.The method comprises the steps that firstly, input data to be detected are obtained, and key evidence for judging whether a risk event happens or not is recognized through visual text association; secondly, masking or converting the key evidence to generate a corresponding visual text anti-fact sample pair; thirdly, obtaining the large model, and carrying out the reasoning of the original input data and the double anti-fact sample, and obtaining an original logits and an anti-fact logits; when decoding and making a final determination, outputs that are not based on key evidence, but based on model prejudice or priori knowledge, are dynamically punished by "subtracting" the anti-fact logits from the original logits. According to the method, the capability of accurately detecting which real risk events exist by the model can be remarkably improved, so that more reliable technical support is provided for risk prevention and control in the real world.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence security and anomaly detection technology, specifically relating to a risk event anomaly detection method and system based on double anti-fact decoding. Background Technology

[0002] Anomaly detection of risk events is a critical task in fields such as public safety, emergency response, industrial production, and autonomous driving. Its core objective is to automatically and accurately identify low-probability, high-impact anomalies such as "natural disasters" (e.g., earthquakes, fires), "public safety incidents" (e.g., shootings, traffic accidents), or "industrial anomalies" (e.g., equipment failures, leaks) from massive, multimodal data streams (e.g., text reports, images, surveillance videos). Traditional risk event detection methods primarily rely on manually designed rule bases, keyword templates, or traditional machine learning models. These methods have significant drawbacks: first, they lack flexibility and adaptability; manually designed rules struggle to cover all complex, ambiguous, or non-standard event descriptions, resulting in weak generalization ability; second, they are inefficient; traditional models struggle to adapt quickly to novel or unseen risk events, leading to high false negative rates. In recent years, with the rapid development of large-scale visual-language models (LVLMs), their powerful contextual understanding and cross-modal reasoning capabilities have brought new possibilities to risk event detection. Theoretically, these models can understand the semantics of events more deeply, rather than simply matching surface features. However, despite significant progress made by large-scale models on multiple tasks, they remain severely hampered by the "hallucination" problem in practical applications. Hallucination refers to a model generating content that appears credible but is inconsistent with reality or lacks supporting input evidence. This hallucination problem is particularly prominent in risk event anomaly detection tasks, manifesting itself primarily in two aspects: (1) False positives: The model relies too heavily on "linguistic priors" or statistical biases in the training data. For example, when the input text report contains "this team"... Full power (on fire) or "This decision is fatal one hit When using metaphorical words such as "(shot)," the model may "illusory" the risk of a real "fire" or "shooting incident" based solely on linguistic priors, even though there is no real threat in the context.

[0003] (2) False Negative: The model suffers from "attention bias" or "lack of visual attention," ignoring key visual evidence. For example, in a surveillance video, the model may focus on pedestrians or vehicles in the foreground while ignoring key risk signals such as "abnormal smoke" or "ice on the ground" in a small area in the background, thus failing to identify real fire or traffic risks.

[0004] In high-stakes applications such as risk event detection (e.g., emergency response, medical diagnosis, or autonomous driving), false alarms caused by hallucinations can waste valuable emergency resources, while missed alarms can lead to catastrophic consequences.

[0005] Currently, methods for alleviating model hallucinations mainly fall into two categories: (1) Training phase methods: These methods typically require fine-tuning on large-scale datasets, using reinforcement learning (RLHF), or constructing specialized negative sample datasets. While effective, these methods are usually computationally expensive, resource-intensive, and require extensive manual annotation, making them difficult to deploy and scale quickly.

[0006] (2) Inference-stage methods: These methods intervene without modifying the model parameters, such as by modifying the input (e.g., adding noise) or using a conservative decoding strategy. While these methods are computationally efficient, they may not be able to fundamentally solve the problem and may even "disrupt cross-modal alignment" or "fail to address the root cause of the illusion." Summary of the Invention

[0007] To address the aforementioned problems, this invention proposes a risk event anomaly detection method and system based on double counterfactual decoding. This invention dynamically constructs "double counterfactual samples" (i.e., visual counterfactual samples and textual counterfactual samples) that are opposite to key evidence (visual regions or textual entities) during inference, and utilizes a novel contrastive decoding mechanism to calibrate the model's original output and counterfactual output in real time. This method forces the model to rely on real input evidence (rather than linguistic priors or attention biases) to generate high-confidence judgments. This invention requires no additional model training or fine-tuning, providing a plug-and-play framework that fundamentally alleviates the problems of "false positives" (e.g., misjudging a metaphor as a fire) and "false negatives" (e.g., ignoring key visual signals) caused by model illusions. It offers an efficient, robust, and reliable technical solution for the application of large models in high-risk fields such as public safety and emergency response.

[0008] To alleviate the illusions that large visual-language models (LVLMs) may exhibit when performing risk event anomaly detection tasks, this invention proposes a risk event anomaly detection method based on double antifactual decoding. A risk event anomaly detection method based on dual-fact decoding includes the following steps: Data acquisition: Acquire the input data to be detected, including raw image data and raw text data; Key target region localization: Perform visual-text semantic association localization on the original image data and original text data to identify key evidence for determining risk events. The key evidence includes: key visual regions that are strongly semantically associated with the text and key text entities that are strongly semantically associated with the image. Constructing counterfactual sample pairs: Based on the key evidence, counterfactual sample pairs are generated, including: visual counterfactual samples generated by occluding the key visual regions; and textual counterfactual samples generated by replacing the key text entities. Large model inference: Obtain the raw logits of a large model for a candidate event under the original input data conditions; Obtain the counterfactual logits output by the large model for the same candidate event under the conditions of the visual counterfactual samples and the text counterfactual samples, including: the visual counterfactual logits output under the conditions of the visual counterfactual samples; and the text counterfactual logits output under the conditions of the text counterfactual samples. Comparison Decoding and Risk Judgment: By performing comparison decoding operations on the original logits, the visual counterfactual logits, and the text counterfactual logits, the calibrated final logits are obtained. Based on the obtained calibrated final logits, the final judgment result of the risk event anomaly detection is generated.

[0009] As a further aspect of the present invention, the visual-text semantic association localization method is as follows: Using a bias-free small-scale VQA model, the gradient contribution value of each visual object and text entity in the original input data to the predicted answer is calculated. The top K visual objects and text entities with the highest contribution values ​​are selected as key target regions. The gradient contribution value is calculated using the following formula: in, The probability of predicting the answer for the small-scale VQA model. for visual features gradient, for visual features gradient, It is a vector of all ones. Visual features gradient contribution value, Visual features The gradient contribution value.

[0010] As a further aspect of the present invention, the visual counterfactual samples generated in the construction of counterfactual sample pairs are obtained by applying zero-fill or mean-fill masking to the key visual regions.

[0011] As a further aspect of the present invention, the text counterfactual samples generated in the construction of counterfactual sample pairs are obtained by replacing the embeddings corresponding to the key text entities with the embeddings of a special tag [CLS] at the word embedding level.

[0012] As a further aspect of the present invention, the comparison decoding operation is implemented using the following double-reverse fact decoding formula: in, and These are hyperparameters used to control the intensity of visual and textual counterfactual penalties, respectively. This indicates the current generation step. candidate word groups The final logarithmic probability after double-fact calibration. This indicates that the large model is based on the original input data and performs a process for the current candidate word units. The output is the raw log probability. The large model is based on visual counterfactual samples for the candidate nouns. The output visual counterfactual log probability, The large model is based on the counterfactual samples of the text, targeting the candidate nouns. The output text counterfactual log probability.

[0013] Furthermore, the present invention also provides a risk event anomaly detection system based on dual-fact decoding, comprising: The data acquisition module is used to acquire the input data to be detected, which includes raw image data and raw text data. The key target area localization module is used to perform visual-text semantic association localization on the raw image data and raw text data to identify key evidence for determining risk events. Key evidence includes: key visual areas that are strongly semantically associated with the text; and key text entities that are strongly semantically associated with the image. The dual counterfactual sample generation module is used to generate counterfactual sample pairs based on key evidence. The sample pairs include: visual counterfactual samples generated by occluding key visual regions; and textual counterfactual samples generated by replacing key text entities. The large model inference module is used to obtain the original logits output by the large model for a candidate event under the original input data conditions; and to obtain the counterfactual pair logits output by the large model for the same candidate event under the condition of two counterfactual sample pairs. The counterfactual pair logits include: visual counterfactual logits output under the condition of visual counterfactual samples, and textual counterfactual logits output under the condition of textual counterfactual samples. The dual counterfact decoding and judgment module is used to perform comparative decoding operations on the original logits, visual counterfact logits, and text counterfact logits to obtain the calibrated final logits. This is used to simultaneously punish and alleviate the illusions generated by language priors and visual attention biases in the model. Based on the obtained calibrated final logits, the final judgment result of risk event anomaly detection is generated.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention does not require any additional model training or fine-tuning, and is a plug-and-play framework that can fundamentally alleviate the problems of "false alarms" (such as misjudging a metaphor as a fire) and "false negatives" (such as ignoring key visual signals) caused by model illusions. It provides an efficient, robust and reliable technical solution for the application of large models in high-risk fields such as public safety and emergency response. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention; Detailed Implementation The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] like Figure 1 As shown, the present invention proposes a risk event anomaly detection method based on dual counterfactual decoding. As a plug-and-play inference framework that requires no training, its core idea is to construct high-precision counterfactual sample pairs in both visual and textual dimensions, and to punish "illusion" signals caused by language priors or attention biases in real time by comparing the logits output of the original sample and the counterfactual sample pairs during decoding, thereby forcing the model's final judgment to be strictly based on the input.

[0017] The implementation method includes five steps: data acquisition, key target area location, construction of counterfactual sample pairs, large model inference, comparative decoding, and risk assessment. The detailed implementation method for each step is as follows: A risk event anomaly detection method based on dual-fact decoding includes the following steps: (1) Step 1: Data Acquisition The input data to be detected is obtained, which includes on-site images and text descriptions of events that may contain risk events.

[0018] (2) Step Two: Locating Key Target Areas This step performs visual-text semantic association to accurately identify and locate the most critical semantic evidence connecting the image and text, i.e., key target regions. The output of this step is a "double anti-fact anchor," specifically comprising: key visual regions strongly correlated semantically with the text; and key text entities strongly correlated semantically with the image. Through cross-modal feature fusion, text phrases are dynamically aligned with specific bounding boxes in the image and filtered using a confidence threshold to obtain highly reliable anchors.

[0019] (3) Step 3: Construct counterfactual sample pairs Based on the "double counterfactual anchor points" identified in step two, construct counterfactual sample pairs: a: Visual counterfactual sample: This sample answers the question, "What if this object wasn't in the image?". It is generated by occluding the key visual region.

[0020] b: Textual counterfactual sample: This sample answers the question, "What if this entity wasn't mentioned in the question?". It is generated by replacing the key textual entity.

[0021] (4) Step 4: Large Model Reasoning The original input data of the risk events to be detected and the counterfactual sample pairs are respectively input into a large visual language model (LVLM). When decoding and generating each token, three sets of log probability distributions are calculated in parallel: The original logits are the model's outputs given the original risk events to be detected as input. This indicates that the large model is based on the original input data and performs a process for the current candidate word units. The output is the raw log probability. The parameter is The probability distribution function of a large visual-language model (LVLM). Represents the original image data. Represents the original text data. This indicates that the model is at the current time step. The previously generated word sequence.

[0022] Visual counterfactual logits, which are the model's outputs under visual counterfactual samples of the risk events to be detected: The large model represents the current generation step based on visual counterfactual samples and the original text data. candidate word groups The output visual counterfactual log probability, The parameter is The probability distribution function of a large visual-language model (LVLM). Indicates visual counterfactual samples, Represents the original text data. This indicates that the model is at the current time step. The previously generated word sequence.

[0023] Textual counterfactual logits, which are the model's outputs on textual counterfactual samples of the risk event to be detected: The large model represents the current generation step based on the original image data and text counterfactual samples. candidate word groups The counterfactual log probability of the output text. The parameter is The probability distribution function of a large visual-language model (LVLM). Represents the original image data. This represents a counterfactual sample of text. This indicates that the model is at the current time step. The previously generated word sequence.

[0024] (5) Step 5: Compare decoding and risk assessment The calibrated final logits are calculated using a novel decoding formula. The core mechanism of this operation is to force the model to make judgments strictly based on evidence by subtracting (penalizing) those "illusory" logits that still maintain a high probability in the counterfactual world (i.e., when key evidence is lacking) from the original logits. This comparative decoding operation is implemented using the following double counterfactual decoding formula: in, and is a hyperparameter used to control the intensity of visual and textual counterfactual penalties, respectively. This indicates the current generation step. candidate word groups The final logarithmic probability after double-fact calibration. This indicates that the large model is based on the original input data and performs a process for the current candidate word units. The output is the raw log probability. The large model is based on visual counterfactual samples, and it evaluates the current candidate nouns. The output visual counterfactual log probability, The large model is based on the counterfactual samples of the text, targeting the candidate nouns. The output text is the counterfactual log probability. Finally, based on the calibrated final log probabilities (logits), the token with the highest probability is selected. The process then repeats steps four and five until a complete final judgment result regarding the anomaly detection of the risk event is generated.

[0025] The method of the present invention will be illustrated below through an example.

[0026] (1) Step 1: Data Acquisition This step marks the beginning of the method, where the system acquires the input data to be detected. The input data is a multimodal data pair, including raw image data and raw text data. For example, in a risk event detection scenario, the raw image data could be a surveillance screenshot or a photograph of the scene, and the raw text data could be a corresponding text description or a prompt used to query risks, such as "Is there a fire risk in the image?" or "Analyze any abnormalities at the scene."

[0027] (2) Step Two: Locating Key Target Areas The purpose of this step is to perform visual-text semantic association localization, the core of which is cross-modal evidence localization. One of the theoretical foundations is that illusions stem from the model's "attention bias" towards irrelevant visual regions or linguistic priors detached from visual context. Therefore, in order to construct counterfactual samples that can effectively calibrate these biases, it is essential to first locate the most critical semantic evidence connecting the image and text with high precision, namely the "counterfactual anchors."

[0028] This step is achieved using an unbiased small model or a similar cross-modal model. An unbiased small model can align any text phrase in the original text data with a specific visual region in the original image data with high precision through cross-modal feature fusion.

[0029] The positioning process includes: 1. Feature Extraction and Preliminary Prediction: First, feature extraction is performed on the original image data. Features of a visual object And extract from the original text data Text entity features The original image data and original text data are input into the bias-reduced small-scale VQA model to obtain a preliminary probability distribution of predicted answers based on this small model. .

[0030] 2. Calculate gradient contribution values: To identify which input features contribute the most to the predicted answer, this step calculates... The gradient relative to each visual object feature and each text entity feature.

[0031] 3. Locating key visual regions: The gradient contribution value of the visual object is determined by the following formula: in, The probability of predicting the answer for the small-scale VQA model. For this probability, the first... Gradient of a visual feature It is a vector of all ones. Quantified the first The "influence" of a visual object on making that judgment. 4. Locate key text entities: Corresponding to the visual region location, this step calculates... Gradient contribution value relative to each text entity feature It is calculated using the following formula: in, The probability of predicting the answer for the small-scale VQA model. For this probability, the first... Text features gradient, It is a vector of all ones. Quantified the first The "influence" of a text entity on making that judgment.

[0032] 5. Filter key target areas: Finally, for the calculated... and The contribution values ​​are sorted, and the top K visual objects and text entities are selected as the key visual regions for constructing counterfactual samples. and key text entities .

[0033] (3) Step 3: Construct counterfactual sample pairs This step generates counterfactual sample pairs based on the key evidence (the top K visual regions and text entities) identified in step two.

[0034] a. Visual counterfactual samples Generation: By analyzing the key visual regions This is achieved using masking operations. In one embodiment, a zero-padding or mean-padding strategy is employed. This refers to the visual counterfactual sample. This represents the occlusion operation function. Represents the original image data. Represents the set of elements. One key visual region was identified as having a strong semantic association with the original text data. This represents the set of key evidence indexes that have been filtered.

[0035] b. Textual counterfactual samples Generation: By analyzing the key text entities This is achieved through replacement or masking. In a preferred embodiment, to achieve high computational efficiency, the operation is performed at the word embedding level, where the key text entities are... The corresponding embedding is replaced with a word embedding of a uniform special tag (such as [CLS]).

[0036] This represents a counterfactual sample of text. This represents the occlusion operation function. Represents the original text data. Represents the set of elements. One key text entity was identified as having a strong semantic association with the original image data. This represents the set of key evidence indexes that have been filtered.

[0037] (4) Step Four: Large Model Reasoning This step is a preparatory step for comparative decoding. The Large LVLM model automatically and regressively generates each step of the judgment regarding the risk event, with the system computing three different log-probability distributions in parallel: a. Original logarithmic probability That is, the model generates under the original input conditions. 'logits'.

[0038] This indicates that the large model is based on the original input data and performs a process for the current candidate word units. The output is the raw log probability. The parameter is The probability distribution function of a large visual-language model (LVLM). Represents the original image data. Represents the original text data. This indicates that the model is at the current time step. The previously generated word sequence.

[0039] b. Visual counterfactual log probability That is, the model generates the same [model] under the input condition of visual counterfactual samples. 'logits'.

[0040] The large model represents the current generation step based on visual counterfactual samples and the original text data. candidate word groups The output visual counterfactual log probability, The parameter is The probability distribution function of a large visual-language model (LVLM). Indicates visual counterfactual samples, Represents the original text data. This indicates that the model is at the current time step. The previously generated word sequence.

[0041] c. Logarithmic probability of the text counterfactual That is, the model generates the same text under the input condition of counterfactual text samples. 'logits'.

[0042] The large model represents the current generation step based on the original image data and text counterfactual samples. candidate word groups The counterfactual log probability of the output text. The parameter is The probability distribution function of a large visual-language model (LVLM). Represents the original image data. This represents a counterfactual sample of text. This indicates that the model is at the current time step. The previously generated word sequence.

[0043] (5) Step Five: Compare Decoding and Risk Assessment By subtracting (penalizing) two counterfactual logits from the original logits, the probability of truly cross-modal key evidence is "purified".

[0044] in, and is a hyperparameter used to control the intensity of visual and textual counterfactual penalties, respectively. This indicates the current generation step. candidate word groups The final logarithmic probability after double-fact calibration. This indicates that the large model is based on the original input data and performs a process for the current candidate word units. The output is the raw log probability. The large model is based on visual counterfactual samples, and it evaluates the current candidate nouns. The output visual counterfactual log probability, The large model is based on the counterfactual samples of the text, targeting the candidate nouns. The output text counterfactual log probability.

[0045] Finally, the system uses the calibrated final log probability. Select the next token and append it to the output sequence. middle.

[0046] The system iterates through steps four and five until the model generates a complete judgment result (e.g., a complete sentence "Based on the image, no fire risk was found", or a special token [EOS] representing a risk event), thus completing a high-confidence risk event anomaly detection.

[0047] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0048] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for risk event anomaly detection based on double counterfactual decoding, characterized in that, The method comprises the following steps: data acquisition: acquiring input data to be detected, the input data comprising original image data and original text data; key target area positioning: performing visual-text semantic association positioning on the original image data and original text data to identify key evidence for determining a risk event, the key evidence comprising: a key visual area that is strongly associated with text in semantics, and a key text entity that is strongly associated with an image in semantics; constructing a pair of counterfactual samples: generating a pair of counterfactual samples based on the key evidence, the pair of samples comprising: a visual counterfactual sample generated by masking the key visual area, and a text counterfactual sample generated by replacing the key text entity; large model inference: acquiring original logits output by a large model for a candidate event under the condition of original input data; acquiring counterfactual logits output by the large model for the same candidate event under the condition of the visual counterfactual sample and the text counterfactual sample, the counterfactual logits comprising: visual counterfactual logits output under the condition of the visual counterfactual sample, and text counterfactual logits output under the condition of the text counterfactual sample; comparative decoding and risk judgment: obtaining calibrated final logits by performing comparative decoding operation on the original logits, the visual counterfactual logits, and the text counterfactual logits, and generating a final judgment result of the risk event anomaly detection based on the obtained calibrated final logits.

2. The method of claim 1, wherein, The method of visual-text semantic association positioning is as follows: calculating gradient contribution values of each visual object and text entity in the original input data to a predicted answer by a debiased small-scale VQA model, and selecting visual objects and text entities with top K contribution values as key target areas, the gradient contribution values being calculated by the following formula: in, The probability of predicting the answer for the small-scale VQA model. for visual features gradient, for visual features gradient, It is a vector of all ones. Visual features gradient contribution value, Visual features The gradient contribution value.

3. The method of claim 1, wherein, The visual counterfactual sample generated in the construction of the pair of counterfactual samples is obtained by applying zero padding or mean padding masking to the key visual area.

4. The method of claim 1, wherein, The text counterfactual sample generated in the construction of the pair of counterfactual samples is obtained by replacing an embedding corresponding to the key text entity with an embedding of a special mark [CLS] at a word embedding level.

5. The method of claim 1, wherein, The comparative decoding operation is realized by the following double counterfactual decoding formula: wherein, and are hyperparameters for controlling the strength of visual and textual counterfactual penalties, respectively, represents the final log probability of a candidate token at the current generation step after double counterfactual calibration, represents the original log probability output by the large model based on the original input data for the current candidate token , represents the visual counterfactual log probability output by the large model based on the visual counterfactual sample for the candidate token , represents the textual counterfactual log probability output by the large model based on the textual counterfactual sample for the candidate token .

6. A risk event anomaly detection system based on double counterfactual decoding, characterized in that, The method comprises the following steps: a data acquisition module configured to acquire input data to be detected, the input data comprising original image data and original text data; a key target area positioning module configured to perform visual-text semantic association positioning on the original image data and original text data to identify key evidence for determining a risk event, the key evidence comprising: a key visual area that is strongly associated with text in semantics, and a key text entity that is strongly associated with an image in semantics; a double counterfactual sample generation module configured to generate a pair of counterfactual samples based on the key evidence, the pair of samples comprising: a visual counterfactual sample generated by masking the key visual area, and a text counterfactual sample generated by replacing the key text entity; The large model inference module is configured to obtain original logits output by the large model for a candidate event under a condition of original input data, and obtain counterfactual pair logits output by the large model for the same candidate event under a condition of a counterfactual sample pair, the counterfactual pair logits including visual counterfactual logits output under a condition of a visual counterfactual sample and text counterfactual logits output under a condition of a text counterfactual sample; The counterfactual decoding and determination module is configured to perform comparative decoding operation on the original logits, the visual counterfactual logits and the text counterfactual logits to obtain calibrated final logits, so as to punish and relieve hallucinations caused by language prior and visual attention bias deviation in the model, and generate a final determination result of risk event anomaly detection based on the calibrated final logits.

Images