A dual-process cognition-based open world road anomaly segmentation method and system

Abstract

This invention discloses an open-world road anomaly segmentation method and system based on dual-process cognition. The method includes acquiring an image of a road scene to be segmented, generating an initial anomaly response prior map and semantic context description; obtaining two candidate anomaly target sets through reactive saliency extraction and deliberate logical reasoning respectively; performing cross-source consistency arbitration on the two sets, and obtaining clean prompt words through semantic cue decoupling; obtaining an anomaly instance mask through cascaded detection and segmentation based on the clean prompt words, and generating a final anomaly segmentation heatmap through fusion modulation. This invention, through a dual-process cognitive collaborative mechanism, integrates reactive perception and deliberate reasoning, significantly reducing the false positive rate without sacrificing recall; suppressing background leakage at the source through semantic cue decoupling, ensuring that the segmentation mask boundary accurately fits the target contour; and outputting an interpretable explicit adjudication evidence chain, improving the safety and interpretability of the autonomous driving perception system.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and autonomous driving environment perception technology, specifically involving an open-world road anomaly segmentation method and system based on dual-process cognition. Background Technology

[0002] In the safe deployment of autonomous vehicles in open-world environments, highly robust anomaly segmentation and environmental anomaly perception technologies are the core foundation for the safe and reliable operation of autonomous driving systems. This task requires autonomous driving perception systems to accurately identify and segment out-of-domain objects or abnormal obstacles that deviate significantly from the distribution of known training data. However, existing anomaly segmentation and anomaly perception technologies generally have inherent limitations, making it difficult to achieve an effective balance between low-level visual perception sensitivity and high-level semantic cognitive reasoning. Existing anomaly detection and segmentation technology chains typically focus on a single technical dimension in their design and implementation, failing to simultaneously consider pixel-level perception accuracy and scene-level semantic understanding capabilities. This results in existing technologies generally exhibiting insufficient generalization ability, poor adaptability, and high practical application safety risks when facing complex road scenarios in open worlds.

[0003] In existing technologies, a mainstream anomaly detection scheme heavily relies on low-level visual features for anomaly identification. This type of scheme typically defines anomaly targets as statistical distribution deviations relative to the feature manifold of the training data, primarily achieving anomaly detection through uncertainty estimation of local features or comparison of image reconstruction errors. While this type of low-level visual response-based perception method boasts fast detection response speeds, it lacks semantic cognitive capabilities due to the absence of high-level global semantic information and scene contextual constraints. Without global scene constraints, the model becomes overly sensitive to local pixel gradient changes and local texture features, easily misidentifying high-frequency normal visual features in road scenes as obstacles, such as tree shadows, water reflections on the road surface, and road surface joints. This leads to numerous perception artifacts and false positive alarms, affecting the normal traffic control decisions of autonomous vehicles. Furthermore, for dangerous targets whose statistical visual features are similar to those in the training samples but whose actual spatial locations are abnormal, such as unconventional obstacles like tires fallen from the road surface, this type of method is prone to feature confusion and target masking problems, resulting in missed detections and segmentations. Furthermore, this type of method cannot effectively handle complex road scenarios with high contextual ambiguity, and it is difficult to accurately distinguish road scenario elements with similar visual features but completely different safety attributes. For example, it is difficult to accurately distinguish and identify safe rock and soil embankments on both sides of the road from dangerous falling rocks and debris on the road surface. The actual perception reliability is difficult to meet the requirements for safe operation of autonomous driving.

[0004] To overcome the inherent shortcomings of low-level visual perception solutions, some existing technical solutions introduce large-parameter basic model architectures such as large visual language models, achieving target-level semantic understanding through high-level semantic reasoning to compensate for the lack of semantic cognition in pure visual perception. However, these technical solutions, which rely entirely on high-level semantic reasoning, lack the ability to model pixel-level dense visual geometric constraints and precise spatial positions. When directly applied to pixel-level high-precision anomaly segmentation tasks, they are prone to spatial positioning deviations and semantic illusions, and the segmentation and positioning accuracy cannot meet the safety requirements of autonomous driving perception. More critically, existing visual language basic models generally suffer from the inherent structural defects of the bag-of-words model, making it difficult to effectively decouple the target entity attribute description from the target ontology features, which easily leads to semantic attention overflow problems. For example, when the system performs anomaly detection and segmentation tasks based on the semantic instruction of "dangerous debris on the road surface," the model's attention mechanism is prone to large-scale overflow activation, over-activating large areas of the road background, which weakens and inhibits the accurate focusing and fine segmentation of real dangerous anomalies, seriously reducing the segmentation accuracy and detection effectiveness of anomalies. Summary of the Invention

[0005] To address the problems existing in the background technology, this invention provides an open-world road anomaly segmentation method and system based on dual-process cognition. This solves the technical problems of existing open-world anomaly segmentation methods being susceptible to visual artifact interference and having spatial illusions in dense perception due to large visual language models, thus making it difficult to simultaneously achieve high recall, high positioning accuracy, and high interpretability.

[0006] This invention aims to construct a full-process cognitive perception computing pipeline that integrates metacognitive arbitration and semantic prompting decoupling mechanisms by simulating the dual collaborative mechanism of reactive perception and deliberate reasoning in the human cognitive system. This effectively filters visual artifacts and significantly suppresses the illusion problem in pixel-level application scenarios of large visual language models, achieving abnormal target segmentation with high recall, high localization accuracy and high interpretability.

[0007] The technical solution adopted in this invention is: I. An Open-World Road Anomaly Segmentation Method Based on Dual-Process Cognition: S1. Obtain the image to be segmented in the road scene. The data is then input into a discriminative anomaly detection model or a generative anomaly detection model for processing to obtain an initial anomaly response prior map. .

[0008] S2. Segment the image to be anomaly. The image to be segmented is obtained by inputting the preset first prompt word into a visual language model. Semantic context description .

[0009] S3. Prior diagram of initial abnormal response Reaction saliency extraction was performed to obtain the first candidate anomaly target set. .

[0010] S4. Describe according to semantic context Treating abnormal segmentation images By conducting careful logical reasoning, a second set of candidate abnormal targets is obtained. .

[0011] S5. For the first candidate abnormal target set Second candidate anomaly target set Perform cross-source consistency arbitration to obtain the total set of abnormal targets.

[0012] S6. Perform semantic decoupling on each anomalous target in the total anomalous target set to obtain clean prompt words. .

[0013] S7. Based on the pure prompt words Treating abnormal segmentation images Perform cascaded detection and segmentation to obtain anomaly instance masks. .

[0014] S8. Based on the exception instance mask Prior graph of initial abnormal response Adaptive fusion modulation is performed to obtain the final anomaly segmentation heatmap. .

[0015] The first prompt word is used to guide the visual language model from the image to be segmented. Extracting road scene description, road element list, and potential point of interest information; the semantic context description It is structured data that includes road scene descriptions, a list of road elements, and information on potential points of interest.

[0016] Step S3 specifically involves: processing the initial anomaly response prior map. Perform salient region feature extraction to select the initial abnormal response prior map. Regions with median values ​​higher than a preset threshold are designated as highly significant regions. Each highly significant region is then selected as a candidate anomaly target region, forming the first candidate anomaly target set. .

[0017] Step S4 specifically involves: describing the semantic context. Image to be segmented due to anomalies The second, pre-set prompt word is input into the visual language model, where scene analysis, multi-dimensional anomaly deviation mining, and saliency selection are performed sequentially, outputting a set of second candidate anomaly targets. .

[0018] The second cue word is used to guide the visual language model to perform the following processing: D1. Describe in conjunction with semantic context Image to be segmented from anomaly Infer and summarize the expected normal road scenarios.

[0019] D2. Segmenting the image from the anomaly to be identified All candidate abnormal targets are marked, and the deviation of each candidate abnormal target is evaluated from three dimensions: appearance mismatch, semantic inconsistency, and spatial irregularity. The name of each candidate abnormal target, the deviation evaluation result of each dimension, and the corresponding deviation level are obtained.

[0020] D3. Quantify the deviation levels of each dimension of the candidate anomaly targets into deviation scores. Weight the deviation scores and confidence levels of all dimensions to obtain a deviation evaluation value. Sort the candidate anomaly targets in descending order according to the deviation evaluation values, and select the first preset number of candidate anomaly targets. Generate corresponding bounding boxes and confidence levels for the selected candidate anomaly targets. Integrate the deviation evaluation results of each dimension into a description. The names, descriptions, bounding boxes, and confidence levels of the selected candidate anomaly targets constitute the second candidate anomaly target set. .

[0021] The appearance mismatch is used to evaluate whether the texture, shape, color, or material state of the candidate anomalous target does not match the normal road surface; the semantic inconsistency is used to evaluate whether the category of the candidate anomalous target does not match the prior knowledge of the road scene; the spatial irregularity is used to evaluate whether the position, size, orientation, or occupancy state of the candidate anomalous target relative to the drivable area is abnormal.

[0022] Step S5 specifically involves: S51. For candidate abnormal targets that appear in two candidate abnormal target sets at the same time, the dual-source consistency is determined by spatial intersection-union ratio calculation, and the candidate abnormal targets with intersection-union ratio greater than a preset threshold are determined as dual-source consistent abnormal targets.

[0023] S52. After removing the dual-source consistent abnormal targets from the two sets, perform single-source authenticity verification on the remaining candidate abnormal targets.

[0024] S53, For the first candidate abnormal target set The remaining candidate anomalous targets undergo spatial authenticity verification to check whether there are real physical obstacles in the area corresponding to the candidate anomalous targets. Candidate anomalous targets that have real physical obstacles in the area are retained.

[0025] S54, For the second set of candidate abnormal targets The remaining candidate abnormal targets are subjected to semantic authenticity verification to determine whether they are real abnormal targets or false alarms. Candidate abnormal targets corresponding to those determined to be real abnormal targets are retained.

[0026] S55. Merge all candidate abnormal targets retained in the dual-source consistency determination and single-source authenticity verification to obtain a total abnormal target set; the total abnormal target set includes the name of each abnormal target.

[0027] Step S6 specifically involves: The names of each anomalous target in the overall anomalous target set are subjected to text normalization, spatial phrase stripping, environmental phrase stripping, lexical filtering, and attribute extraction to obtain clean prompt words centered on the anomalous target entity and free of background environmental words. .

[0028] Step S7 specifically involves: S71. Segment the image to be anomaly. and all clean prompt words They are input together into an open vocabulary detector for processing, resulting in a set of candidate bounding boxes for each anomalous target.

[0029] S72. Using the candidate bounding box set as geometric constraints, a zero-shot segmentation model is employed for the image to be segmented due to anomalies. Perform image segmentation to obtain anomaly instance masks. .

[0030] The adaptive fusion modulation in step S8 is set according to the following formula: ; ; in, This is the final anomaly segmentation heatmap; This is the prior graph of the initial abnormal response; For Hadamard product; For abnormal instance masks; The attenuation coefficient for non-target regions; To enhance the gain coefficient; For response domain energy modulation parameters; To respond to the lower limit value.

[0031] II. An Open-World Road Anomaly Segmentation System Based on Dual-Process Cognition: The image acquisition module and the underlying anomaly prior generation module acquire images of the road scene to be segmented. The data is then input into a discriminative anomaly detection model or a generative anomaly detection model for processing to obtain an initial anomaly response prior map. .

[0032] The global scene perception module segments the image to be anomaly. The image to be segmented is obtained by inputting the preset first prompt word into a visual language model. Semantic context description .

[0033] The reactive saliency extraction sensing module processes the prior map of the initial abnormal response. Reaction saliency extraction was performed to obtain the first candidate anomaly target set. .

[0034] The deliberate logical reasoning module describes based on semantic context. Treating abnormal segmentation images By conducting careful logical reasoning, a second set of candidate abnormal targets is obtained. .

[0035] The cross-source consistency arbitration module, for the first candidate abnormal target set Second candidate anomaly target set Perform cross-source consistency arbitration to obtain the total set of abnormal targets.

[0036] The semantic suggestion decoupling module performs semantic suggestion decoupling on each anomalous target in the total anomalous target set to obtain clean suggestion words. .

[0037] Cascaded open vocabulary detection and segmentation module, based on clean prompt words Treating abnormal segmentation images Perform cascaded detection and segmentation to obtain anomaly instance masks. .

[0038] The adaptive fusion modulation module, based on the abnormal instance mask Prior graph of initial abnormal response Adaptive fusion modulation is performed to obtain the final anomaly segmentation heatmap. .

[0039] The beneficial effects of this invention are: 1. Effectively overcomes the contradiction between the sensitivity of layered visual perception and the specificity of high-level semantic reasoning. A reactive approach ensures recall, while a deliberate approach utilizes high-level contextual logic to filter perceptual artifacts, significantly reducing the false positive rate without sacrificing recall.

[0040] 2. Solve the background leakage problem in pixel-level tasks. Effectively decouple target entities from background attributes from the semantic input source. Suppress background vocabulary from the attention network's features through a semantic cue decoupler, ensuring that the boundary of the abnormal target segmentation mask accurately matches the actual contour of the target.

[0041] 3. Possesses strong logical interpretability. The metacognitive arbitration mechanism generates an explicit chain of evidence containing the reasons for the ruling while outputting the segmentation result, providing interpretable and traceable logical judgment basis, significantly improving the safety and interpretability of the autonomous driving perception system. Attached Figure Description

[0042] Figure 1 This is a flowchart of the method of the present invention.

[0043] Figure 2 This is a block diagram of the system of the present invention.

[0044] Figure 3 This is a schematic diagram of a general computing device structure for executing a method, provided in an embodiment of the present invention.

[0045] Figure 4 This is a visual comparison of the effects of suppressing background leakage in an open-world scene according to an embodiment of the present invention.

[0046] Figure 5 This is a comparison diagram of the segmentation effect of the present invention in dealing with perceptual artifacts caused by high-frequency normal visual features.

[0047] Figure 6 This is a comparison diagram of the segmentation performance of the present invention in solving the problem of missed detection caused by semantic occlusion of targets with abnormal spatial positions.

[0048] Figure 7 This is a comparison diagram of the segmentation effect of the present invention in eliminating contextual ambiguity of similar-looking targets.

[0049] In the diagram: Cascaded open vocabulary detection and segmentation module 501, adaptive fusion modulation module 502, cross-source consistency arbitration module 401, semantic prompt decoupling module 402, prudent logical reasoning module 302, reactive saliency extraction perception module 301, local scene perception module 201, processor 701, memory 702, storage device 703, output interface 705, system bus 706, program instructions 707, model parameters and configuration 708. Detailed Implementation

[0050] The present invention will now be described in more detail with reference to the accompanying drawings and embodiments. However, the present invention is not limited thereto. For those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention. Contents not described in detail in this specification are prior art known to those skilled in the art.

[0051] like Figure 1 As shown, the open-world road anomaly segmentation method of this embodiment is implemented according to the following steps: S1. Obtain the image to be segmented in the road scene. The data is then input into a discriminative anomaly detection model or a generative anomaly detection model for processing to obtain an initial anomaly response prior map. .

[0052] In practice, the discriminative anomaly detection model adopts the Mask2Anomaly model (from a 2014 IEEE paper) in existing technology; the generative anomaly detection model adopts the DOOD (Diffusion for Out-of-Distribution Detection on Road Scenes and Beyond) model (from a 2014 ECCV paper) in existing technology.

[0053] Specifically, the initial anomaly response prior graph This provides a basic visual response benchmark for subsequent dual-path anomaly candidate extraction and adaptive fusion modulation.

[0054] S2. Segment the image to be anomaly. The image to be segmented is obtained by inputting the preset first prompt word into a visual language model (VLM) for processing. Macro-semantic context description .

[0055] The first cue word is used to guide the visual language model from the image to be segmented. Extract road scene description, road element list, and potential point of interest information; semantic context description. This consists of structured data containing road scene descriptions, a list of road elements, and information on potential points of interest. Preset first cue words enable the visual language model to achieve an objective and comprehensive understanding of the scene without relying on any prior information or focusing on anomalies.

[0056] In specific implementations, the visual language model adopts, but is not limited to, the Qwen-VL series models (such as Qwen-VL-Plus). Those skilled in the art should understand that, without departing from the core idea of ​​this invention, other large visual language models with equivalent cross-modal alignment capabilities, such as LLaVA and MiniGPT-4, can also be used as replacements, and all of these fall within the scope of protection of this invention.

[0057] The list of road elements includes at least road infrastructure, vehicles, and environmental elements; potential points of interest are objects or areas that may warrant further attention based on their location, appearance, or context.

[0058] Semantic context description It serves as a semantic anchor for subsequent logical reasoning, constraining the reasoning process of subsequent abnormal candidates and preventing illusions that are out of context.

[0059] In this embodiment, the first prompt is set as: "You are a visual language assistant for road scene analysis. Your task is to objectively and comprehensively understand the provided road scene image. Do not assume any prior information, nor deliberately focus on abnormal targets; your core objective is to objectively describe the actual content in the image. Please strictly and only output valid JSON format data. All string values ​​and array elements must be enclosed in double quotes. Do not modify the JSON structure key names provided below. Task: Global Scene Awareness. Step 1: Road Scene Description. Provide a brief, macroscopic description of the overall scene (e.g., 'city streets during the day,' 'highway at night,' 'rural road with vegetation'). Step 2: Road Element List. List the road elements..." Identify and categorize key elements existing in the road and its immediate vicinity. The categorization should include at least: 1) Road infrastructure: asphalt pavement, lane markings, traffic signs, traffic lights, guardrails, etc.; 2) Vehicles: cars, trucks, buses, motorcycles, etc.; 3) Environmental elements: buildings, trees, sky, sidewalks, etc. Step 3: Potential Points of Interest. Identify any objects or areas that may warrant further attention due to their location, appearance, or context, but do not directly label them as anomalous. Please maintain a neutral and objective description. Examples: 'A dark block on the road surface', 'A stationary vehicle parked on the shoulder', 'An object partially obscured by bushes'. Output format (strict JSON): {"scene_description": "A brief macroscopic description of the scene."} ", "road_elements": {"infrastructure": ["Infrastructure Element 1", "Infrastructure Element 2"], "vehicles": ["Vehicle Element 1", "Vehicle Element 2"], "environment": ["Environment Element 1", "Environment Element 2"]}, "points_of_interest": [{"description": "Neutral description of potential points of interest.", "location_guess": "Rough location estimate (e.g., 'middle right', 'far left', 'center of view')"}]}." By setting the aforementioned first cue word, the visual language model is forced to perform an objective and comprehensive understanding of the scene in the initial stage. Through strict JSON format constraints, it outputs structured data containing scene descriptions, a list of road elements, and information on potential points of interest. This structured data then serves as a solid semantic anchor for subsequent logical reasoning, effectively constraining the subsequent reasoning process for anomalous candidates and preventing the model from developing the illusion of being out of context.

[0060] Step S3 specifically involves: processing the initial anomaly response prior graph. Reaction saliency extraction was performed to obtain the first candidate anomaly target set. .

[0061] S3. Prior diagram of initial abnormal response Perform salient region feature extraction to select the initial abnormal response prior map. Regions with response values ​​higher than a preset threshold are designated as high-significance regions, and each of these high-significance regions is selected as a candidate anomaly target region, forming the first candidate anomaly target set. .

[0062] The reactive saliency extraction perception module is used to simulate the reactive perception path in human cognition. By rapidly responding to the underlying visual salient regions, it prioritizes the recall rate of anomalies and ensures that potential abnormal targets are not missed.

[0063] S4. Describe according to semantic context Treating abnormal segmentation images By conducting careful logical reasoning, a second set of candidate abnormal targets is obtained. .

[0064] Step S4 specifically involves: describing the semantic context. Image to be segmented due to anomalies The second, pre-set prompt word is input into the visual language model, where scene analysis, multi-dimensional anomaly deviation mining, and saliency selection are performed sequentially, outputting a set of second candidate anomaly targets. .

[0065] The second cue word is used to guide the visual language model to perform the following processing: Scenario Analysis: D1, Description in conjunction with semantic context and the image to be segmented due to anomalies Infer and summarize the expected normal road scenarios.

[0066] Multi-dimensional anomaly deviation mining: D2, segmenting images from images to be anomaly All candidate abnormal targets are marked, and the deviation of each candidate abnormal target is evaluated from three dimensions: appearance mismatch, semantic inconsistency, and spatial irregularity. The name of each candidate abnormal target, the deviation evaluation result of each dimension, and the corresponding deviation level are obtained.

[0067] Saliency Selection: D3. Quantify the deviation level of each dimension of each candidate anomaly into a deviation score. Weight the deviation scores and confidence levels of all dimensions for each candidate anomaly to obtain a deviation evaluation value. Sort all candidate anomalies in descending order according to the deviation evaluation values, select the first preset number of candidate anomalies, and generate corresponding bounding boxes and confidence levels for the selected candidate anomalies. Integrate the deviation evaluation results of each dimension into a description. The names, descriptions, bounding boxes, and confidence levels of all selected candidate anomalies constitute the second candidate anomaly set. .

[0068] Appearance mismatch is used to evaluate whether the texture, shape, color, or material state of the candidate anomalous target does not match the normal road surface; semantic inconsistency is used to evaluate whether the category of the candidate anomalous target does not match the prior knowledge of the road scene; spatial irregularity is used to evaluate whether the position, size, orientation, or occupancy state of the candidate anomalous target relative to the drivable area is abnormal.

[0069] Specifically, semantic context description As a semantic anchor for prudent reasoning, it provides contextual constraints for scene analysis, enabling the model to infer the normal object types and spatial layout that should exist in the current road scene, and preventing subsequent deviation assessments from deviating from the actual scene context and creating illusions.

[0070] The deviation level is a qualitative assessment of the severity of deviation for each dimension by the visual language model, such as: severe deviation, moderate deviation, slight deviation, or no deviation. Furthermore, severe deviation, moderate deviation, slight deviation, and no deviation can be quantified by numerical values ​​of 4, 3, 2, and 1, respectively. The deviation assessment value for each candidate anomalous target is obtained by linearly weighting the deviation scores and confidence levels for each dimension.

[0071] In this embodiment, the preset quantity is configured to be one or more (e.g., selecting the top 3 candidate anomalies) to ensure that the system has a high recall rate and provides sufficient cross-validation samples for the subsequent cross-source consistency arbitration module. The description explains that the deviation evaluation results of the three dimensions—appearance mismatch, semantic inconsistency, and spatial irregularity—are integrated into a structured descriptive text to provide an interpretable decision-making basis for the subsequent cross-source consistency arbitration module.

[0072] In this embodiment, the second prompt is set to: "You are a visual language assistant for road safety anomaly detection. Please strictly and only output valid JSON format data. All string values ​​and array elements must be enclosed in double quotes, and modification of the following JSON key names is prohibited. This branch is entirely autonomous inference; the use of any anomaly heatmap priors is strictly prohibited. Provided scenario context:" {scene_context} Task: Three-Step Reasoning Chain. Step 1: Scene Analysis. Combining the provided scene context and images, infer and summarize the expected normal road scene. Step 2: Deviation Mining. Generate anomalous candidate targets and evaluate the deviation degree of each candidate target from the following three dimensions: 1) Appearance Mismatch: Abnormal texture, shape, color, or material state; 2) Semantic Inconsistency: Object categories that do not match the prior knowledge of the road scene; 3) Spatial Irregularity: Abnormal position, size, orientation, or occupancy state relative to the drivable space. Step 3: Saliency Selection. Quantify the deviation degree level of each candidate anomalous target in each dimension into a deviation score. Weight the deviation scores of all dimensions with confidence to obtain a deviation evaluation value. Sort all candidate anomalous targets in descending order according to the deviation evaluation value, and select at most {top_k} key anomalous targets with the highest direct driving risk.

[0073] You must spatially anchor the selected anomaly by providing two-dimensional bounding box coordinates in the format [xmin, ymin, xmax, ymax] (normalized between 0.0 and 1.0). Your output format must strictly adhere to the following JSON structure: {"step1_scene_profiling": "single-line string description", "step2_deviation_mining": [{"candidate_name": "name", "appearance_mismatch": "description and degree of deviation", "semantic_inconsistency": "description and degree of deviation", "spatial_irregularity": "description and degree of deviation", "risk_note": "risk note"}], "autonomous_anomalies": [{"name": "name", "reasoning": "must explicitly cite the scene profiling and the results and reasons for selection across 3 dimensions as a description", "anomaly_type": "appearance|semantic|spatial|mixture", "confidence": 0.95, "bbox_2d": [0.1,0.2, 0.3, 0.4]}]}. Important constraints: The `autonomous_anomalies` list must contain at most `{top_k}` elements sorted in descending order of risk; deviation mining must include explicit text for all three dimensions of evidence; only JSON formatted content should be output, and no code block syntax or additional text is allowed. S5. For the first candidate abnormal target set Second candidate anomaly target set Perform cross-source consistency arbitration to obtain the total set of abnormal targets.

[0074] Each anomalous target in the total set of anomalous targets includes a name, description, bounding box, and confidence level.

[0075] S51. For candidate abnormal targets that appear in two candidate abnormal target sets at the same time, the dual-source consistency is determined by spatial intersection-union ratio calculation, and the candidate abnormal targets with intersection-union ratio greater than a preset threshold are determined as dual-source consistent abnormal targets.

[0076] Furthermore, S151 specifically refers to: for targets that simultaneously appear in the first candidate anomaly target set Second candidate anomaly target set For each candidate anomalous target in the set, calculate the spatial intersection-union ratio (IoU) of the corresponding region of the current candidate anomalous target in the two sets. If the spatial intersection-union ratio is greater than a preset threshold, then the current candidate anomalous target is determined as a dual-source consistent anomalous target and assigned to the first anomalous target set.

[0077] S52. After removing the dual-source consistent abnormal targets from the two sets, perform single-source authenticity verification on the remaining candidate abnormal targets.

[0078] Furthermore, S152 specifically involves: deleting all candidate anomalous targets identified as dual-source consistent anomalous targets from both candidate anomalous target sets, thereby obtaining the updated first candidate anomalous target set. Second candidate anomaly target set .

[0079] S53, For the first candidate abnormal target set The remaining candidate anomalous targets undergo spatial authenticity verification to check whether there are real physical obstacles in the area corresponding to the candidate anomalous targets. Candidate anomalous targets that have real physical obstacles in the area are retained.

[0080] Furthermore, S53 specifically involves: traversing the updated set of first candidate anomaly targets. Each candidate anomaly target region and semantic context description The information, along with the preset third prompt word, is input into the visual language model for spatial authenticity verification to obtain the verification conclusion of the current candidate abnormal target. If the verification conclusion is that there is a real physical obstacle, the current candidate abnormal target is classified into the third abnormal target set.

[0081] The third cue word is used to guide the visual language model to focus on the current candidate abnormal target region, combined with semantic context description. The system verifies whether there are real physical obstacles in the area corresponding to the current candidate abnormal target, and outputs the verification conclusion. For candidate abnormal targets whose verification conclusion is that there are real physical obstacles, the system also outputs the name, description and confidence level of the candidate abnormal target.

[0082] Spatial authenticity verification is performed by a visual language model based on candidate abnormal target regions and semantic context descriptions. The system checks whether there are real physical obstacles within the candidate anomaly target area. If they exist, they are identified as confirmed anomalies and retained; otherwise, they are identified as false alarms and removed.

[0083] S54, For the second candidate abnormal target set The remaining candidate abnormal targets are subjected to semantic authenticity verification to determine whether they are real abnormal targets or false alarms. Candidate abnormal targets corresponding to those determined to be real abnormal targets are retained.

[0084] Furthermore, S54 specifically involves: traversing the updated set of second candidate anomaly targets. The name, description, bounding box, confidence score, and semantic context description of each candidate anomaly target are listed below. The input, along with the preset fourth prompt word, is fed into the visual language model for semantic authenticity verification to obtain the verification conclusion of the current candidate abnormal target. If the verification conclusion is that the current candidate abnormal target is a real abnormal target, then the current candidate abnormal target is assigned to the second abnormal target set.

[0085] The fourth cue word guides the visual language model to focus on the region where the bounding box of the current candidate anomaly is located, combined with the name, description, confidence level, and semantic context of the candidate anomaly. It determines whether the current candidate abnormal target is a real abnormal target or a false alarm abnormal target, and then outputs the review conclusion.

[0086] Semantic authenticity verification is performed by a visual language model based on the candidate anomaly's name, description, bounding box, confidence level, and semantic context. The system focuses on the region within the bounding box and determines whether the candidate anomalous target is a real anomalous target or a false alarm. If it is determined to be a real anomalous target, it is retained; if it is determined to be a false alarm, it is removed.

[0087] S55. Merge all candidate abnormal targets retained in the dual-source consistency judgment and single-source authenticity verification to obtain the total abnormal target set.

[0088] Furthermore, S55 specifically involves merging the first set of abnormal targets, the second set of abnormal targets, and the third set of abnormal targets to obtain the total set of abnormal targets.

[0089] Specifically, the spatial intersection-union ratio is defined as the ratio of the intersection area to the union area of ​​the bounding box regions corresponding to two candidate anomaly targets.

[0090] The preset threshold is an empirical value, set to 0.30 in this embodiment. When the spatial intersection-union ratio is greater than the preset threshold, it indicates that the reactive perception path and the deliberate reasoning path have independently confirmed the same abnormal target. The candidate abnormal target has a high degree of confidence and does not require secondary verification.

[0091] Furthermore, the third prompt can guide the visual language model to output a reasoning chain (an interpretable chain of evidence for the ruling), facilitating subsequent source tracing. The fourth prompt can also guide the visual language model to output a reasoning chain (an interpretable chain of evidence for the ruling), facilitating subsequent source tracing.

[0092] In this embodiment, the third prompt is set to: "Please strictly and only output valid JSON format data. All string values ​​must be enclosed in double quotes. Task: Reanalyze the anomaly candidate targets proposed by the reactive sensing path. Input information includes: global context description, the name of the anomaly hypothesized by the reactive sensing path, the reasoning explanation for the hypothesis, and the bounding box defining the region of interest in the input image. Core question: Is the hypothesized anomaly object located within the region of interest a real, visually significant road anomaly that should be detected by the system (note: anomalies include any unexpected obstacles, animals, debris, or road hazards)? Your output format must strictly follow the following JSON structure: {"Verdict":"Output ANOMALY_CONFIRMED (confirmed anomaly) or FALSE_ALARM (false alarm)", "Justification": "Brief explanation and adjudication evidence", "visual_description": "A concise noun phrase only for the foreground object (e.g., 'white cow', 'traffic cone')." Strictly prohibit the inclusion of spatial prepositions or environmental terms (such as 'road', 'street', or 'background'). Important constraint: Output only JSON formatted content; no code block syntax or additional text is allowed. The fourth cue word is set to: "Please strictly and only output valid JSON formatted data. All string values ​​must be enclosed in double quotes. Task: Reanalyze visually salient regions proposed by a deliberate reasoning path. Input information includes: a global context description, the names of anomalous candidate targets proposed by the reasoning path, and bounding boxes defining regions of interest in the input image. Core question: In conjunction with the global context, is the object a harmless background element (determined as FALSE_ALARM), or a genuine anomaly and road hazard that deviates from expectations (determined as ANOMALY_CONFIRMED)? Your output format must strictly follow the following JSON structure: {"Verdict": "Output ANOMALY_CONFIRMED or FALSE_ALARM", "Justification": "A brief explanation and evidence of the judgment"}. Important constraint: Output only JSON formatted content; no code block syntax or additional text is allowed." S6. Perform semantic decoupling on each anomalous target in the total anomalous target set to obtain clean prompt words. .

[0093] Step S6 specifically involves: performing text standardization, spatial phrase stripping, environmental phrase stripping, lexical filtering, and attribute extraction on the name of each target in the total abnormal target set to obtain clean prompt words centered on the abnormal target entity and free of background environmental vocabulary. .

[0094] Specifically, text standardization unifies the names of all anomalous targets into a standardized format; spatial phrase stripping removes prepositional phrases indicating spatial location from the names; environmental phrase stripping removes words indicating road scene background from the names; lexical filtering eliminates invalid or interfering words; and attribute extraction retains the core words that describe the essential attributes of anomalous targets.

[0095] The clean prompt word generated after the above processing It achieves the suppression of attention overflow from the source of semantic input, avoids the pollution of the attention mechanism of the subsequent segmentation model by background words, and ensures that the boundary of the segmentation mask for abnormal targets accurately matches the actual contour of the target.

[0096] S7. Based on the pure prompt words Treating abnormal segmentation images Perform cascaded detection and segmentation to obtain anomaly instance masks. .

[0097] S71. Segment the image to be anomaly. and clean prompt words for all anomalous targets They are input together into an open vocabulary detector for processing, resulting in a set of candidate bounding boxes for each anomalous target.

[0098] S72. Using the candidate bounding box set of the anomalous target as a geometric constraint, a zero-shot segmentation model is used to segment the image of the anomalous target. Perform image segmentation to obtain the image to be segmented for anomalies. Pixel-level anomaly instance mask This yields pixel-level anomaly segmentation results.

[0099] Specifically, the open vocabulary detector can employ the GroundingDINO model. When the open vocabulary detector fails to detect a valid bounding box, it outputs an empty candidate structure to avoid abnormal interruptions in subsequent segmentation. The zero-shot segmentation model can employ the SAM instance segmentation model.

[0100] Furthermore, both the open vocabulary detector and the zero-shot segmentation model can be replaced with other models that have equivalent functionality.

[0101] Cascaded open vocabulary detection and segmentation employs a two-stage process: the first stage involves an open vocabulary detector using clean prompt words. Treating abnormal segmentation images The first stage locates abnormal target regions and generates candidate bounding boxes. The candidate bounding box set is then subjected to threshold filtering, non-maximum suppression, and Top-K retention processing. The filtered bounding boxes are then mapped to the input image coordinate system. In the second stage, the mapped bounding boxes are used as geometric constraints to guide the zero-shot segmentation model to perform fine segmentation within the defined region, avoiding interference from the background region on the segmentation results.

[0102] S8. Based on the exception instance mask Prior graph of initial abnormal response Adaptive fusion modulation is performed to obtain the final anomaly segmentation heatmap. .

[0103] Specifically, anomaly segmentation heatmap This is a continuous-valued anomaly confidence map, where the value of each pixel represents the confidence level that the pixel belongs to an anomalous target. Anomaly instance mask. It provides accurate instance-level segmentation results confirmed by careful reasoning, and anomaly segmentation heatmaps. Building upon this, the global anomaly sensitivity of the reactive sensing path is integrated, enhancing the response strength and providing a smooth confidence transition within the confirmed anomaly target area, while preserving potential anomaly signals discovered by the reactive path in non-target areas.

[0104] Adaptive fusion modulation is set according to the following formula: ; ; in, This is the final anomaly segmentation heatmap; This is the prior graph of the initial abnormal response; For Hadamard product; For abnormal instance masks; This is the preset attenuation coefficient for non-target areas; This is the preset enhancement gain coefficient; For response domain energy modulation parameters; To respond to the lower limit value.

[0105] Specifically, when the underlying anomaly prior generation module adopts a generative anomaly detection model, the following is taken: ,pass mask for abnormal instances Enhance; When the underlying anomaly prior generation module adopts a discriminative anomaly detection model, take ,pass mask for abnormal instances Provide a minimum guarantee.

[0106] Specifically, This is used to ensure that anomalous target areas have a minimum identifiable response intensity in the final heatmap. In this embodiment, when the underlying anomalous prior generation module uses a discriminative anomalous detection model... A preset constant of 0.90 is used; when the underlying anomaly prior generation module adopts a generative anomaly detection model. Set to 0. Preferably, It can also be based on The statistical quantiles of the pixel response are determined, but this embodiment uses a fixed value.

[0107] Specifically, the processing logic for adaptive fusion modulation is as follows: for the abnormal instance mask... The region marked as an anomalous target is enhanced by increasing the gain coefficient. Enhance the response intensity of this region, and use energy modulation parameters As the lower bound of the response for this region, it ensures that the abnormal target region is sufficiently enhanced and stably presented in the final heatmap; for the abnormal instance mask... The region not marked as an anomalous target is attenuated by the non-target region attenuation coefficient. Response attenuation is performed to suppress anomalous responses in background regions. This process enhances the performance of anomalous target regions and suppresses background regions, effectively reducing the false positive rate while maintaining high recall.

[0108] In this embodiment, to best adapt to the underlying anomaly prior generation module of different architectures, the system has configured the parameters of adaptive fusion modulation as follows: When the underlying anomaly prior generation module adopts a discriminative anomaly detection model, the attenuation coefficient of the non-target region... Set to 0.90 to enhance the gain coefficient. Set to 0, Set to 0.90; when the underlying anomaly prior generation module adopts a generative anomaly detection model, the attenuation coefficient of the non-target region is... Set to 0.65 to enhance the gain coefficient. Set to 0.15, Set to 0.

[0109] Furthermore, based on the anomaly segmentation heatmap By calculating ranking metrics such as AUPRC, AUROC, and FPR@95, segmentation metrics such as precision, recall, F1 score, and intersection-union ratio can also be calculated to quantitatively evaluate the results. These metrics are for performance evaluation only and do not constitute essential technical steps in the method of this invention.

[0110] like Figure 2 As shown, this embodiment also provides an open-world road anomaly segmentation system based on dual-process cognition, including: The image acquisition module and the underlying anomaly prior generation module acquire images of the road scene to be segmented. The data is then input into a discriminative anomaly detection model or a generative anomaly detection model for processing to obtain an initial anomaly response prior map. (Basic probability heatmap).

[0111] The global scene perception module 201 segments the image to be anomaly. The image to be segmented is obtained by inputting the preset first prompt word into a visual language model. Semantic context description .

[0112] The reactive saliency extraction sensing module 301 processes the initial anomalous response prior map. Reaction saliency extraction was performed to obtain the first candidate anomaly target set. .

[0113] 302, a deliberate logical reasoning module, describes based on semantic context. Treating abnormal segmentation images By conducting careful logical reasoning, a second set of candidate abnormal targets is obtained. .

[0114] Cross-source consistency arbitration module 401, for the first candidate abnormal target set Second candidate anomaly target set Perform cross-source consistency arbitration to obtain the total set of abnormal targets.

[0115] The semantic prompt decoupling module 402 performs semantic prompt decoupling on each abnormal target in the total abnormal target set to obtain clean prompt words. .

[0116] Cascaded open vocabulary detection and segmentation module 501, based on clean prompt words Treating abnormal segmentation images Perform cascaded detection and segmentation to obtain anomaly instance masks. .

[0117] The adaptive fusion modulation module 502, based on the abnormal instance mask Prior graph of initial abnormal response Adaptive fusion modulation is performed to obtain the final anomaly segmentation heatmap. .

[0118] like Figure 3As shown, this embodiment also provides an electronic device (general-purpose computing device) for performing the above-described method, including a processor 701 (such as a CPU / GPU), a memory 702 (such as RAM), a storage device 703 (such as an SSD / ROM) connected via a system bus 706, and an input interface 704 and an output interface 705. The input interface 704 is used to receive image files or data streams to be processed; the output interface 705 is used to output the final anomaly segmentation heatmap, instance mask, and analysis results. The storage device 703 stores program instructions 707 and model parameters and configurations 708 (including VLM, GroundingDINO, SAM, and fusion parameters). After the program instructions 707 are loaded into the memory 702 by the processor 701 and executed, the processor 701 implements the specific steps of the above-described method (S1 to S8).

[0119] To further demonstrate the outstanding substantive features and significant progress of the technical solution of this invention, this embodiment combines... Figures 4 to 7 The qualitative verification effect of the present invention in typical complex scenarios is described in detail below: like Figure 4 The image shows a visual comparison of the effectiveness of this invention in suppressing background leakage in an open-world scenario. Without using the semantic prompting decoupling module, directly guiding the visual base model using the original complex long sentence resulted in severe overflow of the attention network, activating a large area of ​​the background and causing serious background leakage and mask out-of-bounds errors. However, by employing the semantic prompting decoupling mechanism in step S6 of this invention, spatial and environmental prepositions were successfully removed, core object labels were locked, and the downstream segmentation model was guided to generate a mask that perfectly fits the target edge, completely eliminating false positive background activation.

[0120] like Figure 5 The image shows a comparison of the segmentation performance of this invention in dealing with perceptual artifacts caused by high-frequency normal visual features. When faced with high-frequency visual features such as strong sunlight shadows and road surface reflections, existing single-response perceptual models, lacking semantic cognition, misclassify local pixel gradient abrupt changes as abnormal obstacles. This invention introduces a dual-process architecture, where a metacognitive arbitration mechanism identifies that the reactive feature lacks high-level semantic support. In the secondary verification, it correctly classifies it as harmless background, successfully filtering out perceptual artifacts and significantly reducing the false positive rate (FPR).

[0121] like Figure 6The image shown is a comparison of the segmentation performance of this invention in addressing the issue of missed detections due to semantic occlusion caused by targets with abnormal spatial locations. For objects such as detached car tires, existing models are prone to "semantic occlusion" and thus miss detections because their local textures conform to the statistical distribution of "safe vehicle components" learned during model training. The deliberate logical reasoning module of this invention proactively identifies anomalies in its "spatial deviation" (i.e., the tire should not exist isolated in the center of the lane), enabling the system to accurately capture this hidden dangerous target.

[0122] like Figure 7 The image shows a comparison of the segmentation performance of this invention in eliminating contextual ambiguity for visually similar targets. Complex edge scenes often simultaneously contain embankments (safe backgrounds) and landslides / rockfalls (dangerous anomalies) that are extremely similar in appearance. Existing technologies suffer from severe semantic ambiguity within the same frame, manifesting as false positives for safe embankments and false negatives for dangerous rockfalls. The cross-source consistency arbitration module of this invention, by comprehensively evaluating the target's physical location, global contextual constraints, and traffic congestion risk, accurately suppresses embankment activation and recalls rockfall targets within the same scene, demonstrating extremely strong high-order scene disambiguation capabilities and logical coherence.

[0123] This invention utilizes a dual-process cognitive collaboration mechanism, integrating reactive perception and deliberate reasoning, to significantly reduce the false positive rate without sacrificing recall. It also suppresses background leakage at the source through semantic prompt decoupling, ensuring that the segmentation mask boundary accurately matches the target contour. Simultaneously, it outputs an interpretable explicit adjudication evidence chain, enhancing the safety and interpretability of the autonomous driving perception system.

[0124] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. An open-world road anomaly segmentation method based on dual-process cognition, characterized in that, Includes the following steps: S1. Obtain the image to be segmented in the road scene. The data is then input into a discriminative anomaly detection model or a generative anomaly detection model for processing to obtain an initial anomaly response prior map. ; S2. Segment the image to be anomaly. The image to be segmented is obtained by inputting the preset first prompt word into a visual language model. semantic context description ; S3. Prior graph of initial abnormal response Reaction saliency extraction was performed to obtain the first candidate anomaly target set. ; S4. Describe according to semantic context Treating abnormal segmentation images By conducting careful logical reasoning, a second set of candidate abnormal targets is obtained. ; S5. For the first candidate abnormal target set Second candidate anomaly target set Perform cross-source consistency arbitration to obtain the total set of abnormal targets; S6. Perform semantic decoupling on each anomalous target in the total anomalous target set to obtain clean prompt words. ; S7. Based on the pure prompt words Treating abnormal segmentation images Perform cascaded detection and segmentation to obtain anomaly instance masks. ; S8. Based on the exception instance mask Prior graph of initial abnormal response Adaptive fusion modulation is performed to obtain the final anomaly segmentation heatmap. .

2. The open-world road anomaly segmentation method according to claim 1, characterized in that: The first prompt word is used to guide the visual language model from the image to be segmented. Extracting road scene description, road element list, and potential point of interest information; the semantic context description It is structured data that includes road scene descriptions, a list of road elements, and information on potential points of interest.

3. The open-world road anomaly segmentation method according to claim 1, characterized in that, Step S3 specifically involves: Prior graph of initial abnormal response Perform salient region feature extraction to select the initial abnormal response prior map. Regions with median values ​​higher than a preset threshold are designated as highly significant regions. Each highly significant region is then selected as a candidate anomaly target region, forming the first candidate anomaly target set. .

4. The open-world road anomaly segmentation method according to claim 1, characterized in that, Step S4 specifically involves: Describing semantic context Image to be segmented due to anomalies The second, pre-set prompt word is input into the visual language model, where scene analysis, multi-dimensional anomaly deviation mining, and saliency selection are performed sequentially, outputting a set of second candidate anomaly targets. ; The second cue word is used to guide the visual language model to perform the following processing: D1. Describe in conjunction with semantic context Image to be segmented from anomaly Infer and summarize the expected normal road scenarios; D2. Segmenting the image from the anomaly to be identified The abnormal candidate targets are marked, and the deviation of each candidate abnormal target is evaluated from three dimensions: appearance mismatch, semantic inconsistency and spatial irregularity. The name of each candidate abnormal target, the deviation evaluation result of each dimension and the corresponding deviation level are obtained. D3. Quantify the deviation levels of each dimension of the candidate anomaly targets into deviation scores. Weight the deviation scores and confidence levels of all dimensions to obtain a deviation evaluation value. Sort the candidate anomaly targets in descending order according to the deviation evaluation values, and select the first preset number of candidate anomaly targets. Generate corresponding bounding boxes and confidence levels for the selected candidate anomaly targets. Integrate the deviation evaluation results of each dimension into a description. The names, descriptions, bounding boxes, and confidence levels of the selected candidate anomaly targets constitute the second candidate anomaly target set. .

5. The open-world road anomaly segmentation method according to claim 4, characterized in that: The appearance mismatch is used to evaluate whether the texture, shape, color, or material state of the candidate anomalous target does not match the normal road surface; the semantic inconsistency is used to evaluate whether the category of the candidate anomalous target does not match the prior knowledge of the road scene; the spatial irregularity is used to evaluate whether the position, size, orientation, or occupancy state of the candidate anomalous target relative to the drivable area is abnormal.

6. The open-world road anomaly segmentation method according to claim 1, characterized in that, Step S5 specifically involves: S51. For candidate abnormal targets that appear in two candidate abnormal target sets at the same time, the dual-source consistency is determined by spatial intersection-union ratio calculation, and the candidate abnormal targets with an intersection-union ratio greater than a preset threshold are determined as dual-source consistent abnormal targets. S52. After removing the dual-source consistent abnormal targets from the two sets, perform single-source authenticity verification on the remaining candidate abnormal targets. S53, For the first candidate abnormal target set The remaining candidate abnormal targets are subjected to spatial authenticity verification to check whether there are real physical obstacles in the area corresponding to the candidate abnormal targets. Candidate abnormal targets that have real physical obstacles in the area are retained. S54, For the second set of candidate abnormal targets The remaining candidate abnormal targets are subjected to semantic authenticity verification to determine whether they are real abnormal targets or false alarms. Candidate abnormal targets that are determined to be real abnormal targets are retained. S55. Merge all candidate abnormal targets retained in the dual-source consistency determination and single-source authenticity verification to obtain a total abnormal target set; the total abnormal target set includes the name of each abnormal target.

7. The open-world road anomaly segmentation method according to claim 6, characterized in that, Step S6 specifically involves: The names of each anomalous target in the overall anomalous target set are subjected to text normalization, spatial phrase stripping, environmental phrase stripping, lexical filtering, and attribute extraction to obtain clean prompt words centered on the anomalous target entity and free of background environmental words. .

8. The open-world road anomaly segmentation method according to claim 1, characterized in that, Step S7 specifically involves: S71. Segment the image to be anomaly. and all clean prompt words They are input together into an open vocabulary detector for processing to obtain a set of candidate bounding boxes for each anomalous target; S72. Using the candidate bounding box set as geometric constraints, a zero-shot segmentation model is employed for the image to be segmented due to anomalies. Perform image segmentation to obtain anomaly instance masks. .

9. The open-world road anomaly segmentation method according to claim 1, characterized in that: The adaptive fusion modulation in step S8 is set according to the following formula: ； ； in, This is the final anomaly segmentation heatmap; This is the prior graph of the initial abnormal response; For Hadamard product; For abnormal instance masks; The attenuation coefficient for non-target regions; To enhance the gain coefficient; For response domain energy modulation parameters; To respond to the lower limit value.

10. An open-world road anomaly segmentation system employing the method described in any one of claims 1-9, characterized in that... include: The image acquisition module and the underlying anomaly prior generation module acquire images of the road scene to be segmented. The data is then input into a discriminative anomaly detection model or a generative anomaly detection model for processing to obtain an initial anomaly response prior map. ; The global scene perception module (201) segments the image to be anomaly. The image to be segmented is obtained by inputting the preset first prompt word into a visual language model. semantic context description ; The reactive saliency extraction sensing module (301) processes the initial anomalous response prior map. Reaction saliency extraction was performed to obtain the first candidate anomaly target set. ; The deliberate logical reasoning module (302) describes the semantic context. Treating abnormal segmentation images By conducting careful logical reasoning, a second set of candidate abnormal targets is obtained. ; The cross-source consistency arbitration module (401) arbitrates the first set of candidate abnormal targets. Second candidate anomaly target set Perform cross-source consistency arbitration to obtain the total set of abnormal targets; The semantic prompt decoupling module (402) performs semantic prompt decoupling on each abnormal target in the total abnormal target set to obtain clean prompt words. ; Cascaded open vocabulary detection and segmentation module (501), based on clean prompt words Treating abnormal segmentation images Perform cascaded detection and segmentation to obtain anomaly instance masks. ; The adaptive fusion modulation module (502) uses an anomaly instance mask. Prior graph of initial abnormal response Adaptive fusion modulation is performed to obtain the final anomaly segmentation heatmap. .

Images