Embodied question answering method based on empirical active vision verification

Abstract

The present application provides a kind of embodiment question answering method based on empirical active vision verification, belong to artificial intelligence and robot field.The present application constructs candidate image set by collecting multi-view image in each observation position;Experience context is formed by four-channel multimodal retrieval to call historical experience;Relying on experience context, " selection-focusing-back" active vision cycle is executed, image is filtered in situ, virtual zoom is acquired to obtain fine-grained evidence, and stratified back is divided when information is insufficient;If evidence is sufficient, answer is output, otherwise, observation point position is switched to cycle collection;After round is finished, three-layer reflection of task, environment and hypothesis is executed, track is distilled into standardized skill, stored in skill library and updated meta-skill.The present application breaks through the limitation of fixed view passive observation, relies on virtual zoom to reduce physical movement;Cross-task strategy reuse is realized through structured experience, closed-loop optimization mechanism of experience-driven exploration and exploration iterative updating experience is formed, and question answering accuracy and scene exploration efficiency are significantly improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and embodied intelligence in robotics, and in particular to an embodied question-answering method based on empirical active visual verification. Background Technology

[0002] Active embodied question answering requires agents to perform autonomous navigation in complex 3D environments, actively collect visual evidence, and reason based on partial observations to answer given questions. This task places high demands on both spatial navigation and active scene understanding capabilities.

[0003] In the active exploration process of embodied agents, existing methods generally use fixed-viewpoint snapshots as observation input. This passive observation paradigm has fundamental limitations: when the agent reaches an observation location, that location contains rich multi-angle, multi-scale scene information, but the fixed-viewpoint observation mode can only capture a very small subset of it. The spatial resolution of distant targets or small objects drops sharply, making it difficult to effectively identify key subtle visual features such as text labels on objects, instrument readings, and the texture of objects. When the agent determines that a certain area may contain key information but the current observation resolution is insufficient to confirm it, it often has to rely on a lot of physical movement to obtain a better observation perspective. This paradigm, which emphasizes physical movement and neglects perceptual adjustment, not only has a significant bottleneck in exploration efficiency but is also prone to irreversible mission failure due to path planning errors in complex topological environments.

[0004] Furthermore, existing methods operate entirely independently of each exploration round. Even if an agent has mastered an effective exploration pattern for a certain type of problem or identified common reasoning traps in its historical exploration, it can only start from scratch and blindly explore when faced with similar new problems, lacking the accumulation and transfer mechanism of historical experience.

[0005] In summary, existing embodied question-answering technologies still suffer from problems such as rigid observation paradigms, insufficient perceptual flexibility, and the inability to reuse experience across rounds, making it difficult to simultaneously meet the needs of fine-grained visual verification and efficient exploration. Therefore, there is an urgent need for a new technological solution that can fully utilize the hardware perceptual degrees of freedom of intelligent agents and achieve experience accumulation and transfer. Summary of the Invention

[0006] To address the aforementioned issues, this invention aims to propose an embodied question-answering method based on empirical active visual verification. This method overcomes the limitations of passive observation from a fixed perspective and reduces physical movement by relying on virtual zoom. Through structured experience, it enables cross-task strategy reuse, forming a closed-loop optimization mechanism that drives exploration and iteratively updates experience. This significantly improves the accuracy of embodied question-answering and the efficiency of scene exploration, and can be widely applied to scenarios such as indoor inspection robots and home embodied intelligent agents.

[0007] To achieve the above objectives, the technical solution of the present invention is implemented as follows: An embodied question-answering method based on empirical active visual verification includes the following steps: Step S1: At each observation location of the embodied agent, capture multiple egocentric observation images covering different orientations, and combine them with historical snapshots and boundary rendering maps to construct a candidate image set; Step S2: Based on the visual embedding of the current problem and scene, obtain matching structured experience knowledge from the skill base through multimodal retrieval to generate experience context; Step S3: Execute an active visual verification loop of “selection-focus-back” based on the empirical context: filter images containing key evidence from the candidate image set, perform high-resolution scaling on the region of interest of the selected image to extract fine-grained visual evidence, and perform image-level or location-level hierarchical backoff when information is insufficient; Step S4: After the visual evidence obtained from active visual verification and the historical snapshot are filtered by the visual evidence manager, the responder determines whether the evidence is sufficient and generates an answer; if the evidence is insufficient, the planner selects the next navigation direction and switches to a new observation position, and repeats steps S1 to S4. Step S5: After a single exploration round ends, perform a three-level retrospective reflection on the complete exploration trajectory, distill the reflection results into structured skills and write them into the skill library, and simultaneously update and summarize the meta-skills that generate and store all skills and global general strategies.

[0008] Furthermore, the process of generating empirical context through multimodal retrieval in step S2 includes: Step S21: Simultaneously enable four channels—semantic similarity, question type matching, target object overlap, and scene visual similarity—to rank the skills in the skill library; Step S22: Calculate the overall skill score by fusing the four-channel sorting results using the reciprocal ranking fusion method. The scoring formula is as follows:

[0009] in, The overall skill score is represented by c, which represents the four search channels, and k is a smoothing constant. Let i be the ranking of the i-th skill within channel c. For skill confidence; Step S23: Filter the Top-K matching skills based on the comprehensive score, and combine the global guidance rules of the meta-skills to hierarchically encapsulate and obtain the structured experience context.

[0010] Furthermore, the "select-focus-rewind" active visual verification loop in step S3 includes three sub-skill processes: Step S31 Multi-view selection: Input the candidate image set, the current problem, and the empirical context into the visual language model, and output the optimal image index and selection confidence; when the confidence is lower than a preset threshold... At that time, multiple candidate images are retained for subsequent rollback; Step S32 Region scaling: Locate the coordinates of the region of interest in the selected image using a visual language model, and re-render a high-definition detail image based on the region's 3D pose to simulate optical zoom and obtain fine-grained visual evidence. Step S33 Layered rollback: When the scaling evaluation determines that the information is insufficient, a two-level rollback strategy is executed. The first level is image-level rollback, which switches to other candidate images at the same observation position and repeats the scaling and verification. The second level is position-level rollback, which abandons the current observation position and jumps to the navigation planning process.

[0011] Furthermore, the three-level retrospective reflection in step S5 includes: Task-level reflection: Evaluate the completion status of task objectives, identify key exploration nodes, and sort out the logical chain of reasoning; Environmental layer reflection: Discover rooms, areas and objects with high information value, and extract reusable spatial layout and navigation patterns; Hypothetical reflection: Based on counterfactual reasoning, we can deduce the effectiveness of alternative exploration strategies and expand the strategy search space; The results of the three-layer reflection are input into a visual language model for distillation to generate structured skills.

[0012] Furthermore, the structured skill obtained by distillation in step S5 is expressed using standardized units, as follows:

[0013] In the formula, The general trigger condition for skills, For the set of condition-action-reasoning triple rules, For the visual embedding set of related scenes, , These are positive behavior suggestions and negative behavior constraints. Provides prompts for proactive visual operation. For the scope of skill application, It includes statistical information such as confidence level and success / failure count.

[0014] Furthermore, in step S1, the number of images acquired at each observation location... The three images together cover a 360° panoramic view; the maximum number of zooms per observation position in step S3 is 3; and the confidence threshold for multi-view selection is set. .

[0015] Furthermore, the weights of the four channels are configured as follows: semantic channel 0.3, visual channel 0.3, object channel 0.2, type channel 0.2, and a smoothing constant. The default number of matching skills is searched. .

[0016] Furthermore, the visual language model outputs the region of interest as continuous relative coordinates. Based on the 3D scene pose corresponding to these coordinates, high-resolution re-rendering is performed to generate detailed images. The scaled-down detailed images are stored in a historical snapshot set as evidence for reasoning.

[0017] Furthermore, each skill can store a maximum of [number missing] records. Visual embedding of related scenes; similarity threshold for skill matching and merging. Meta-skills are obtained by distilling the entire skill library, storing general exploration strategies and global active visual experience, and are updated synchronously with the skill library.

[0018] Furthermore, the maximum number of navigation steps in a single complete exploration round is limited to 50 steps; if sufficient visual evidence is not collected even after traversing to the maximum number of steps, the exploration round is terminated and the three-level retrospective reflection process in step S5 is executed.

[0019] Beneficial effects: This invention breaks through the perception limitations of the fixed-viewpoint passive observation paradigm. By using an active visual skill module, it endows the agent with active perception capabilities such as multi-viewpoint selection, region scaling, and intelligent backtracking, and obtains fine-grained visual evidence that traditional methods cannot capture without increasing the cost of physical movement.

[0020] This invention extracts structured skill knowledge from historical exploration trajectories through a three-level retrospective reflection using an experience playback skill module, enabling cross-round experience accumulation and precise transfer, allowing the agent to quickly locate key evidence in similar scenarios.

[0021] This invention employs a two-way iterative optimization mechanism that drives exploration through experience and verifies skills through execution. Knowledge in the skill base guides the selection of proactive visual strategies, and the exploration results are fed back to the skill base for updates via a reflection mechanism. This allows knowledge to be continuously refined and evolved through ongoing exploration, surpassing the current best methods in several key metrics. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is an overall flowchart of the embodied question-answering method based on empirical active visual verification as described in an embodiment of the present invention. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0024] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] Active embodied question answering requires an intelligent agent to perform autonomous navigation in a complex 3D environment, actively collect visual evidence, and reason based on partial observations to answer a given question. Given a natural language question... and three-dimensional indoor environment Active embodied question answering requires intelligent agents to... The system autonomously plans its exploration path, collects sufficient visual evidence through a limited number of navigation and observation steps, and ultimately generates the correct answer. The agent at each time step Maintain the following information: the current egocentric RGB-D image Historical snapshot collection And the candidate set of boundaries obtained by clustering the truncated signed distance function (TSDF) boundary voxels. .

[0026] Existing methods follow a passive observation model based on fixed-viewpoint snapshots, and the decision-making process of the responder is as follows:

[0027] in, Indicates the action of the responder. This represents the responder agent. Indicates the current problem. This represents a collection of snapshots. The Visual Evidence Manager dynamically filters the snapshot collection:

[0028] in, This represents the filtered set of snapshots. This refers to the visual evidence manager. When further exploration is needed, the planner selects the navigation direction.

[0029] in, Indicates the selected navigation direction. Represents the planner, This represents the candidate set of the boundary.

[0030] This baseline has two interconnected limitations: at the perception level, fixed-viewpoint observation cannot proactively adjust the observation strategy according to the semantic requirements of the problem; at the experience level, each exploration round is completely independent, lacking an experience accumulation and transfer mechanism. To address these limitations at both levels, this embodiment proposes an Active Visual Skills Module (Act-V) and an Experience Replay Skills Module (Re-Exp.).

[0031] Example 1 Based on the shortcomings of the existing technology mentioned above, such as Figure 1 As shown, the embodied question-answering method based on empirical active visual verification provided in this embodiment includes the following steps: Step S1: Constructing the candidate image set At each observation location, the embodied agent first captures multiple egocentric observation images covering different orientations. Specifically, at time... The three-dimensional pose is represented as:

[0032] in, For position coordinates, The camera is tilted at an angle. This refers to the camera's tilt angle. Based on the above posture, a first-person view image is obtained:

[0033] in, A collection of images observed from a first-person perspective. It is an RGB image. This is for depth images. At initialization, six self-centered viewpoints covering 360 degrees are captured. In each subsequent step, three self-centered viewpoints covering 120 degrees are captured, aligned with the current orientation. These observed images, along with a set of historical snapshots and a boundary rendering map obtained by clustering TSDF boundary voxels, constitute a candidate image set. .

[0034] Step S2: Pre-exploration experience retrieval At the start of a new exploration round, the system first considers the current problem. Visual embedding of the current scene From the skill library Parallel sorting is performed through four complementary channels: (1) Semantic channel: The question text is encoded using the lightweight sentence embedding model all-MiniLM-L6-v2 and cosine similarity is calculated with the skill embedding vector; (2) Type channel: Determine whether the set of applicable question types for the skill matches the current question type; (3) Object channel: The degree of overlap between the target set extracted from the problem and the skill applicable target set; (4) Visual channel: The visual embedding of the current scene is extracted using CLIP ViT-B-32 encoder, and the similarity between it and the embedding of the skill-related historical scene is calculated.

[0035] The sorting results of the four channels are weighted and summed using the inverse ranking fusion method:

[0036] in, It is a smoothing constant. For skills In the passage The ranking in For skills The confidence scores were 0.3 for visual and textual semantics, 0.3 for type, and 0.2 for object. (Default search) Skills. Retrieved skills and meta-skills. The global guiding principles are hierarchically formatted into a structured experiential context. This information is then injected into each node of the subsequent active visual exploration.

[0037] Step S3: Active Visual Verification Loop Based on empirical context Perform an active visual verification loop of "select-focus-revert": Step S31, Multi-view selection sub-skill: Select candidate image set With experience context The images are input into a visual language model, which evaluates and selects the images most likely to contain key evidence relevant to the current problem.

[0038] in, For the index of the selected image, To select the confidence level. Empirical context. The structured rules and proactive visual guidance included in the algorithm provide crucial semantic priors for visual selection. For example, a structured rule for a skill states, "If the question involves the color attributes of an object, prioritize images that include a frontal view of the target object," and the selector will precisely adjust its preference weights accordingly. When the selection confidence falls below a preset threshold... At this time, the system strategically retains multiple candidate images for subsequent rollback.

[0039] Step S32, visual localization-based region scaling sub-skill: Selecting the optimal image Then, the region of interest is located in a continuous relative coordinate space by the Vision-Language Model (VLM):

[0040] in, and These are the relative coordinates of the top-left and bottom-right corners of the region of interest, respectively. Subsequently, based on the pose information of this region, the system performs high-resolution re-rendering of the region in the 3D scene, generating corresponding detail images. This process computationally simulates the optical zoom operation of a camera, enabling the agent to acquire high-resolution visual details of a selected area without performing physical movement. Each zoomed image is fed into the VLM for adequacy evaluation to determine whether it contains sufficient information to answer the question.

[0041] It should be noted that the high-resolution re-rendering of this invention refers to regenerating a local image with a resolution higher than the original observed image based on the three-dimensional geometric information of the scene, targeting the region of interest in the image, in order to obtain target details.

[0042] Step S33, Multi-view rollback sub-skill: When the zoom evaluation result indicates insufficient information, a two-level hierarchical rollback strategy is provided:

[0043] in, This is the current attempt history. The first level is image-level backtracking: when there are still untried candidate images and the zoom count has not reached the limit (maximum 3 times per step), the currently selected image is discarded, and the next candidate image is selected for region zooming exploration. The second level is location-level backtracking: when all candidate images have been tried or the zoom count has reached the limit, all candidate images at the current location are discarded, and the system returns to the exploration planning stage, where the planner selects a new boundary direction for navigation.

[0044] The three sub-skills described above together constitute the "select-focus-back" active visual verification loop, which can be executed multiple times at each observation position until sufficient visual evidence is obtained or a location-level backoff is triggered. The high-resolution detail images acquired during each region zoom are also included in the snapshot set. This serves as supplementary visual evidence for subsequent reasoning and responses.

[0045] Step S4: Evidence Assessment and Navigation Decision The visual evidence acquired through proactive visual verification, along with historical snapshots, is filtered by the visual evidence manager, and then the responder determines whether the evidence is sufficient. The responder receives the current question. Text structure memory Memory with visual evidence Based on a comprehensive assessment, determine whether the existing evidence is sufficient to generate a reliable answer:

[0046] in, This indicates the action of the responder agent, including outputting an answer or outputting "continue exploring". If the output is a valid answer, the process terminates; if the output is "continue exploring", the planner selects the next navigation direction.

[0047] The agent moves in the selected direction Navigation, gathering new egocentric observations and update the snapshot set. The maximum number of moves per exploration round is limited to 50.

[0048] Step S5: Post-exploration reflection and skill updating After the exploration round ends, the system displays the completed exploration path. The three-level retrospective reflection process formalizes the skills retrieval process as follows:

[0049] in, For task-level reflection, assess the achievement of task objectives, identify key exploration nodes, and analyze the reasoning logic chain; For environmental reflection, analyze rooms, areas and objects with significant informational value, and extract spatial layout patterns that facilitate navigation; For hypothetical reflection, we explore alternative strategies and their potential consequences through counterfactual reasoning, thereby expanding the strategy search space.

[0050] The results of the reflection were distilled into structured skills Each skill is defined as a highly structured knowledge unit for exploration strategies:

[0051] in, For generalized triggering conditions; It is a set of structured rules encoded by condition-action-reasoning triples. Each rule explicitly specifies the triggering condition, the action to be performed, and the reasoning basis behind it. For each skill, a set of associated scene visual embeddings extracted by a pre-trained visual encoder is maintained at most [number missing]. Embed a historical scene; and These are positive behavioral suggestions and negative behavioral constraints, respectively. For proactive visual guidance prompts, record when zooming or viewing angle adjustment operations are performed; To define the scope, it covers problem types, room types, and target object categories; Statistical information, including confidence level, success / failure count, etc. The similarity threshold for skill matching and merging is set to... .

[0052] Meta Skills Then it is generated by summarizing and condensing from the entire skill library:

[0053] Covering general exploration principles, strategy summaries for various problem types, and global proactive visual experience, accompanied by a skill set. It is updated synchronously with the evolution.

[0054] The three stages described above constitute a bidirectional iterative optimization mechanism of feedforward exploration and feedback updates. In the feedforward direction, the skill base provides multi-level semantic priors for multi-view selection, region scaling, and intelligent backtracking through experience retrieval, enabling the agent to quickly locate key evidence in similar scenarios. In the feedback direction, the results of each round of exploration are distilled into reusable structured skill knowledge through three layers of reflection, continuously enriching the content and coverage of the skill base. This bidirectional iterative optimization allows the agent to continuously improve visual search efficiency and exploration quality through ongoing exploration.

[0055] Experimental verification: To verify the technical effectiveness of this invention, tests were conducted on two publicly available benchmark datasets: A-EQA and Express-Bench. Both datasets are based on the Habitat-Matterport 3D (HM3D) platform and provide hundreds of highly realistic and diverse indoor 3D scanning scenes. A-EQA contains 184 question samples, and Express-Bench contains 2,044 question-track pairs.

[0056] In terms of evaluation metrics, the A-EQA uses two metrics, LLM-Match and LLM-SPL, to measure answer correctness and path efficiency. LLM-Match is defined as:

[0057] in, This is the discrete accuracy score assigned by GPT-4. LLM-SPL further incorporates exploration efficiency:

[0058] in, This represents the minimum path length along the true trajectory sufficient to answer the question. It is the actual path length traversed by the agent.

[0059] For Express-Bench usage , , and Four complementary indicators:

[0060]

[0061]

[0062] in, This represents the correctness score assigned by VLM. This represents the grounding score, which reflects visual-semantic consistency.

[0063] In terms of implementation details, this study implements ActiveEye as a VLM-based end-to-end process, using the vLLM inference framework to locally deploy the Qwen3-VL model. Multi-view selection skills are captured at each observation location. A set of candidate images is prepared, covering a 360-degree egocentric view. The region scaling skill employs a VLM-driven visual localization approach, where the model outputs the bounding box of the region of interest in relative coordinate space, and the image is re-rendered at high resolution. A confidence threshold is used to select the multi-view rollback skill. Set to 0.6, the maximum number of zooms per step is limited to 3. The skill library uses the lightweight sentence embedding model all-MiniLM-L6-v2 for semantic retrieval, and the visual scene channel uses the CLIP ViT-B-32 encoder to extract scene embeddings. Each skill maintains a maximum of [number missing]. Embedded historical scenes. Smoothness constant. Default search One skill. The maximum number of moves per exploration round is limited to 50.

[0064] Table 1. Comparative experimental results of the method of this invention with other methods on the A-EQA dataset.

[0065] Table 2. Comparison of the method of this invention with other methods on the Express-Bench dataset.

[0066] On the A-EQA dataset, ActiveEye achieved state-of-the-art performance on both the LLM-Match and LLM-SPL metrics. In terms of response accuracy, ActiveEye outperformed the previous best method based on a commercial VLM, 3D-Mem (+3.8%). Notably, ActiveEye surpassed all GPT-4o-based methods using only an 8B-scale open-source VLM, demonstrating that the significant performance improvement stems from a more intelligent observation strategy and experience-driven exploration mechanism, rather than relying on a more powerful base model. Regarding exploration efficiency, ActiveEye significantly outperformed the second-best method, MTU3D (+8.7%), on the LLM-SPL metric, clearly illustrating that the active visual verification mechanism effectively reduces a large amount of redundant physical movement overhead by performing fine-grained evidence collection in the target region.

[0067] On the Express-Bench dataset, ActiveEye achieves state-of-the-art results in both general correctness and scene-aware correctness. Compared to baseline methods using the same VLM backbone, ActiveEye... The indicator improved by +2.0%. The indicator improved by +3.3%. The indicator improved by +2.5%.

[0068] Table 3. Ablation Experiment Analysis of Each Module of the Invention

[0069] Ablation experiments showed that introducing Act-V alone improved LLM-Match and LLM-SPL on A-EQA by +5.6% and +6.0%, respectively, while introducing Re-Exp. alone improved them by +4.4% and +4.6%, respectively. When both modules were enabled simultaneously, the performance improved by +8.2% and +8.7% compared to the baseline, respectively, verifying the synergistic effect of the two modules.

[0070] Table 4 Scalability analysis on models with different parameter sizes

[0071] Scalability experiments show that ActiveEye exhibits stable and significant performance gains across different parameter scales, demonstrating the good adaptability and scalability of the proposed framework to the underlying VLM capabilities.

[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An embodied question-answering method based on empirical active visual verification, characterized in that, Includes the following steps: Step S1: At each observation location of the embodied agent, capture multiple egocentric observation images covering different orientations, and combine them with historical snapshots and boundary rendering maps to construct a candidate image set; Step S2: Based on the visual embedding of the current problem and scene, obtain matching structured experience knowledge from the skill base through multimodal retrieval to generate experience context; Step S3: Execute an active visual verification loop of "selection-focus-back" based on the empirical context: filter images containing key evidence from the candidate image set, perform high-resolution scaling on the region of interest of the selected image to extract fine-grained visual evidence, and perform image-level or location-level hierarchical backoff when information is insufficient. Step S4: After the visual evidence obtained from active visual verification and the historical snapshot are filtered by the visual evidence manager, the responder determines whether the evidence is sufficient and generates an answer; if the evidence is insufficient, the planner selects the next navigation direction and switches to a new observation position, and repeats steps S1 to S4. Step S5: After a single exploration round ends, perform a three-level retrospective reflection on the complete exploration trajectory, distill the reflection results into structured skills and write them into the skill library, and simultaneously update and summarize the meta-skills that generate and store all skills and global general strategies.

2. The embodied question-answering method based on empirical active visual verification according to claim 1, characterized in that, The process of generating empirical context through multimodal retrieval in step S2 includes: Step S21: Simultaneously enable four channels—semantic similarity, question type matching, target object overlap, and scene visual similarity—to rank the skills in the skill library; Step S22: Calculate the overall skill score by fusing the four-channel sorting results using the reciprocal ranking fusion method. The scoring formula is as follows: in, The overall skill score is represented by c, which represents the four search channels, and k is a smoothing constant. Let i be the ranking of the i-th skill within channel c. For skill confidence; Step S23: Filter the Top-K matching skills based on the comprehensive score, and combine the global guidance rules of the meta-skills to hierarchically encapsulate and obtain the structured experience context.

3. The embodied question-answering method based on empirical active visual verification according to claim 1, characterized in that, The "select-focus-undo" active visual verification loop in step S3 includes three sub-skill processes: Step S31 Multi-view selection: Input the candidate image set, the current problem, and the empirical context into the visual language model, and output the optimal image index and selection confidence; when the confidence is lower than a preset threshold... At that time, multiple candidate images are retained for subsequent rollback; Step S32 Region scaling: Locate the coordinates of the region of interest in the selected image using a visual language model, and re-render a high-definition detail image based on the region's 3D pose to simulate optical zoom and obtain fine-grained visual evidence. Step S33 Layered rollback: When the scaling evaluation determines that the information is insufficient, a two-level rollback strategy is executed. The first level is image-level rollback, which involves switching to other candidate images at the same observation position and repeating the scaling verification. Level 2 is a position-level rollback, abandoning the current observation position and jumping to the navigation planning process.

4. The embodied question-answering method based on empirical active visual verification according to claim 1, characterized in that, The three-level retrospective reflection in step S5 includes: Task-level reflection: Evaluate the completion status of task objectives, identify key exploration nodes, and sort out the logical chain of reasoning; Environmental layer reflection: Discover rooms, areas and objects with high information value, and extract reusable spatial layout and navigation patterns; Hypothetical reflection: Based on counterfactual reasoning, we can deduce the effectiveness of alternative exploration strategies and expand the strategy search space; The results of the three-layer reflection are input into a visual language model for distillation to generate structured skills.

5. The embodied question-answering method based on empirical active visual verification according to claim 1, characterized in that, The structured skills obtained by distillation in step S5 are expressed using standardized units, and the expression is as follows: In the formula, These are the general trigger conditions for skills. For the set of condition-action-reasoning triple rules, For the visual embedding set of related scenes, , These are positive behavior suggestions and negative behavior constraints. Provides prompts for proactive visual operation. For the scope of skill application, It includes statistical information such as confidence level and success / failure count.

6. The embodied question-answering method based on empirical active visual verification according to claim 1, characterized in that, Step S1: Number of images acquired at each observation location The three images together cover a 360° panoramic view; the maximum number of zooms in a single observation position in step S3 is 3. Multi-view confidence threshold selection .

7. The embodied question-answering method based on empirical active visual verification according to claim 2, characterized in that, The weights of the four channels are configured as follows: semantic channel 0.3, visual channel 0.3, object channel 0.2, type channel 0.2, and smoothing constant. The default number of matching skills is [number]. .

8. The embodied question-answering method based on empirical active visual verification according to claim 3, characterized in that, The visual language model outputs the region of interest as continuous relative coordinates. Based on the 3D scene pose corresponding to these coordinates, high-resolution re-rendering is performed to generate detailed images. The scaled-down detailed images are stored in a historical snapshot set as evidence for reasoning.

9. The embodied question-answering method based on empirical active visual verification according to claim 5, characterized in that, Each skill can store a maximum of [number] records. Visual embedding of related scenes; similarity threshold for skill matching and merging. ; Meta-skills are distilled from the entire skill library, storing general exploration strategies and global active visual experience, and are updated synchronously with the skill library.

10. The embodied question-answering method based on empirical active visual verification according to claim 1, characterized in that, The maximum number of navigation steps in a single complete exploration round is limited to 50 steps; if sufficient visual evidence is not collected after traversing to the maximum number of steps, the exploration round is terminated and the three-level retrospective reflection process in step S5 is executed.

Images