An active sensing and target positioning method and system in a panoramic scene

Abstract

The application relates to an active perception and target positioning method and system in a panoramic scene, a spherical rendering environment of a panoramic image based on a real scene is constructed, and a multi-modal active perception and reasoning is performed by an intelligent agent in the spherical rendering environment; when a target has not appeared in a field of view, an explorer module is activated, a multi-modal large model is called based on a current field of view picture, a natural language target instruction and a dynamic memory bank retrieval result to perform reasoning; environment reference objects in the current field of view picture are identified, the position of the target relative to the current visual angle is inferred, visual angle adjustment instructions are continuously generated to guide the visual angle to approach the region where the target is located; the current field of view picture content and the reasoning result are written into the dynamic memory bank to form a node of a semantic clue chain; when the target enters the field of view, a tracker module is enabled to continuously position the target. Compared with the prior art, the application realizes deep space reasoning, and significantly improves the task adaptation capability of embodied intelligent equipment in a real environment.

Description

Technical Field

[0001] This invention relates to the fields of embodied intelligence and machine vision navigation technology, and in particular to an active perception and target localization method in panoramic scenes. Background Technology

[0002] With the rapid development of embodied intelligence and multimodal deep learning technologies, visual-language navigation and visual localization tasks have become research hotspots in the field of computer vision. Represented by the REVERIE dataset and related navigation models proposed by the University of Adelaide, current research has entered a stage requiring intelligent agents to perform long-range navigation and accurately locate targets in unknown indoor environments based on natural language instructions (such as: "Go to the study and find the red cup on the desk"). These tasks not only require models to possess strong cross-modal semantic understanding capabilities but also demonstrate broad application prospects for service robots and intelligent inspection systems in complex real-world scenarios.

[0003] However, existing visual navigation and search methods still have significant shortcomings in reproducing real physical environments and human behavior patterns. Current mainstream methods (such as most models based on the REVERIE benchmark) typically model the environment as a discrete topological graph, with the agent moving between predefined nodes in a jump-like fashion, often assuming the agent has a 360-degree panoramic view as input. This assumption is significantly out of sync with the hardware conditions of real robots. Real-world robots or drones are usually equipped with cameras with limited fields of view and require smooth perspective control in continuous space, rather than discrete jumps.

[0004] Furthermore, when processing instructions containing complex spatial orientation descriptions (e.g., "Find the chair to the left of the door"), existing models largely rely on end-to-end imitation learning, lacking explicit spatial reasoning and active exploration mechanisms. When the target is not within the current field of view, traditional models often fail to effectively utilize landmark objects in the instruction (e.g., doors) as intermediary cues for inference, resulting in blind rotation or spinning in place under limited field of view conditions, leading to low search efficiency and a high risk of losing the target. While some existing reinforcement learning-based exploration methods attempt to solve the path planning problem, they still lack a systematic solution for translating fine-grained spatial relationships in language into specific viewpoint actions (e.g., precise pitch or rotation amplitudes). Summary of the Invention

[0005] The purpose of this invention is to address the problems of existing visual navigation models, which generally rely on global maps and have difficulty understanding language commands containing complex spatial relationships, and are prone to low search efficiency and navigation loss in environments with limited field of view. This invention provides an active perception and target localization method and system for panoramic scenes.

[0006] The objective of this invention can be achieved through the following technical solutions: As a first aspect of the present invention, a method for active perception and target localization in a panoramic scene is provided, comprising the following steps: Construct a spherical rendering environment based on panoramic images of real-world scenes, and have an agent perform multimodal proactive perception and reasoning within this environment: A dynamic memory bank is constructed to store the agent's historical observation frames and corresponding reasoning records in chronological order, forming a semantic clue chain. When the target is not yet in the field of view, the explorer module is activated. The explorer module takes the current field of view, the natural language target command, and the dynamic memory retrieval results as input, and calls the multimodal large model to perform inference: identify environmental reference objects and their positions in the current field of view; combine the target spatial orientation description in the natural language command with the historical observation records of the reference objects in the dynamic memory and the corresponding view parameters to infer the target's orientation relative to the current view; generate a view adjustment command and execute the view movement; Each time a reasoning decision is made, the current visual content and reasoning record are immediately written into the dynamic memory bank, forming a new semantic clue node and adding it to the end of the semantic clue chain. Once the target comes into view, the tracker module is activated to continuously locate the target.

[0007] As a preferred technical solution, the spherical rendering environment based on panoramic images is constructed as follows: Capture panoramic images of real-world scenes and map the panoramic image onto a unit sphere using equidistant cylindrical projection; The robot renders a local viewpoint in real time based on its current yaw and pitch angle parameters, which serves as the observation input for the intelligent agent.

[0008] As a preferred technical solution, the semantic clue node records the following information: current yaw angle and pitch angle; types of environmental reference objects identified in the current image and their relative positions; description of the azimuth clues used in this reasoning; target azimuth judgment conclusion; and the viewpoint adjustment command output this time. Adjacent semantic clue nodes are sequentially connected through temporal indexes and the changes in perspective state are recorded to form a spatially coherent reasoning path.

[0009] As a preferred technical solution, the explorer module employs a group-relative strategy optimization for end-to-end training, specifically implemented as follows: For each training sample, multiple exploration trajectories are sampled from the same initial perspective to form a trajectory group; A hybrid reward function is designed to continuously optimize the explorer's perspective decision-making strategy by calculating the strategy gradient through relative rewards within a group. The hybrid reward function includes: an efficiency reward based on the exploration path length and a termination reward based on the target's positioning accuracy in the field of view. The efficiency reward is the sum of the efficiency reward values ​​of all trajectory points in the trajectory. The efficiency reward value of each trajectory point is calculated as follows: multiply the difference between the angle distance between the view center of the previous trajectory point and the target's true azimuth angle distance and the current view center and the target's true azimuth angle distance by the progress reward coefficient, and then subtract the step size penalty. The termination reward is specifically calculated as the angular distance between the center of the viewpoint at the end of the trajectory and the true orientation of the target. When the angular distance is less than the set distance threshold, the termination reward is the weighted sum of the preset success reward and the intersection-union ratio between the target bounding box generated by the tracker module and the real target bounding box; If the angular distance is greater than or equal to the set distance threshold, the reward will be terminated and the preset failure penalty value will be applied.

[0010] As a preferred technical solution, the tracker module takes the current frame as input, outputs the bounding box coordinates of the target through the target detection and visual positioning model, and further fine-tunes the viewing angle according to the offset of the bounding box center, so that the target gradually moves closer to the center of the screen; when the target bounding box meets the preset size and position conditions, the task is determined to be successfully completed, and the final target positioning result is output.

[0011] As a second aspect of the present invention, an active perception and target localization system in a panoramic scene is provided. The system constructs a spherical rendering environment based on panoramic images of a real scene, and an intelligent agent performs multimodal active perception and reasoning within this environment. The intelligent agent includes: The Explorer module is activated when the target is not yet in the agent's field of vision. It takes the current field of vision, natural language target instructions, and dynamic memory retrieval results as inputs and calls the multimodal large model to perform inference: identify environmental reference objects in the current field of vision; combine the target's spatial orientation description in the natural language instructions with historical memory information in the dynamic memory to infer the target's orientation relative to the current viewpoint; and continuously generate viewpoint adjustment instructions to guide the viewpoint toward the target's location. The dynamic memory bank stores the agent's historical observation frames and corresponding semantic descriptions, as well as the visual content and reasoning results of each inference by the explorer module. The tracker module is activated to locate the target once it enters the agent's field of vision.

[0012] As a preferred technical solution, the construction of the spherical rendering environment is specifically implemented as follows: Capture panoramic images of real-world scenes and map the panoramic image onto a unit sphere using equidistant cylindrical projection; The robot renders a local viewpoint in real time based on its current yaw and pitch angle parameters, which serves as the observation input for the intelligent agent.

[0013] As a preferred technical solution, the semantic clue node records the following information: current yaw angle and pitch angle; types of environmental reference objects identified in the current image and their relative positions; description of the azimuth clues used in this reasoning; target azimuth judgment conclusion; and the viewpoint adjustment command output this time. Adjacent semantic clue nodes are sequentially connected through temporal indexes and the changes in perspective state are recorded to form a spatially coherent reasoning path.

[0014] As a preferred technical solution, the explorer module employs a group-relative strategy optimization for end-to-end training, specifically implemented as follows: For each training sample, multiple exploration trajectories are sampled from the same initial perspective to form a trajectory group; Design a hybrid reward function to calculate the policy gradient through relative rewards within the group and continuously optimize the explorer's perspective decision-making strategy; The hybrid reward function includes: Efficiency rewards based on the length of the exploration path and termination rewards based on the accuracy of the target's location in the field of view; The efficiency reward is the sum of the efficiency reward values ​​of all trajectory points in the trajectory. The efficiency reward value of each trajectory point is calculated as follows: multiply the difference between the angle distance between the view center of the previous trajectory point and the target's true azimuth angle distance and the current view center and the target's true azimuth angle distance by the progress reward coefficient, and then subtract the step size penalty. The termination reward is specifically calculated as the angular distance between the center of the viewpoint at the end of the trajectory and the true orientation of the target. When the angular distance is less than the set distance threshold, the termination reward is the weighted sum of the preset success reward and the intersection-union ratio between the target bounding box generated by the tracker module and the real target bounding box; If the angular distance is greater than or equal to the set distance threshold, the reward will be terminated and the preset failure penalty value will be applied.

[0015] As a preferred technical solution, the tracker module takes the current frame as input, outputs the bounding box coordinates of the target through the target detection and visual positioning model, and further fine-tunes the viewing angle according to the offset of the bounding box center, so that the target gradually moves closer to the center of the screen; when the target bounding box meets the preset size and position conditions, the task is determined to be successfully completed, and the final target positioning result is output.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1) This invention proposes an active perception and target localization method and system for panoramic scenes, effectively bridging the perception gap between virtual simulation and real physical hardware. Unlike traditional navigation models that rely on panoramic views or discrete topological maps and move in a jump-like manner, this invention simulates the limited field of view and continuous motion space of a real robot through spherical panoramic rendering technology. The intelligent agent must actively rotate and adjust its pitch perspective to gradually acquire environmental information. This mechanism not only allows the model to be directly transferred to actual robot hardware, but also fundamentally solves the problem of decreased positioning accuracy and blind searching caused by information loss when the intelligent agent cannot obtain global observations, significantly improving the task adaptability of embodied intelligent devices in real environments.

[0017] 2) This invention achieves deep spatial reasoning through semantic cue chains, significantly improving the model's zero-shot target localization capability and search efficiency in complex scenes. Through the collaborative efforts of adaptive memory filtering and fine-tuning of a multimodal large model, the agent possesses strong logical inference capabilities. When faced with instructions containing complex directional relationships, the system no longer relies on simple end-to-end mapping but can autonomously identify road signs in the scene, gradually construct semantic cue chains based on historical memory, and achieve dynamic path planning that involves reasoning while observing.

[0018] 3) This invention employs a group-based policy optimization reinforcement learning algorithm to replace traditional supervised fine-tuning, coupled with a designed hybrid reward function, enabling the model to possess stronger long-term planning capabilities. In unseen complex indoor environments, this method can lock onto targets with shorter paths and fewer redundant actions, demonstrating good robustness in both task success rate and navigation efficiency. Attached Figure Description

[0019] Figure 1 This is a flowchart of an active perception and target localization method for panoramic scenes proposed in this invention.

[0020] Figure 2 This is a flowchart illustrating the deep reinforcement learning-based training method proposed in this invention.

[0021] Figure 3 This is a flowchart illustrating the multimodal active perception and semantic reasoning method of the present invention in a real panoramic environment. Detailed Implementation

[0022] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0023] Example 1 This invention proposes an active perception and target localization method in panoramic scenes. By constructing a continuous simulation environment based on spherical panoramic rendering, it introduces a hierarchical collaborative framework of "search-reasoning-tracking" and a group relative policy optimization (GRPO) reinforcement learning training mechanism. This enables the agent to actively adjust its perspective and gradually build semantic clue chains based on the spatial orientation description in natural language instructions when a global map is unavailable. This allows for efficient localization of target objects, providing a reliable, practical, and highly generalizable technical solution for the autonomous navigation and human-computer interaction of embodied intelligent robots.

[0024] The technical solution of this invention consists of two core parts: a limited field-of-view simulation environment based on spherical panoramic images and a multimodal active perception and reasoning framework operating in this environment, such as... Figure 1 As shown, the specific steps are as follows: S1. Construction of a spherical panoramic rendering environment.

[0025] Using panoramic images captured from real indoor scenes as data sources, the panoramic image is mapped onto a unit sphere through equidistant cylindrical projection. Based on given yaw and pitch angle parameters, a local viewpoint corresponding to the current perspective is rendered in real time, serving as the agent's observation input. This rendering process supports continuous adjustment of the viewpoint, and the rendering frame resolution and field of view (FoV) can be flexibly configured according to the target hardware specifications to adapt to the camera parameters of different robot platforms. Simultaneously, a dynamic memory module is constructed to store the agent's historical observation frames and corresponding semantic descriptions in chronological order, forming a semantic cue chain for retrieval and recall in subsequent reasoning steps.

[0026] S2. Semantic reasoning and perspective decision-making in the explorer module.

[0027] The explorer module is activated when the target object is not yet in the current frame. This module takes the current local frame, the natural language target command, and the dynamic memory retrieval results as input, and calls a multimodal large model fine-tuned with domain data to perform the following inferences sequentially: First, it identifies environmental references in the current frame (such as door frames, sofas, corners, etc.); then, combining the spatial orientation descriptions in the natural language target command (such as "left side of the door," "directly above the table"), it infers the approximate orientation of the target relative to the current viewpoint; finally, it outputs the next viewpoint adjustment command, including the horizontal rotation angle. With pitch adjustment After each reasoning decision is completed, the system creates a semantic cue node in the dynamic memory and appends it to the end of the semantic cue chain. Each semantic cue node records the following information: current yaw angle. With pitch angle The types and relative positions of environmental reference objects identified in the current scene; the description of the directional clues used in this reasoning; the conclusion of the target's location judgment; and the viewpoint adjustment command output this time. , Adjacent semantic cue nodes are sequentially connected via temporal indexes and record changes in viewpoint state, forming a spatially coherent reasoning path. Before each new reasoning initiation, the explorer module performs semantic similarity retrieval on the dynamic memory, extracts relevant historical semantic cue nodes to form a context summary, and inputs it into the model along with the current screen and the natural language target command. The application of the semantic cue chain is reflected in two aspects: first, by comparing the viewpoint parameters and reference information recorded in historical nodes, it avoids repeatedly exploring observed areas; second, it supports multi-step spatial reasoning mediated by reference objects, the specific process of which is as follows: Step 1: Identify environmental reference objects and their positions in the current image; Step 2: Search historical clue chain nodes to find historical observation records and corresponding perspective parameters of reference objects involved in natural language target instructions; Step 3: Combine the current frame reference position with historical records to infer the target's orientation relative to the current viewpoint.

[0028] Step 4, output the view adjustment command ( , The perspective shifts, and the current reasoning record is written into the dynamic memory to form a new semantic clue node.

[0029] The above steps are repeated iteratively until the target comes into view.

[0030] S3. Target localization and locking of the tracker module.

[0031] When a target object appears in the current frame, the tracker module is activated. Taking the current frame as input, the tracker module outputs the bounding box coordinates of the target using a target detection and visual localization model. It then fine-tunes the viewing angle based on the offset of the bounding box center, gradually bringing the target closer to the center of the frame. When the target bounding box meets the preset size and position conditions, the task is considered successfully completed, and the final target localization result is output.

[0032] S4. Reinforcement learning training based on the GRPO algorithm.

[0033] Using the spherical rendering environment constructed in step S1 as the training scene, the explorer module is trained end-to-end using the Group Relative Policy Optimization (GRPO) algorithm. For each training sample, multiple exploration trajectories are sampled from the same starting viewpoint to form a trajectory group. A hybrid reward function is designed to comprehensively evaluate the exploration path length (efficiency reward) and the target's positioning accuracy in the field of view (termination reward). The policy gradient is calculated through the relative reward within the group to continuously optimize the explorer's viewpoint decision-making strategy, enabling it to autonomously learn efficient active search behavior without expert trajectory demonstration. For the trajectory... Mixed rewards It consists of two parts: efficiency reward and termination reward. in, Indicates the first t The state of the agent during the step; Indicates the first t The actions performed by the intelligent agent during the step; This represents the final state of the agent when the trajectory terminates.

[0034] Efficiency Rewards The agent is encouraged to gradually shorten the distance to the target, and a fixed step size penalty is imposed to suppress redundant exploration. in, Centered on the current perspective The target's true location, and the progress reward coefficient. Step length penalty , This represents the angular distance between the current viewpoint center and the target's true location.

[0035] Termination of rewards Evaluate positioning accuracy upon mission termination: In this embodiment, a success reward IoU weight , Target bounding boxes generated for the tracker module Bounding box of the real target Intersection over union ratio, distance threshold Failure penalty .

[0036] Table 1 shows a performance comparison of different methods on the panoramic target localization task. SR represents the task success rate, AS represents the average number of exploration steps (lower is better), SPL represents the path length-weighted success rate, and mIoU represents the average intersection-union ratio. Higher values ​​for SR, SPL, and mIoU are better, while lower values ​​for AS are better.

[0037] Table 1. Performance comparison of different methods on panoramic target localization tasks. * Static methods for panoramas Figure 1 One-time processing, SPL is not applicable.

[0038] † Heuristic scans are performed in a fixed order: upper hemisphere → equator → lower hemisphere.

[0039] Example 2 As another embodiment of the present invention, this embodiment also provides an active perception and target localization system in a panoramic scene. This system executes the active perception and target localization method described in Embodiment 1 above, constructs a spherical rendering environment based on panoramic images of the real scene, and an intelligent agent performs multimodal active perception and reasoning within the spherical rendering environment. The intelligent agent includes: The Explorer module is activated when the target is not yet in the agent's field of vision. It takes the current field of vision, natural language target instructions, and dynamic memory retrieval results as inputs and calls the multimodal large model to perform inference: identify environmental reference objects in the current field of vision; combine the target's spatial orientation description in the natural language instructions with historical memory information in the dynamic memory to infer the target's orientation relative to the current viewpoint; and continuously generate viewpoint adjustment instructions to guide the viewpoint toward the target's location. Furthermore, the explorer module employs group-based relative strategy optimization for end-to-end training. For each training sample, multiple exploration trajectories are sampled from the same initial viewpoint to form a trajectory group. A hybrid reward function is designed to continuously optimize the explorer's viewpoint decision-making strategy by calculating the policy gradient through intra-group relative rewards. The hybrid reward function includes: efficiency rewards for exploration path length and termination rewards for target positioning accuracy within the field of view.

[0040] The dynamic memory bank stores the agent's historical observation frames and corresponding semantic descriptions, as well as the visual content and reasoning results of each inference by the explorer module. The tracker module is activated to locate the target once it enters the agent's field of view. Taking the current frame as input, the tracker module outputs the target's bounding box coordinates using a target detection and visual localization model. It further fine-tunes the viewing angle based on the offset of the bounding box center, gradually bringing the target closer to the center of the image. When the target's bounding box meets preset size and position conditions, the task is considered successfully completed, and the final target localization result is output.

[0041] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0042] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for active perception and target localization in panoramic scenes, characterized by the following steps: include: Construct a spherical rendering environment based on panoramic images of real-world scenes, and have an agent perform multimodal proactive perception and reasoning within this environment: A dynamic memory bank is constructed to store the agent's historical observation frames and corresponding reasoning records in chronological order, forming a semantic clue chain. When the target has not yet appeared in the field of view, the explorer module is activated. The explorer module takes the current field of view, the natural language target command, and the dynamic memory retrieval results as input, and calls the multimodal large model to perform inference: identify environmental reference objects and their positions in the current field of view; combine the target spatial orientation description in the natural language command with the historical observation records of the reference objects in the dynamic memory and the corresponding viewpoint parameters to infer the orientation of the target relative to the current viewpoint. Generate view adjustment commands and execute view movement; Each time a reasoning decision is made, the current visual content and reasoning record are immediately written into the dynamic memory bank, forming a new semantic clue node and adding it to the end of the semantic clue chain. Once the target comes into view, the tracker module is activated to continuously locate the target.

2. The active perception and target localization method in a panoramic scene according to claim 1, characterized in that, The spherical rendering environment is constructed as follows: Capture panoramic images of real-world scenes and map the panoramic image onto a unit sphere using equidistant cylindrical projection; The robot renders a local viewpoint in real time based on its current yaw and pitch angle parameters, which serves as the observation input for the intelligent agent.

3. The active perception and target localization method in a panoramic scene according to claim 1, characterized in that, The semantic clue node records the following information: current yaw and pitch angles; types of environmental reference objects identified in the current image and their relative positions; description of the orientation clues used in this inference; target orientation judgment conclusion; and the viewpoint adjustment command output this time. Adjacent semantic clue nodes are sequentially connected through temporal indexes and the changes in perspective state are recorded to form a spatially coherent reasoning path.

4. The active perception and target localization method in a panoramic scene according to claim 1, characterized in that, The explorer module employs a group-relative strategy optimization for end-to-end training, as specifically implemented below: For each training sample, multiple exploration trajectories are sampled from the same initial perspective to form a trajectory group; Design a hybrid reward function to calculate the policy gradient through relative rewards within the group and continuously optimize the explorer's perspective decision-making strategy; The hybrid reward function includes: an efficiency reward based on the length of the exploration path and a termination reward based on the accuracy of the target's location in the field of view; The efficiency reward is the sum of the efficiency reward values ​​of all trajectory points in the trajectory. The efficiency reward value of each trajectory point is calculated as follows: multiply the difference between the angle distance between the view center of the previous trajectory point and the target's true azimuth angle distance and the current view center and the target's true azimuth angle distance by the progress reward coefficient, and then subtract the step size penalty. The termination reward is specifically calculated as the angular distance between the center of the viewpoint at the end of the trajectory and the true orientation of the target. When the angular distance is less than the set distance threshold, the termination reward is the weighted sum of the preset success reward and the intersection-union ratio between the target bounding box generated by the tracker module and the real target bounding box; If the angular distance is greater than or equal to the set distance threshold, the reward will be terminated and the preset failure penalty value will be applied.

5. The active perception and target localization method in a panoramic scene according to claim 1, characterized in that, The tracker module takes the current frame as input, outputs the bounding box coordinates of the target through the target detection and visual positioning model, and further fine-tunes the viewing angle according to the offset of the bounding box center, so that the target gradually moves closer to the center of the screen; when the target bounding box meets the preset size and position conditions, the task is determined to be successfully completed, and the final target positioning result is output.

6. An active perception and target localization system for panoramic scenes, characterized in that, The system constructs a spherical rendering environment based on panoramic images of real-world scenes. An intelligent agent performs multimodal active perception and reasoning within this environment. The intelligent agent includes: The explorer module is activated when the target is not yet in the agent's field of vision. It takes the current field of vision, natural language target instructions, and dynamic memory retrieval results as inputs and calls the multimodal large model to perform inference: identify environmental reference objects in the current field of vision; combine the target's spatial orientation description in the natural language instructions with historical memory information in the dynamic memory to infer the target's orientation relative to the current viewpoint; and continuously generate viewpoint adjustment instructions to guide the viewpoint toward the target's location. The dynamic memory bank stores the agent's historical observation frames and corresponding semantic descriptions, as well as the visual content and reasoning results of each inference by the explorer module. The tracker module is activated to locate the target once it enters the agent's field of vision.

7. The active perception and target localization system in a panoramic scene according to claim 6, characterized in that, The specific implementation of constructing the spherical rendering environment is as follows: Capture panoramic images of real-world scenes and map the panoramic image onto a unit sphere using equidistant cylindrical projection; The robot renders a local viewpoint in real time based on its current yaw and pitch angle parameters, which serves as the observation input for the intelligent agent.

8. The active perception and target localization system in a panoramic scene according to claim 6, characterized in that, The semantic cue node records the following information: current yaw and pitch angles; types of environmental reference objects identified in the current image and their relative positions; description of the orientation cue used in this inference; target orientation judgment conclusion; and the viewpoint adjustment command output this time. Adjacent semantic clue nodes are sequentially connected through temporal indexes and the changes in perspective state are recorded to form a spatially coherent reasoning path.

9. The active perception and target localization system in a panoramic scene according to claim 6, characterized in that, The explorer module employs a group-relative strategy optimization for end-to-end training, as specifically implemented below: For each training sample, multiple exploration trajectories are sampled from the same initial perspective to form a trajectory group; Design a hybrid reward function to calculate the policy gradient through relative rewards within the group and continuously optimize the explorer's perspective decision-making strategy; The hybrid reward function includes: Efficiency rewards based on the length of the exploration path and termination rewards based on the accuracy of the target's location in the field of view; The efficiency reward is the sum of the efficiency reward values ​​of all trajectory points in the trajectory. The efficiency reward value of each trajectory point is calculated as follows: multiply the difference between the angle distance between the view center of the previous trajectory point and the target's true azimuth angle distance and the current view center and the target's true azimuth angle distance by the progress reward coefficient, and then subtract the step size penalty. The termination reward is specifically calculated as the angular distance between the center of the viewpoint at the end of the trajectory and the true orientation of the target. When the angular distance is less than the set distance threshold, the termination reward is the weighted sum of the preset success reward and the intersection-union ratio between the target bounding box generated by the tracker module and the real target bounding box; If the angular distance is greater than or equal to the set distance threshold, the reward will be terminated and the preset failure penalty value will be applied.

10. The active perception and target localization system in a panoramic scene according to claim 6, characterized in that, The tracker module takes the current frame as input, outputs the bounding box coordinates of the target through the target detection and visual positioning model, and further fine-tunes the viewing angle according to the offset of the bounding box center, so that the target gradually moves closer to the center of the screen; when the target bounding box meets the preset size and position conditions, the task is determined to be successfully completed, and the final target positioning result is output.

Images