An autonomous roaming method in a 3D Gaussian splash scene
By acquiring multi-dimensional features in a 3D Gaussian splash scene and inputting them into a lightweight Actor-Critic network, the problems of high cost and error accumulation in existing technologies are solved, and an efficient and stable autonomous roaming method is realized.
Patent Information
- Application Number
- CN202611419068.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-14
- Publication Date
- 2026-11-17
AI Technical Summary
In existing autonomous roaming methods for 3D Gaussian splash scenes, directly using rendered images as policy input leads to high training and inference costs. Furthermore, pure behavior cloning training methods are prone to accumulating errors due to state distribution shifts, making it difficult to effectively adapt across scenes.
By acquiring multi-dimensional directional detection features and basic scalar features, they are concatenated into a scalar state vector and input into a lightweight fully connected multilayer perceptron Actor-Critic network. Combined with behavior cloning pre-training and proximal policy optimization algorithms, online closed-loop fine-tuning is performed to reduce visual encoding requirements and alleviate error accumulation.
It reduces the need for high-dimensional visual encoding in the policy network, lowers computational resource consumption, and reduces the risk of getting stuck through rollback and forced escape mechanisms, thereby improving the efficiency and stability of autonomous roaming.
Smart Images

Figure CN123118982A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to an autonomous roaming method in a 3D Gaussian splash scene. Background Technology
[0002] 3D Gaussian Splatter (3DGS) uses a large number of 3D Gaussian splats with parameters for position, scale, rotation, transparency, and color to represent a scene. It can synthesize images with a specified camera pose using differentiable or real-time rasterization. In a 3DGS scene, autonomous roaming refers to the automatic control of the virtual camera's position and orientation by a strategy to generate a continuous, viewable camera trajectory that covers as much of the scene content as possible.
[0003] However, current technologies directly use rendered images as policy input, which typically requires convolutional neural networks to process tens of thousands of pixels of data, resulting in high training and inference costs. Furthermore, variations in texture, color, and lighting in images may cause the policy to learn correlations unrelated to spatial exploration, and differences in appearance and Splat density between different scenes further increase the difficulty of cross-scene training.
[0004] The training method used in existing policy networks is generally pure behavior cloning training. Although pure behavior cloning can learn basic motion patterns from demonstration trajectories, errors will accumulate during closed-loop execution due to state distribution shifts. Summary of the Invention
[0005] In view of this, the purpose of this application is to provide an autonomous roaming method in a 3D Gaussian splash scene, which can effectively solve the technical problems in the prior art.
[0006] This application provides an autonomous roaming method for a 3D Gaussian splash scene, the autonomous roaming method comprising: Acquire the target 3D Gaussian splash scene data and the current camera parameters; The target 3D Gaussian splash scene data and current camera parameters are processed to determine multi-dimensional directional detection features and multi-dimensional basic scalar features describing the camera's own motion state, the view quality of the current main viewpoint, and geometric safety information. The multidimensional basic scalar features and the multidimensional orientation detection features are concatenated into a scalar state vector, which is then input into a pre-trained target policy network to output the multidimensional pose change of the camera. The target policy network is formed by pre-training with behavior cloning and online closed-loop fine-tuning with a proximal policy optimization algorithm. The camera pose is updated based on the multidimensional pose change, and the feasibility of the candidate camera poses determined after the update is checked. If the check is successful, the current camera parameters of the camera are updated according to the candidate camera pose, and the above steps are repeated until the autonomous roaming termination condition is met.
[0007] Optionally, the processing of the target 3D Gaussian splash scene data and current camera parameters to determine multi-dimensional directional detection features and multi-dimensional basic scalar features describing the camera's own motion state, the view quality of the current main viewpoint, and geometric safety information includes: Based on the current camera parameters, the target 3D Gaussian splash scene is rendered from the main viewpoint. The no-reference rendering validity score, the number of visible Gaussian primitives in the current viewpoint, and the distance to the nearest valid Gaussian primitive are calculated. The linear motion vector and angular motion vector of the camera are calculated by combining historical camera parameters. The validity history score and validity trend are obtained by combining historical validity data. The continuous invalid count is obtained by combining continuous invalid action records. Together, they form the multi-dimensional basic scalar feature. Based on the current camera orientation, multiple horizontal yaw detection directions and pitch detection directions are generated. Detection rendering is performed on each detection direction to obtain the number of visible Gaussian pixels and the distance to the nearest Gaussian pixel in each direction. After in-sample relative normalization, the multi-dimensional direction detection features are formed.
[0008] Optionally, the multidimensional basic scalar feature is a 15-dimensional basic scalar feature, specifically including: 5-dimensional validity history score, 1-dimensional validity trend, 1-dimensional number of visible Gaussian units from the current viewpoint, 1-dimensional distance to the nearest valid Gaussian unit, 3-dimensional linear motion vector, 3-dimensional angular motion vector, and 1-dimensional consecutive invalid count. The multi-dimensional orientation detection feature is a 28-dimensional orientation detection feature, specifically including: based on the current camera orientation, performing detection and rendering on 12 horizontal yaw directions and 2 pitch directions respectively, obtaining the number of visible Gaussian pixels and the nearest Gaussian pixel distance in each direction; normalizing the number of visible Gaussian pixels and the nearest Gaussian pixel distance in each direction to obtain the 28-dimensional orientation detection feature.
[0009] Optionally, the step of concatenating the multidimensional basic scalar features and the multidimensional orientation detection features into a scalar state vector and inputting it into a pre-trained target policy network to output the multidimensional pose change of the camera includes: The multidimensional basic scalar features and the multidimensional directional detection features are concatenated in a fixed order to form a 43-dimensional scalar state vector; The 43-dimensional scalar state vector is input into the target policy network to predict camera pose changes, outputting a 6-dimensional continuous action, which is then defined as the multi-dimensional pose change. The target policy network is a fully connected multilayer perceptron Actor-Critic structure, and the 6-dimensional continuous action consists of three dimensions: the first three dimensions are translation commands in the local camera coordinate system, and the last three dimensions are axis-angle rotation commands.
[0010] Optionally, the multidimensional pose change includes translation increment and rotation increment. The step of updating the camera pose based on the multidimensional pose change and performing a feasibility check on the updated candidate camera pose includes: Update the camera's position and orientation based on the translation and rotation increments to generate candidate camera poses; Perform a pre-collision check based on the depth data rendered from the main viewpoint of the previous frame to determine if there is a risk of collision; Perform full rendering of the candidate camera poses, and perform validity constraint verification based on the rendering results; If both items pass, the feasibility check is deemed satisfactory; if either item fails, the feasibility check is deemed unsatisfactory.
[0011] Optionally, if the inspection fails, the autonomous roaming method further includes: Backtracking along the accepted historical camera pose sequence, the system regresses to the most recent safe pose that satisfies the feasibility check as the current camera parameter, and continues to perform the next valid camera pose prediction starting from the safe pose. If the number of consecutive backtracking steps reaches a preset threshold and a valid pose is still not obtained, a forced escape operation is triggered.
[0012] Optionally, the target policy network is constructed through the following steps; Prepare an expert trajectory dataset; the expert trajectory dataset includes roaming trajectories recorded by human interaction or roaming trajectories generated by converting existing camera keyframe paths, and each trajectory contains scalar states and corresponding action data under the corresponding pose; Based on the expert trajectory dataset, behavior cloning pre-training is performed to enable the initial policy network to learn basic roaming motion patterns, thus obtaining the pre-trained policy network. Using a pre-trained policy network as initial parameters, an online closed-loop fine-tuning algorithm is performed in a 3D Gaussian splash scene using a proximal policy optimization algorithm. The roaming effect is optimized through a dense reward function to obtain the final target policy network.
[0013] Optionally, the step of performing behavior cloning pre-training based on the expert trajectory dataset, enabling the initial policy network to learn basic roaming motion patterns and obtain a pre-trained policy network, includes: Using the state-action pairs in the expert trajectories of the expert trajectory dataset as supervised samples, a weighted Gaussian negative log-likelihood loss is adopted as the behavior cloning loss, and a standard deviation lower bound term and an entropy reward term are added to the behavior cloning loss; wherein, the weighted Gaussian negative log-likelihood loss weights each action dimension according to the dimension weight, and jointly trains the mean output of the policy network and the shared log-standard deviation parameter; the standard deviation lower bound term is used to penalize the standard deviation below the preset lower bound, and the entropy reward term is used to reward the entropy of the action distribution; The AdamW optimizer is used to optimize the behavior cloning loss, and the upper limit of the gradient norm and the cosine hot restart learning rate are set. The parameters of the initial policy network are iteratively updated to obtain the pre-trained policy network.
[0014] Optionally, the step of using a pre-trained policy network as initial parameters, performing online closed-loop fine-tuning in a 3D Gaussian splash scene using a proximal policy optimization algorithm, and optimizing the roaming effect through a dense reward function to obtain the final target policy network includes: Using the pre-trained policy network output by the behavior clone as the training starting point, autonomous roaming closed-loop interaction is performed in a 3D Gaussian splash scene. The instantaneous reward for each step is calculated through a preset dense reward function, and trajectory data of a fixed number of steps is collected. The trajectory data includes the scalar state, the executed action, the instantaneous reward, and the scalar state at the next moment. For trajectories that have reached the maximum number of steps, the timeout is truncated, and the terminal state is processed by value function bootstrapping to complete the reward estimation. Based on the collected trajectory data, the generalized dominance estimate at each time point is calculated, and the obtained dominance values are standardized. The total loss function is constructed based on the standardized advantage value, and the parameters of the pre-trained policy network are updated by backpropagation after setting the upper limit of the gradient norm. Repeat the above steps of closed-loop interactive sampling, advantage estimation calculation and policy iterative update until the roaming performance index meets the preset convergence condition to obtain the final target policy network.
[0015] Optionally, when fine-tuning using the near-end policy optimization algorithm, the dense reward function used includes: effectiveness score term, effectiveness change term, farthest distance progress term, actual movement term, continuous stationary penalty term, heading consistency term, visible quantity term, catastrophic descent penalty term, revisit penalty term, and constant cost term.
[0016] This application embodiment also provides a structural schematic diagram of an autonomous roaming device for a 3D Gaussian splash scene. The autonomous roaming device includes: The acquisition module is used to acquire target 3D Gaussian splash scene data and the current camera parameters; The processing module is used to process the target 3D Gaussian splash scene data and the current camera parameters to determine multi-dimensional directional detection features and multi-dimensional basic scalar features that describe the camera's own motion state, the view quality of the current main viewpoint, and geometric safety information. The prediction module is used to concatenate the multidimensional basic scalar features and the multidimensional orientation detection features into a scalar state vector, and input it into a pre-trained target policy network to output the multidimensional pose change of the camera; wherein, the target policy network is formed by pre-training with behavior cloning and online closed-loop fine-tuning with a proximal policy optimization algorithm; The inspection module is used to update the camera pose based on the multidimensional pose change and to perform a feasibility check on the candidate camera poses determined after the update. The update module is used to update the current camera parameters of the camera according to the candidate camera pose if the check is qualified, and repeat the above steps until the autonomous roaming termination condition is met.
[0017] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the autonomous roaming method described above are performed.
[0018] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the autonomous roaming method described above.
[0019] This application provides an autonomous roaming method for a 3D Gaussian splash scene. The autonomous roaming method includes: acquiring target 3D Gaussian splash scene data and current camera parameters; processing the target 3D Gaussian splash scene data and current camera parameters to determine multi-dimensional directional detection features and multi-dimensional basic scalar features describing the camera's own motion state, the view quality of the current main viewpoint, and geometric safety information; concatenating the multi-dimensional basic scalar features and the multi-dimensional directional detection features into a scalar state vector and inputting it into a pre-trained target policy network to output the multi-dimensional pose change of the camera; wherein the target policy network is formed by pre-training with behavior cloning and online closed-loop fine-tuning with a proximal policy optimization algorithm; updating the camera pose according to the multi-dimensional pose change, and performing a feasibility check on the candidate camera poses determined after the update; if the check is qualified, updating the current camera parameters according to the candidate camera poses, and repeating the above steps until the autonomous roaming termination condition is met.
[0020] Thus, this application reduces the need for high-dimensional visual encoding on the policy side by fixing the policy input to 43 scalars and not inputting RGB or depth tensors into the policy network. Furthermore, this scheme allows action and value estimation to be completed using a small number of parameters in the MLPActor-Critic network. 14-direction detection enables the policy to obtain content and proximity information in the horizontal and vertical directions outside the current field of view; intra-sample relative normalization reduces the direct impact of absolute Gaussian density differences on direction selection. Moreover, the target policy network provided in this application is formed by pre-training with behavioral cloning and online closed-loop fine-tuning using a proximal policy optimization algorithm. Behavioral cloning provides feasible movement patterns for the policy, and the proximal policy optimization algorithm (PPO) handles the state after deviation from the demonstration through closed-loop reward processing. Furthermore, the backoff and forced escape mechanisms in this application reduce the risk of getting stuck due to consecutive invalid actions.
[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 A flowchart illustrating an autonomous roaming method in a 3D Gaussian splash scene provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of an autonomous roaming device in a 3D Gaussian splash scene provided in an embodiment of this application; Figure 3 A schematic diagram of the structure of another autonomous roaming device in a 3D Gaussian splash scene provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.
[0025] 3D Gaussian Splatter (3DGS) uses a large number of 3D Gaussian splats with parameters for position, scale, rotation, transparency, and color to represent a scene. It can synthesize images with a specified camera pose using differentiable or real-time rasterization. In a 3DGS scene, autonomous roaming refers to the automatic control of the virtual camera's position and orientation by a strategy to generate a continuous, viewable camera trajectory that covers as much of the scene content as possible.
[0026] However, current technologies directly use rendered images as policy input, which typically requires convolutional neural networks to process tens of thousands of pixels of data, resulting in high training and inference costs. Furthermore, variations in texture, color, and lighting in images may cause the policy to learn correlations unrelated to spatial exploration, and differences in appearance and Splat density between different scenes further increase the difficulty of cross-scene training.
[0027] The training method used in existing policy networks is generally pure behavior cloning training. Although pure behavior cloning can learn basic motion patterns from demonstration trajectories, errors will accumulate during closed-loop execution due to state distribution shifts.
[0028] Based on this, this application provides an autonomous roaming method in a 3D Gaussian splash scene, which expresses the validity of the current view, the distribution of directional content, nearby obstacles and motion state without inputting RGB images and depth maps into the policy network; and reduces the impact of the order-of-magnitude difference in Splat in different 3DGS scenes on the policy; and reduces the cost of random exploration and alleviates the closed-loop cumulative error of behavior cloning by combining expert demonstration and environmental interaction.
[0029] Please see Figure 1 , Figure 1 This is a flowchart illustrating an autonomous roaming method in a 3D Gaussian splash scene, as provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the autonomous roaming method includes: S101. Obtain the target 3D Gaussian splash scene data and the current camera parameters; S102. Process the target 3D Gaussian splash scene data and the current camera parameters to determine multi-dimensional directional detection features and multi-dimensional basic scalar features that describe the camera's own motion state, the view quality of the current main viewpoint, and geometric safety information. S103. The multidimensional basic scalar features and the multidimensional orientation detection features are concatenated into a scalar state vector and input into the pre-trained target policy network to output the multidimensional pose change of the camera. S104. Update the camera pose based on the multidimensional pose change, and perform a feasibility check on the candidate camera poses determined after the update. S105. If the inspection is qualified, update the current camera parameters of the camera according to the candidate camera pose, and repeat the above steps until the autonomous roaming termination condition is met.
[0030] The exemplary steps of the embodiments of this application are described below: Regarding step S101, in this embodiment of the application, the target 3D Gaussian splash scene data refers to the scene representation obtained after reconstructing the real scene using 3D Gaussian splash technology. It contains a large number of three-dimensional Gaussian primitives, each with parameters such as position, scale, rotation, transparency, and color. This scene data can be stored as a Gaussian set in the form of a point cloud, and supports the synthesis of corresponding two-dimensional images using differential rasterization or real-time rasterization methods at a given camera pose.
[0031] The camera's current camera parameters include at least its position and orientation in the world coordinate system. A homogeneous transformation matrix T is typically used. t ∈R 4×4 Let R represent the transformation from the camera to the world coordinate system at step t, with rotation matrix and position respectively. t With P t The local x, y, and z axes of the camera correspond to the horizontal, vertical, and forward directions, respectively. The camera's upward direction in the initial pose is normalized and used as the world direction for yaw, pitch, and roll correction. RGB rendering values are processed in the range [0,1], distances are in meters, and rotation angles are calculated using radians.
[0032] At the initial moment, the camera parameters can be a manually specified initial pose, the starting pose of an expert trajectory, or a preset fixed observation point. The camera's upward direction in the initial pose is normalized and used as the world direction for subsequent yaw, pitch, and roll processing.
[0033] Regarding step S102, in one embodiment provided in this application, the processing of the target 3D Gaussian splash scene data and current camera parameters to determine multi-dimensional directional detection features and multi-dimensional basic scalar features describing the camera's own motion state, the view quality of the current main viewpoint, and geometric safety information includes: S1021. Perform main-view rendering on the target 3D Gaussian splash scene based on the current camera parameters, calculate the no-reference rendering validity score of the current main view, the number of visible Gaussian primitives in the current view, and the distance to the nearest valid Gaussian primitive; calculate the linear motion vector and angular motion vector of the camera by combining historical camera parameters, obtain the validity history score and validity trend by combining historical validity data, and obtain the continuous invalid count by combining continuous invalid action records, which together constitute the multi-dimensional basic scalar feature; S1022. Based on the current camera orientation, generate multiple horizontal yaw detection directions and pitch detection directions. Perform detection rendering on each detection direction to obtain the number of visible Gaussian pixels and the distance to the nearest Gaussian pixel in each direction. After in-sample relative normalization, the multi-dimensional direction detection features are formed.
[0034] For step S1021, the multidimensional basic scalar feature is a 15-dimensional basic scalar feature, specifically including: 5-dimensional validity history score, 1-dimensional validity trend, 1-dimensional number of visible Gaussian units from the current viewpoint, 1-dimensional distance to the nearest valid Gaussian unit, 3-dimensional linear motion vector, 3-dimensional angular motion vector, and 1-dimensional consecutive invalid count.
[0035] The determination process of the multidimensional fundamental scalar features is specifically determined through the following steps: First, the target 3D Gaussian splash scene is rendered under the camera pose corresponding to the current camera parameters to obtain the current rendered image. A no-reference rendering validity score is then calculated based on this rendered image. This score does not depend on the ground truth image, but is obtained by weighting four sub-scores: gray-level variance, edge information, non-black coverage, and spatial uniformity.
[0036] For example, for a non-empty RGB image, the grayscale image is calculated first. ; Then calculate the grayscale variance fraction. ; Marginal score ; Non-black coverage .
[0037] Divide the grayscale image into 4×4 sub-regions and calculate the proportion of non-black pixels in each sub-region. The spatial uniformity coefficient is obtained. and spatial coverage score .
[0038] Final validity score for: .
[0039] This score reflects whether the view is filled with valid scene content, not the fidelity of the reconstruction of the real image.
[0040] Secondly, determine the number of Gaussian splats (Splats) that participate in the effective rendering from the current viewpoint. Distance to the nearest valid Splat These two quantities reflect the geometric security information and content density from the current subjective perspective. For example, Divide by Then used as feature input, Divide by 10 (in meters) and use the result as the feature input.
[0041] Simultaneously, the camera's own motion state is recorded: 3D linear motion. Directly using the position difference in the world coordinate system; 3D angular motion Depend on The resulting axis-angle vector representation. Additionally, a valid history queue of length 5 is maintained. Save the validity scores of the most recently recorded up to 5 views, padding with 0s if necessary; calculate the current validity trend. The initial value is 0 if there is no previous value, and the theoretical range is [-1, 1]; it counts consecutive invalid counts. During the observation phase, the threshold is truncated to no more than 5, and the value is taken before input. .
[0042] These features together constitute 15 basic scalar features.
[0043] Regarding step S1022, the multi-dimensional direction detection feature is a 28-dimensional direction detection feature, which specifically includes: taking the current camera orientation as a reference, performing detection and rendering on 12 horizontal yaw directions and 2 pitch directions respectively, obtaining the number of visible Gaussian pixels and the nearest Gaussian pixel distance in each direction; normalizing the number of visible Gaussian pixels and the nearest Gaussian pixel distance in each direction to obtain the 28-dimensional direction detection feature.
[0044] Here, the determination process of the multi-dimensional directional detection features can be implemented as follows: Using the current orientation as 0°, construct 12 horizontal yaw directions (0°, 30°, ..., 330°) and 2 pitch directions (-30° and +30° relative to the current orientation), for a total of 14 directions. For each direction, perform detection rendering to obtain the number of visible Splats in that direction. Distance to the nearest Splat .
[0045] Here, to reduce the impact of differences in the absolute density and scale of Splat in different scenes, the orientation detection results are relatively normalized within the samples: Visible direction number by Normalization; Directional distance Normalization.
[0046] This yields 14-dimensional directional quantity features and 14-dimensional directional distance features, for a total of 28-dimensional directional detection features.
[0047] For step S103, the target policy network is formed by pre-training with behavior cloning and online closed-loop fine-tuning with a proximal policy optimization algorithm.
[0048] In one embodiment provided in this application, the step of concatenating the multidimensional basic scalar features and the multidimensional orientation detection features into a scalar state vector and inputting it into a pre-trained target policy network to output the multidimensional pose change of the camera includes: concatenating the multidimensional basic scalar features and the multidimensional orientation detection features into a 43-dimensional scalar state vector in a fixed order; inputting the 43-dimensional scalar state vector into the target policy network to predict the camera pose change, outputting a 6-dimensional continuous action, and determining the 6-dimensional continuous action as the multidimensional pose change; wherein, the target policy network is a fully connected multilayer perceptron Actor-Critic structure, and the 6-dimensional continuous action consists of the first 3 dimensions being the camera local coordinate system translation command and the last 3 dimensions being the axis-angle rotation command.
[0049] This embodiment specifically includes: concatenating the 15-dimensional basic scalar features and 28-dimensional directional detection features determined in step S102 into a 43-dimensional scalar state vector in a preset fixed order.
[0050] The fixed order is as follows: first, 5-dimensional historical validity scores, 1-dimensional validity trend, 1-dimensional number of visible Gaussian splats in the current viewpoint, 1-dimensional distance to the nearest valid Gaussian splat, 3-dimensional linear motion vector, 3-dimensional angular motion vector, and 1-dimensional consecutive invalid count are added sequentially, for a total of 15 dimensions; then, 14-dimensional normalized values of the number of visible splats in 14 directions and 14-dimensional normalized values of the nearest splat distances in 14 directions are added sequentially, for a total of 28 dimensions. This concatenation order remains consistent during the training and inference phases to ensure stable semantic correspondence of feature locations. Since this state vector consists only of scalars and does not contain any original RGB image or depth map tensors, it fundamentally avoids the direct processing of high-dimensional visual data by the policy network, significantly reducing the network input size and computational overhead.
[0051] Then, the 43-dimensional scalar state vector is input into a pre-trained target policy network for camera pose change prediction. The target policy network employs a fully connected multilayer perceptron (MLP) Actor-Critic structure. Specifically, the state encoder consists of a 43-dimensional input layer, a 256-dimensional first hidden layer, a 256-dimensional second hidden layer, and a 128-dimensional third hidden layer, with each fully connected layer followed by a ReLU activation function. The Actor branch, based on the 128-dimensional features output by the encoder, passes through a 64-dimensional fully connected layer and a 6-dimensional fully connected layer. The final 6-dimensional output is mapped to the [-1, 1] interval using a Tanh activation function, serving as the mean vector of the action distribution. Similarly, the Critic branch, based on the 128-dimensional features output by the encoder, passes through a 64-dimensional fully connected layer and a 1-dimensional fully connected layer, outputting a state value estimate. The action distribution is modeled as an independent Gaussian distribution for each dimension, with each dimension sharing a learnable 6-dimensional log-standard deviation parameter. That is, the action probability density function is . It is a 6-dimensional motion. The strategy mean, To share the logarithmic standard deviation, the total number of trainable parameters in the entire Actor-Critic network is approximately 126,925, which is much smaller than the number of parameters in an image policy model based on a convolutional neural network.
[0052] The target policy network outputs a 6-dimensional continuous action vector, denoted as... The first three dimensions These are translation commands in the camera's local coordinate system, corresponding to movement control along the camera's local x-axis (horizontal), y-axis (vertical), and z-axis (forward); (Back 3D) This is an axis-angle rotation command used to describe rotation about the camera's local coordinate axes.
[0053] During the online inference phase, the mean vector of the Actor branch output is usually used directly as the deterministic action without random sampling to ensure the stability and reproducibility of the trajectory. During the training phase or when exploration is required, sampling can be performed based on the action mean and the learnable standard deviation, and the sampling results can be clipped to the range of [-1, 1].
[0054] In this way, by concatenating multidimensional basic scalar features and multidimensional orientation detection features into a compact vector of fixed dimensions, and constraining the policy network to a lightweight fully connected structure, the entire autonomous roaming process does not require convolutional neural networks to process images. This preserves the information needed for orientation selection, obstacle perception, and motion state representation while significantly reducing computational resource consumption, which is beneficial for deployment in real-time or near-real-time conditions. Furthermore, by employing a Gaussian action distribution with shared log-standard deviation, action uncertainty can be naturally learned during behavior clone pre-training and proximal policy optimization fine-tuning, improving the policy's exploration efficiency and convergence stability.
[0055] Regarding step S104, in one embodiment provided in this application, the multidimensional pose change includes translation increment and rotation increment. The step of updating the camera pose based on the multidimensional pose change and performing a feasibility check on the updated candidate camera pose includes: S1041. Update the position and orientation of the camera according to the translation increment and rotation increment, and generate candidate camera poses; S1042. Perform a pre-collision check based on the depth data rendered from the main viewpoint of the previous frame to determine whether there is a collision risk. S1043. Perform full rendering of the candidate camera poses and perform validity constraint verification based on the rendering results; S1044. If both items pass, the feasibility check is deemed qualified; if either item fails, the feasibility check is deemed unqualified.
[0056] Specifically, step S1041 includes: In this step, the multidimensional pose change output by the policy network is a 6-dimensional continuous action. The first three dimensions are translation increment commands, and the last three dimensions are rotation increment commands.
[0057] For each translation increment, multiply each component by a preset translation step size of 0.3 meters to obtain the translation vector in the camera's local coordinate system:
[0058] This translation vector is defined in the camera's local coordinate system, where the local x-axis corresponds to the horizontal direction, the y-axis to the vertical direction, and the z-axis to the forward direction. Then, the current camera rotation matrix is used... Transform it to the world coordinate system and compare it with the current position. Add them together to get the candidate positions:
[0059] For the rotation increment, the last 3-dimensional motion components are converted into axis-angle rotation vectors:
[0060] The direction of the rotation vector is the rotation axis, and the magnitude is the rotation angle (in radians). An incremental quaternion is constructed from this axis-angle vector. and with the current camera quaternion Multiplying them together yields the candidate orientation quaternions:
[0061] Then Convert to rotation matrix Then, roll-off processing is performed to keep the camera's orientation consistent with the world coordinate system's orientation, avoiding image tilt. The final candidate camera pose is then obtained. , by location and rotation matrix express.
[0062] It should be noted that the translation step size of 0.3 meters mentioned above is a scaling factor for each local component, not a strict upper limit for the three-dimensional resultant displacement; when all three translation components simultaneously reach their boundary values, the upper limit of the resultant displacement is... Meters. Similarly, each rotation component corresponds to a 15° rotation angle, and the maximum magnitude of the rotation vector when clipping by components is... Spend.
[0063] Regarding step S1042, in order to improve efficiency and avoid potential collisions in advance, before fully rendering the candidate poses, this step uses the depth data obtained from the main view rendering of the previous frame to perform a pre-collision check on the candidate poses.
[0064] Specifically, the world motion direction of the candidate pose relative to the current pose is transformed to the current camera coordinate system to obtain the local motion direction. Depth pre-collision check is performed only if the motion direction has a forward component (i.e., a forward component towards the scene content); for non-forward motions such as pure side-shifting or backward-shifting, this check may be omitted, or other simplified judgments may be used.
[0065] For step S1043, in this step, a complete rendering of the candidate camera pose is performed, and a validity constraint check is performed based on the rendering result.
[0066] Validity constraint verification may include a joint assessment of the following conditions: Nearest Splat Distance Constraint: Calculate the distance to the nearest valid Splat from the current viewpoint. ,Require Meters. This condition is used to prevent the camera from getting too close to the scene surface or entering areas with dense splats.
[0067] Splat number constraint: Count the number of Splats that participate in the effective rendering of the current viewpoint. ,Require This condition is used to prevent the camera from entering extremely sparse or blank areas.
[0068] Validity score lower bound constraint: Calculate the no-reference rendering validity score of the current view. ,Require The validity score is derived from a combination of grayscale variance, edge information, non-black coverage, and spatial uniformity, and is used to exclude black screens, overly dark, or low-texture views.
[0069] Historical consistency constraint: When the valid historical queue is not empty, the current valid score must be no less than 0.2 times the average of the historical scores, i.e.:
[0070] This condition prevents a drastic drop in view quality within a short period of time, ensuring the continuity of the trajectory viewing experience.
[0071] If all the above conditions are met, the validity constraint check is deemed to have passed; if any condition is not met, the validity constraint check is deemed to have failed.
[0072] For step S1044, if the pre-collision check passes and the validity constraint verification passes, the feasibility check is deemed qualified, the candidate camera pose is accepted, it is used as the camera pose for the next step, and the current camera parameters are updated.
[0073] If either the pre-collision check or the validity constraint check fails, the feasibility check is deemed unqualified. In this case, the candidate camera pose is not used, and instead, a corresponding safety recovery operation is performed. For example, in another embodiment provided in this application, if the check fails, the autonomous roaming method further includes: backtracking along the accepted historical camera pose sequence to the most recent safe pose that satisfies the feasibility check as the current camera parameter, and starting from the safe pose, continuing to perform the prediction of the next valid camera pose; wherein, if the number of consecutive backtracking steps reaches a preset threshold and a valid pose is still not obtained, a forced escape operation is triggered.
[0074] In this embodiment, if the feasibility check in step S104 is deemed unqualified, the autonomous roaming method enters a safety recovery process to ensure that the roaming process is not interrupted or stuck due to a single invalid action. The following example, using a 3D Gaussian splash scene, illustrates this rollback and forced escape process.
[0075] When a candidate camera pose fails the feasibility check, it is neither updated nor added to the accepted pose history. Instead, a rollback mechanism is activated: from the accepted historical camera pose sequence, the system backtracks chronologically from most recent to furthest to find the most recent safe pose that still meets the feasibility check. The camera parameters are then restored to this safe pose, and it is used as the new current camera parameter for continued navigation. If no valid pose is found in multiple consecutive attempts (i.e., the number of consecutive rollbacks reaches a preset threshold), a forced escape operation is triggered.
[0076] Here, during normal roaming, for each candidate camera pose received, the system stores the pose and its corresponding status information (including position, orientation, validity score, depth cache, access popularity, etc.) into the history queue in chronological order.
[0077] During rollback, the system starts from the tail of the queue (i.e., the most recently accepted pose) and checks backwards sequentially. For each historical pose, it can be re-rendered to verify whether it still meets the validity constraints and pre-collision conditions; or, if the pose passed all checks when it was previously accepted and the scene is a static 3DGS model, it can be directly considered to meet the safety conditions without re-rendering.
[0078] It also records the number of consecutive backtracking events. When the counter reaches a preset threshold, it is considered that partial backtracking is no longer an effective way to escape, triggering a forced escape.
[0079] For step S105, when all feasibility checks in S104 are passed, the candidate camera pose is accepted, the camera's position and orientation are updated to the candidate values, and the camera's current parameters are updated accordingly.
[0080] At the same time, update the history queue: add the validity score of the current step. Add the most recent validity history; if the queue length exceeds 5, remove the oldest item; update the access heat grid: when the camera accesses the center grid, the heat of that grid is increased by 1, and the heat of adjacent grids with a Chebyshev distance of 1 is increased by 1 / 2, which is used for subsequent revisit penalty calculation; update the depth cache for use in the next step of pre-collision check.
[0081] Then, it is determined whether the autonomous roaming termination conditions are met. Termination conditions include, but are not limited to: reaching the preset maximum number of steps; receiving an external stop command; or meeting the goal set by the application layer (such as reaching a designated area, completing a specific roaming path, etc.). If the termination conditions are not met, the process returns to S102, and the state is re-rendered, reconstructed, and the next action is decided based on the updated camera parameters; if the conditions are met, the roaming process ends, and the camera pose sequence is output.
[0082] Furthermore, in another embodiment provided in this application, the target policy network is constructed through the following steps; S201, Prepare an expert trajectory dataset; the expert trajectory dataset includes roaming trajectories recorded by human interaction or roaming trajectories generated by converting existing camera keyframe paths, and each trajectory contains scalar states and corresponding action data under the corresponding pose; S202, Based on the expert trajectory dataset, perform behavior cloning pre-training to enable the initial policy network to learn the basic roaming motion pattern and obtain the pre-trained policy network; S203 uses a pre-trained policy network as initial parameters and employs a proximal policy optimization algorithm to perform online closed-loop fine-tuning in a 3D Gaussian splash scene. The roaming effect is optimized through a dense reward function to obtain the final target policy network.
[0083] For step S201, this embodiment supports two expert trajectory sources, which can be used alone or in combination.
[0084] Method 1: Manual Interaction Recording The operator controls a virtual camera to navigate through a 3D Gaussian splash scene via an interactive interface. The interface renders the view corresponding to the current camera pose in real time, and the operator controls the camera's translation and rotation using input devices such as a mouse, keyboard, or gamepad. When the operator makes their first human action, the system begins recording a demonstration trajectory, progressively saving the state st, action at, reward rt, next state st+1, end marker, timestamp, human score, good / bad marker, and camera pose; at the end, it saves the total trajectory reward, number of steps, duration, and a final score from 0 to 5.
[0085] Method 2: Generating from existing camera paths When a pre-designed camera path already exists (such as a sequence of camera keyframes created by an animator or generated by a path planning algorithm), it can be converted into an expert trajectory through interpolation and motion inverse kinematics.
[0086] Specifically, a sequence of camera keyframes containing position and quaternion orientation is read. For two adjacent keyframes, translation uses linear interpolation, and rotation uses spherical linear interpolation (SLERP). The number of substeps is the larger of the number of steps required for translation constraints and the number of steps required for rotation constraints, and 70% of the motion range is used as the interpolation step size margin to ensure that the motion components obtained from the inverse solution fall within the range of motion. Within the range.
[0087] For adjacent poses, the translation is solved by the inverse formula:
[0088] The position difference in the world coordinate system is transformed to the camera's local coordinate system and then divided by a step size of 0.3 meters. The rotation motion is solved in reverse as follows:
[0089] in For the reason The shortest arc axis angle vector obtained by transformation is clipped to each component. .
[0090] Perform realistic 3D Gaussian splash rendering at each interpolated pose and calculate the no-reference rendering validity score. The number of currently visible Splats Nearest Splat distance It uses 14-directional detection features to construct a 43-dimensional state that is completely isomorphic to the online environment. The unrendered synthetic state model is used for debugging only and is not used as formal expert data.
[0091] It's important to note that regardless of the method used to obtain the original trajectories, they must be screened before being used for behavior cloning training. First, only trajectories with a final score of 3 or higher are retained. Second, near-stationary steps with excessively small translation norms and total action norms are filtered out, as these steps lack meaningful motion information and would hinder effective translation and rotation behavior learning. The filtered trajectories constitute the expert trajectory dataset, stored as an array of binary data, with a corresponding JSON file containing only readable metadata.
[0092] Regarding step S202, in one embodiment provided in this application, the step of performing behavior cloning pre-training based on the expert trajectory dataset to enable the initial policy network to learn the basic roaming motion pattern and obtain the pre-trained policy network includes: using state-action pairs in the expert trajectories of the expert trajectory dataset as supervised samples, employing a weighted Gaussian negative log-likelihood loss as the behavior cloning loss, and adding a standard deviation lower bound term and an entropy reward term to the behavior cloning loss; optimizing the behavior cloning loss using the AdamW optimizer, setting an upper bound for the gradient norm and a cosine hot-restart learning rate schedule, iteratively updating the parameters of the initial policy network to obtain the pre-trained policy network. Specifically, the weighted Gaussian negative log-likelihood loss weights each action dimension according to the dimension weights, and jointly trains the mean output and shared log-standard deviation parameters of the policy network; the standard deviation lower bound term is used to penalize standard deviations below a preset lower bound, and the entropy reward term is used to reward the entropy of the action distribution.
[0093] The core objective of behavior cloning pre-training is to use state-action pairs from expert demonstrations to train the policy network in a supervised learning manner, so that the distribution of its output actions is as close as possible to the action choices of the expert in the same state, thereby providing a reasonable initial strategy for subsequent PPO online fine-tuning.
[0094] The specific training process is as follows: First, the standard deviation of each action dimension in the statistical expert trajectory dataset. ( Construction and Proportional dimensional weights This approach assigns higher weights to dimensions with smaller amplitudes and lower standard deviations to balance the learning importance of movements at different scales. The mean of all dimension weights is normalized to 1, and the weights of the three rotation dimensions are multiplied by 2 to further enhance the learning accuracy of rotational movements.
[0095] Then, determine the loss function: Behavioral cloning uses weighted Gaussian negative log-likelihood as the primary loss:
[0096] in For batch size, For the strategy in the first A Gaussian distribution of dimensionality. For expert actions. The loss is the action mean output and shared log-standard deviation of the joint training strategy. Instead of freezing the standard deviation.
[0097] exist Based on this, add a standard deviation lower bound term and an entropy reward term:
[0098] Among them, the lower bound of standard deviation is used to prevent the learnable standard deviation from collapsing too early and maintaining a certain ability to explore actions; the entropy reward term encourages the strategy to maintain an appropriate action distribution entropy and avoid falling into deterministic behavior too early.
[0099] Finally, the AdamW optimizer is used, with the gradient norm upper bound set to 1, and cosine-based hot-restart learning rate scheduling is employed. In one embodiment, the initial learning rate is... During training, the linear and angular motion components in the state can be set to zero with a 10% probability to simulate a cold start state and enhance the robustness of the strategy in the initial step or when motion information is missing.
[0100] After training is completed, a pre-trained policy network is obtained, and its parameters will be used as the initial parameters for the PPO fine-tuning stage.
[0101] Regarding step S203, in one embodiment provided in this application, the step of using a pre-trained policy network as initial parameters, performing online closed-loop fine-tuning in a 3D Gaussian splash scene using a proximal policy optimization algorithm, and optimizing the roaming effect through a dense reward function to obtain the final target policy network includes: S2031, using the pre-trained policy network output by the behavior clone pre-training as the training starting point, autonomous roaming closed-loop interaction is performed in a 3D Gaussian splash scene. The instantaneous reward of each step is calculated through a preset dense reward function, and trajectory data of a fixed number of steps is collected. The trajectory data includes the scalar state at the corresponding moment, the executed action, the instantaneous reward, and the scalar state at the next moment. For trajectories that have reached the maximum number of steps, the timeout trajectory is truncated, and the terminal state is processed by value function bootstrapping to complete the reward estimation. S2032, Based on the collected trajectory data, calculate the generalized dominance estimate at each time point, and standardize the obtained dominance values. S2033: Construct the total loss function based on the standardized advantage value, and after setting the upper limit of the gradient norm, backpropagate to update the parameters of the pre-trained policy network. S2034, repeat the above steps of closed-loop interactive sampling, advantage estimation calculation and policy iterative update until the roaming effect index meets the preset convergence condition, and obtain the final target policy network.
[0102] For step S2031, when fine-tuning using the near-end policy optimization algorithm, the dense reward function used includes: effectiveness score term, effectiveness change term, farthest distance progress term, actual movement term, continuous stationary penalty term, heading consistency term, visible quantity term, catastrophic descent penalty term, revisit penalty term, and constant cost term.
[0103] In this step, the pre-trained policy network parameters output from the behavior clone pre-training stage are first loaded as the starting point for PPO fine-tuning.
[0104] In a 3D Gaussian splash scene, a pre-trained policy network is used as the behavior policy to perform autonomous roaming closed-loop interaction. The interaction process at each step is consistent with the online inference stage: render the image in the current camera pose and calculate the no-reference rendering validity score; perform 14-direction detection and construct a 43-dimensional scalar state; input the state into the policy network to output the action; update the candidate camera pose according to the action; perform pre-collision checks and validity constraint verification; if successful, accept; otherwise, perform backtracking or forced escape.
[0105] In each interaction step, a pre-defined dense reward function is used to calculate the immediate reward. The dense reward function is defined as follows:
[0106] The meanings of each item have been explained in the foregoing description, and they correspond to the validity score, validity change, maximum distance progress, actual movement, continuous stationary penalty, course consistency, visible quantity, catastrophic decline penalty, revisit penalty, and constant cost, respectively.
[0107] Regarding step S2032, in this sub-step, based on the trajectory data collected in S2031, the generalized advantage estimate (GAE) is calculated for each time step. GAE is achieved by introducing parameters... Between bias and variance. Example: .
[0108] Specifically, let For time-series difference residuals, where As a discount factor, in this embodiment .
[0109] After calculating the dominance estimates at each time point, all dominance values are standardized by subtracting the mean and dividing by the standard deviation, so that the standardized dominance values have zero mean and unit variance.
[0110] For step S2033, in this sub-step, the standardized advantage value calculated in S2032 is used to construct the PPO total loss function, and the policy network parameters are updated through backpropagation.
[0111] The PPO total loss function is defined as follows:
[0112] Where: L clip For PPO trimming agency losses, L value D represents the mean square error of the value. KL Constraining the policy from the demonstrated prior, L std-floor The standard deviation of each action dimension is limited to a preset lower limit. The gradient norm is limited to 0.5. The PPO learning rate is a configurable parameter; the joint BC-PPO implementation consistent with the current training process uses 3×102 4 No longer fixed as 10 4 .
[0113] For step S2034, this sub-step involves cyclically executing processes S2031 to S2033, which sequentially perform closed-loop interactive sampling, advantage estimation calculation, and policy parameter updates. After each round of the above process, the convergence condition is determined based on a preset roaming performance index.
[0114] After the above iterative training, the resulting target policy network can output appropriate camera pose changes based on the 43-dimensional scalar state, enabling safe, stable, and wide-coverage autonomous roaming in a 3D Gaussian splash scene.
[0115] Based on the same inventive concept, this application also provides an autonomous roaming device corresponding to the autonomous roaming method. Since the principle of the device in this application is similar to the autonomous roaming method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.
[0116] Please see Figure 2 , Figure 3 , Figure 2 This is a schematic diagram of the structure of an autonomous roaming device in a 3D Gaussian splash scene provided in an embodiment of this application. Figure 3 This is a schematic diagram of the structure of another autonomous roaming device in a 3D Gaussian splash scene provided in an embodiment of this application. Figure 2 As shown, the autonomous roaming device 200 includes: The acquisition module 210 is used to acquire the target 3D Gaussian splash scene data and the current camera parameters of the camera; Processing module 220 is used to process the target 3D Gaussian splash scene data and current camera parameters to determine multi-dimensional directional detection features and multi-dimensional basic scalar features describing the camera's own motion state, the view quality of the current main viewpoint and geometric safety information. The prediction module 230 is used to concatenate the multidimensional basic scalar features and the multidimensional orientation detection features into a scalar state vector, and input it into a pre-trained target policy network to output the multidimensional pose change of the camera; wherein, the target policy network is formed by pre-training with behavior cloning and online closed-loop fine-tuning with a proximal policy optimization algorithm; The inspection module 240 is used to update the camera pose based on the multidimensional pose change and to perform a feasibility check on the candidate camera poses determined after the update. The update module 250 is used to update the current camera parameters of the camera according to the candidate camera pose if the check is qualified, and repeat the above steps until the autonomous roaming termination condition is met.
[0117] Optionally, when processing the target 3D Gaussian splash scene data and current camera parameters to determine multi-dimensional directional detection features and multi-dimensional basic scalar features describing the camera's own motion state, the view quality of the current main viewpoint, and geometric safety information, the processing module 220 is used to: Based on the current camera parameters, the target 3D Gaussian splash scene is rendered from the main viewpoint. The no-reference rendering validity score, the number of visible Gaussian primitives in the current viewpoint, and the distance to the nearest valid Gaussian primitive are calculated. The linear motion vector and angular motion vector of the camera are calculated by combining historical camera parameters. The validity history score and validity trend are obtained by combining historical validity data. The continuous invalid count is obtained by combining continuous invalid action records. Together, they form the multi-dimensional basic scalar feature. Based on the current camera orientation, multiple horizontal yaw detection directions and pitch detection directions are generated. Detection rendering is performed on each detection direction to obtain the number of visible Gaussian pixels and the distance to the nearest Gaussian pixel in each direction. After in-sample relative normalization, the multi-dimensional direction detection features are formed.
[0118] Optionally, the multidimensional basic scalar feature is a 15-dimensional basic scalar feature, specifically including: 5-dimensional validity history score, 1-dimensional validity trend, 1-dimensional number of visible Gaussian units from the current viewpoint, 1-dimensional distance to the nearest valid Gaussian unit, 3-dimensional linear motion vector, 3-dimensional angular motion vector, and 1-dimensional consecutive invalid count. The multi-dimensional orientation detection feature is a 28-dimensional orientation detection feature, specifically including: based on the current camera orientation, performing detection and rendering on 12 horizontal yaw directions and 2 pitch directions respectively, obtaining the number of visible Gaussian pixels and the nearest Gaussian pixel distance in each direction; normalizing the number of visible Gaussian pixels and the nearest Gaussian pixel distance in each direction to obtain the 28-dimensional orientation detection feature.
[0119] Optionally, when the prediction module 230 concatenates the multidimensional basic scalar features and the multidimensional orientation detection features into a scalar state vector and inputs it into a pre-trained target policy network to output the multidimensional pose change of the camera, the prediction module 230 is used to: The multidimensional basic scalar features and the multidimensional directional detection features are concatenated in a fixed order to form a 43-dimensional scalar state vector; The 43-dimensional scalar state vector is input into the target policy network to predict camera pose changes, outputting a 6-dimensional continuous action, which is then defined as the multi-dimensional pose change. The target policy network is a fully connected multilayer perceptron Actor-Critic structure, and the 6-dimensional continuous action consists of three dimensions: the first three dimensions are translation commands in the local camera coordinate system, and the last three dimensions are axis-angle rotation commands.
[0120] Optionally, the multidimensional pose change includes translation increment and rotation increment. When the checking module 240 updates the camera pose based on the multidimensional pose change and performs a feasibility check on the updated candidate camera poses, the checking module 240 is used to: Update the camera's position and orientation based on the translation and rotation increments to generate candidate camera poses; Perform a pre-collision check based on the depth data rendered from the main viewpoint of the previous frame to determine if there is a risk of collision; Perform full rendering of the candidate camera poses, and perform validity constraint verification based on the rendering results; If both items pass, the feasibility check is deemed satisfactory; if either item fails, the feasibility check is deemed unsatisfactory.
[0121] Optional, such as Figure 3 As shown, the autonomous roaming device 200 further includes a rollback module 260, which is used for: If the check fails, backtrack along the accepted historical camera pose sequence and backtrack to the most recent safe pose that meets the feasibility check as the current camera parameter. Starting from the safe pose, continue to perform the next effective pose prediction for the camera. If the number of consecutive backtracking reaches a preset threshold and no effective pose is obtained, a forced escape operation is triggered.
[0122] Optional, such as Figure 3 As shown, the autonomous roaming device 200 also includes a training module 270, which is used to construct the target policy network through the following steps; Prepare an expert trajectory dataset; the expert trajectory dataset includes roaming trajectories recorded by human interaction or roaming trajectories generated by converting existing camera keyframe paths, and each trajectory contains scalar states and corresponding action data under the corresponding pose; Based on the expert trajectory dataset, behavior cloning pre-training is performed to enable the initial policy network to learn basic roaming motion patterns, thus obtaining the pre-trained policy network. Using a pre-trained policy network as initial parameters, an online closed-loop fine-tuning algorithm is performed in a 3D Gaussian splash scene using a proximal policy optimization algorithm. The roaming effect is optimized through a dense reward function to obtain the final target policy network.
[0123] Optionally, when the training module 270 performs behavior cloning pre-training based on the expert trajectory dataset to enable the initial policy network to learn basic roaming motion patterns and obtain a pre-trained policy network, the training module 270 is used to: Using the state-action pairs in the expert trajectories of the expert trajectory dataset as supervised samples, a weighted Gaussian negative log-likelihood loss is adopted as the behavior cloning loss, and a standard deviation lower bound term and an entropy reward term are added to the behavior cloning loss; wherein, the weighted Gaussian negative log-likelihood loss weights each action dimension according to the dimension weight, and jointly trains the mean output of the policy network and the shared log-standard deviation parameter; the standard deviation lower bound term is used to penalize the standard deviation below the preset lower bound, and the entropy reward term is used to reward the entropy of the action distribution; The AdamW optimizer is used to optimize the behavior cloning loss, and the upper limit of the gradient norm and the cosine hot restart learning rate are set. The parameters of the initial policy network are iteratively updated to obtain the pre-trained policy network.
[0124] Optionally, when the training module 270 is used to perform online closed-loop fine-tuning in a 3D Gaussian splash scene using a pre-trained policy network as initial parameters and a proximal policy optimization algorithm to optimize the roaming effect through a dense reward function to obtain the final target policy network, the training module 270 is used to: Using the pre-trained policy network output by the behavior clone as the training starting point, autonomous roaming closed-loop interaction is performed in a 3D Gaussian splash scene. The instantaneous reward for each step is calculated through a preset dense reward function, and trajectory data of a fixed number of steps is collected. The trajectory data includes the scalar state, the executed action, the instantaneous reward, and the scalar state at the next moment. For trajectories that have reached the maximum number of steps, the timeout is truncated, and the terminal state is processed by value function bootstrapping to complete the reward estimation. Based on the collected trajectory data, the generalized dominance estimate at each time point is calculated, and the obtained dominance values are standardized. The total loss function is constructed based on the standardized advantage value, and the parameters of the pre-trained policy network are updated by backpropagation after setting the upper limit of the gradient norm. Repeat the above steps of closed-loop interactive sampling, advantage estimation calculation and policy iterative update until the roaming performance index meets the preset convergence condition to obtain the final target policy network.
[0125] Optionally, when fine-tuning using the near-end policy optimization algorithm, the dense reward function used includes: effectiveness score term, effectiveness change term, farthest distance progress term, actual movement term, continuous stationary penalty term, heading consistency term, visible quantity term, catastrophic descent penalty term, revisit penalty term, and constant cost term.
[0126] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.
[0127] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 The steps in the method embodiment shown are specifically implemented in the method embodiment and will not be repeated here.
[0128] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps in the method embodiment shown are specifically implemented in the method embodiment and will not be repeated here.
[0129] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0130] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.
[0131] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0132] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0133] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0134] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An autonomous roaming method in a 3D Gaussian splash scenario, characterized in that, The autonomous roaming method includes: Acquire the target 3D Gaussian splash scene data and the current camera parameters; The target 3D Gaussian splash scene data and current camera parameters are processed to determine multi-dimensional directional detection features and multi-dimensional basic scalar features describing the camera's own motion state, the view quality of the current main viewpoint, and geometric safety information. The multidimensional basic scalar features and the multidimensional orientation detection features are concatenated into a scalar state vector, which is then input into a pre-trained target policy network to output the multidimensional pose change of the camera. The target policy network is formed by pre-training with behavior cloning and online closed-loop fine-tuning with a proximal policy optimization algorithm. The camera pose is updated based on the multidimensional pose change, and the feasibility of the candidate camera poses determined after the update is checked. If the check is successful, the current camera parameters of the camera are updated according to the candidate camera pose, and the above steps are repeated until the autonomous roaming termination condition is met.
2. The method of claim 1, wherein, The process of processing the target 3D Gaussian splash scene data and current camera parameters to determine multi-dimensional directional detection features and multi-dimensional basic scalar features describing the camera's own motion state, the view quality of the current main viewpoint, and geometric safety information includes: Based on the current camera parameters, the target 3D Gaussian splash scene is rendered from the main viewpoint. The no-reference rendering validity score, the number of visible Gaussian primitives in the current viewpoint, and the distance to the nearest valid Gaussian primitive are calculated. The linear motion vector and angular motion vector of the camera are calculated by combining historical camera parameters. The validity history score and validity trend are obtained by combining historical validity data. The continuous invalid count is obtained by combining continuous invalid action records. Together, they form the multi-dimensional basic scalar feature. Based on the current camera orientation, multiple horizontal yaw detection directions and pitch detection directions are generated. Detection rendering is performed on each detection direction to obtain the number of visible Gaussian pixels and the distance to the nearest Gaussian pixel in each direction. After in-sample relative normalization, the multi-dimensional direction detection features are formed.
3. The method of claim 2, wherein, The multidimensional basic scalar features are 15-dimensional basic scalar features, specifically including: 5-dimensional validity history score, 1-dimensional validity trend, 1-dimensional number of visible Gaussian units from the current viewpoint, 1-dimensional distance to the nearest valid Gaussian unit, 3-dimensional linear motion vector, 3-dimensional angular motion vector, and 1-dimensional consecutive invalid count. The multi-dimensional orientation detection feature is a 28-dimensional orientation detection feature, specifically including: based on the current camera orientation, performing detection and rendering on 12 horizontal yaw directions and 2 pitch directions respectively, obtaining the number of visible Gaussian pixels and the nearest Gaussian pixel distance in each direction; normalizing the number of visible Gaussian pixels and the nearest Gaussian pixel distance in each direction to obtain the 28-dimensional orientation detection feature.
4. The method of claim 1, wherein, The step of concatenating the multidimensional basic scalar features and the multidimensional orientation detection features into a scalar state vector, and inputting it into a pre-trained target policy network to output the multidimensional pose change of the camera includes: The multidimensional basic scalar features and the multidimensional directional detection features are concatenated in a fixed order to form a 43-dimensional scalar state vector; The 43-dimensional scalar state vector is input into the target policy network to predict camera pose changes, outputting a 6-dimensional continuous action, which is then defined as the multi-dimensional pose change. The target policy network is a fully connected multilayer perceptron Actor-Critic structure, and the 6-dimensional continuous action consists of three dimensions: the first three dimensions are translation commands in the local camera coordinate system, and the last three dimensions are axis-angle rotation commands.
5. The method of claim 1, wherein, The multidimensional pose change includes translation increment and rotation increment. The process of updating the camera pose based on the multidimensional pose change and performing a feasibility check on the updated candidate camera poses includes: Update the camera's position and orientation based on the translation and rotation increments to generate candidate camera poses; Perform a pre-collision check based on the depth data rendered from the main viewpoint of the previous frame to determine if there is a risk of collision; Perform full rendering of the candidate camera poses, and perform validity constraint verification based on the rendering results; If both items pass, the feasibility check is deemed satisfactory; if either item fails, the feasibility check is deemed unsatisfactory.
6. The method of claim 1, wherein, If the inspection fails, the autonomous roaming method further includes: Backtracking along the accepted historical camera pose sequence, the system regresses to the most recent safe pose that satisfies the feasibility check as the current camera parameter, and continues to perform the next valid camera pose prediction starting from the safe pose. If the number of consecutive backtracking steps reaches a preset threshold and a valid pose is still not obtained, a forced escape operation is triggered.
7. The method of claim 1, wherein, The target policy network is constructed through the following steps; Prepare an expert trajectory dataset; the expert trajectory dataset includes roaming trajectories recorded by human interaction or roaming trajectories generated by converting existing camera keyframe paths, and each trajectory contains scalar states and corresponding action data under the corresponding pose; Based on the expert trajectory dataset, behavior cloning pre-training is performed to enable the initial policy network to learn basic roaming motion patterns, thus obtaining the pre-trained policy network. Using a pre-trained policy network as initial parameters, an online closed-loop fine-tuning algorithm is performed in a 3D Gaussian splash scene using a proximal policy optimization algorithm. The roaming effect is optimized through a dense reward function to obtain the final target policy network.
8. The method of claim 7, wherein, The step of performing behavior cloning pre-training based on the expert trajectory dataset, enabling the initial policy network to learn basic roaming motion patterns, and obtaining a pre-trained policy network includes: Using the state-action pairs in the expert trajectories of the expert trajectory dataset as supervised samples, a weighted Gaussian negative log-likelihood loss is adopted as the behavior cloning loss, and a standard deviation lower bound term and an entropy reward term are added to the behavior cloning loss; wherein, the weighted Gaussian negative log-likelihood loss weights each action dimension according to the dimension weight, and jointly trains the mean output of the policy network and the shared log-standard deviation parameter; the standard deviation lower bound term is used to penalize the standard deviation below the preset lower bound, and the entropy reward term is used to reward the entropy of the action distribution; The AdamW optimizer is used to optimize the behavior cloning loss, and the upper limit of the gradient norm and the cosine hot restart learning rate are set. The parameters of the initial policy network are iteratively updated to obtain the pre-trained policy network.
9. The method of claim 7, wherein, The process involves using a pre-trained policy network as initial parameters, employing a proximal policy optimization algorithm to perform online closed-loop fine-tuning in a 3D Gaussian splash scene, and optimizing the roaming effect through a dense reward function to obtain the final target policy network, including: Using the pre-trained policy network output by the behavior clone as the training starting point, autonomous roaming closed-loop interaction is performed in a 3D Gaussian splash scene. The instantaneous reward for each step is calculated through a preset dense reward function, and trajectory data of a fixed number of steps is collected. The trajectory data includes the scalar state, the executed action, the instantaneous reward, and the scalar state at the next moment. For trajectories that have reached the maximum number of steps, the trajectory is truncated due to timeout, and the terminal state is processed by value function bootstrapping to complete the reward estimation. Based on the collected trajectory data, the generalized dominance estimate at each time point is calculated, and the obtained dominance values are standardized. The total loss function is constructed based on the standardized advantage value, and the parameters of the pre-trained policy network are updated by backpropagation after setting the upper limit of the gradient norm. Repeat the above steps of closed-loop interactive sampling, advantage estimation calculation and policy iterative update until the roaming performance index meets the preset convergence condition to obtain the final target policy network.
10. The method of claim 9, wherein, When fine-tuning using the near-end policy optimization algorithm, the dense reward function used includes: effectiveness score term, effectiveness change term, farthest distance progress term, actual movement term, continuous stationary penalty term, heading consistency term, visible quantity term, catastrophic descent penalty term, revisit penalty term, and constant cost term.
