Virtual reality photography composition optimization method and system based on hierarchical reinforcement learning

Abstract

The application discloses a virtual reality photography composition optimization method and system based on layered reinforcement learning, comprising: acquiring an image frame collected by a virtual camera in a virtual environment in real time; inputting the image frame into a pre-trained layered reinforcement learning model; wherein the layered reinforcement learning model comprises a high-level decision module and a low-level execution module; the high-level decision module performs high-level feature extraction and abstraction on the image frame to generate high-level guidance information; the low-level execution module extracts low-level features of the image frame and generates a motion control instruction for controlling the virtual camera in combination with the high-level guidance information; and the motion control instruction is sent back to the virtual environment to drive the virtual camera to move, thereby realizing real-time optimization of photography composition. The application can realize an efficient, intelligent and real-time photography composition optimization solution in a real virtual engine environment by combining aesthetic evaluation, layered decision and natural interaction.

Description

Technical Field

[0001] This invention relates to the field of computer vision processing technology, specifically to a virtual reality photography composition optimization method and system based on hierarchical reinforcement learning, applicable to photography automation and intelligent guidance systems in virtual engine environments. Background Technology

[0002] Virtual reality (VR), augmented reality (AR), and the "metaverse" concept based on them are increasingly becoming new infrastructure in fields such as digital entertainment, virtual filming, architectural design, and interactive education. In these three-dimensional virtual spaces, the quality of visual presentation directly determines the user's immersion and experience satisfaction. Among these, the selection and composition of the camera's perspective, much like photography and videography in the real world, are core elements determining the aesthetic appeal and narrative tension of the image.

[0003] However, in the current virtual environment, achieving an ideal photographic composition still faces significant challenges, primarily due to the following technical limitations:

[0004] Firstly, most existing 3D cameras rely on manual control, requiring users to have certain photography knowledge and are cumbersome to operate, which reduces the efficiency and experience of content creation.

[0005] Secondly, current mainstream image composition optimization methods mainly target static images and are mostly offline cropping, which is difficult to meet the needs of real-time feedback and high interactivity in 3D scenes. Although some studies have applied reinforcement learning to automatic cropping or UAV path planning, these solutions are mostly limited to simplified simulation environments and are difficult to implement in real-world application scenarios such as high frame rates, low latency, and multimodal interaction, and their adaptability to complex new scenarios is limited.

[0006] Therefore, how to provide a method and system that can be deeply integrated into modern virtual engines, possess layered decision-making capabilities like human photographers, and achieve intelligent photographic composition in real-world application environments is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a document classification and key information extraction method for multi-page bidding documents, which can integrate aesthetic evaluation, hierarchical decision-making and natural interaction in a real virtual engine environment to achieve an efficient, intelligent and real-time photographic composition optimization solution.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] This invention first proposes a virtual reality photography composition optimization method based on hierarchical reinforcement learning, comprising the following steps:

[0010] S1: Acquire image frames captured in real time by a virtual camera in the virtual environment;

[0011] S2: Input the image frame into a pre-trained hierarchical reinforcement learning model; wherein the hierarchical reinforcement learning model includes a high-level decision-making module and a low-level execution module;

[0012] S3: The high-level decision-making module performs high-level feature extraction and abstraction on the image frame to generate high-level guidance information;

[0013] S4: The low-level execution module extracts the low-level features of the image frame and combines them with the high-level guidance information to generate motion control commands for controlling the virtual camera;

[0014] S5: Send the motion control command back to the virtual environment to drive the virtual camera to move, thereby achieving real-time optimization of the photographic composition.

[0015] Preferably, the step of generating high-level guidance information in S3 includes:

[0016] The image frame is used to extract features using a feature extraction network to obtain a first feature representation;

[0017] Channel attention weights are assigned to the first feature representation to perform channel recalibration and obtain the first optimized feature.

[0018] Based on the first optimized feature, feature channel modulation parameters and the state sub-target vector of the virtual camera are generated; wherein, the feature channel modulation parameters are used to perform channel-level conditional modulation on the low-level features; the state sub-target vector serves as the conditional input of the low-level execution module to guide the low-level execution module to generate action outputs that conform to the sub-target direction.

[0019] Preferably, the step of setting the feature channel modulation parameters in S3 includes: inputting the first optimized feature into a multilayer perceptron to generate the feature channel modulation parameters in parallel, including scaling factors and bias terms, for performing linear transform modulation on each channel of the low-level feature respectively.

[0020] Preferably, the step of generating motion control commands for controlling the virtual camera in S4 includes:

[0021] The image frame is used to extract features using a feature extraction network to obtain a second feature representation;

[0022] Channel attention weights are assigned to the second feature representation to perform channel recalibration and obtain the second optimized feature.

[0023] Using the aforementioned feature modulation parameters, the second optimized feature is subjected to FiLM modulation to obtain a conditional feature;

[0024] The conditional features are input into the LSTM unit, the hidden state of the previous time step is retained, and the motion control command is output through smoothing.

[0025] Preferably, the feature extraction network and channel recalibration mechanism in the low-level execution module are the same as those in the high-level decision module.

[0026] Preferably, the motion control command for the virtual camera is a continuous six-degree-of-freedom camera motion vector, which corresponds to the instantaneous motion speed or displacement of the virtual camera in six degrees of freedom.

[0027] Preferably, the hierarchical reinforcement learning model is trained by maximizing a preset composite reward function; the composite reward function includes at least:

[0028] The aesthetic reward item is positively correlated with the aesthetic evaluation score of the image frame;

[0029] The motion smoothing term is negatively correlated with the amplitude of the motion control command;

[0030] A dynamic speed adjustment feature is configured to adaptively adjust the reward and punishment strategy for the amplitude of the motion control command based on the current aesthetic evaluation score.

[0031] Preferably, the dynamic speed adjustment item is configured as follows:

[0032]

[0033] in, A scaling factor that is positively correlated with the aesthetic evaluation score of the current frame, and ∈[0,1], δ is the weighting coefficient, The motion control command action vector for the virtual camera.

[0034] Preferably, the high-level decision-making module synchronizes high-level guidance information to the low-level execution module according to a preset periodic step size.

[0035] The present invention further provides a virtual reality photography composition optimization system according to the aforementioned hierarchical reinforcement learning-based virtual reality photography composition optimization method, comprising:

[0036] The data acquisition module is configured to acquire image frames captured in real time by a virtual camera in the virtual environment;

[0037] A hierarchical decision-making module, which is communicatively connected to the data acquisition module, includes a pre-trained high-level decision-making module and a low-level execution module;

[0038] The high-level decision-making module is configured to extract and abstract high-level features from the image frame to generate high-level guidance information.

[0039] The low-level execution module is configured to extract low-level features of the image frame and combine them with the high-level guidance information to generate motion control commands for controlling the virtual camera.

[0040] The control execution module, which is communicatively connected to the hierarchical decision module, is configured to send the motion control commands back to the virtual environment to drive the virtual camera to move and achieve real-time optimization of the photographic composition.

[0041] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a virtual reality photography composition optimization method and system based on hierarchical reinforcement learning, which has the following beneficial effects:

[0042] This invention integrates an aesthetic evaluation model with hierarchical reinforcement learning, enabling the system to not only learn how to control the camera but also how to pursue compositions that conform to human aesthetics. Experiments show that photographic compositions optimized by this system exhibit significantly improved aesthetic scores across various virtual scenes, effectively lowering the professional threshold for content creation and ensuring professional-grade quality in output images.

[0043] This invention employs a hierarchical decision-making system and an LSTM temporal smoothing network in its model structure to ensure stable and effective graphing performance even when facing unknown scenes, varying lighting conditions, and different object layouts. This allows the invention to be deployed directly without retraining for each new scene, greatly expanding its applicability and reducing usage costs.

[0044] This invention achieves end-to-end low-latency control and high-frame-rate rendering. This performance fully meets the stringent real-time response requirements of applications such as virtual reality and metaverse, greatly enhancing user immersion and creative freedom.

[0045] This invention can be directly integrated into mainstream virtual engine workflows, and can be directly integrated into various practical workflows such as metaverse and virtual shooting, providing an end-to-end automated and intelligent photography solution. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0047] Figure 1 A flowchart of a virtual reality photography composition optimization method based on hierarchical reinforcement learning provided in an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of the model principle of the virtual reality photography composition optimization method based on hierarchical reinforcement learning provided in an embodiment of the present invention;

[0049] Figure 3 A comparison diagram of the photographic perspective before and after virtual reality photographic composition optimization provided in an embodiment of the present invention;

[0050] Figure 4 The image shows the effect of automatically selecting shooting angles in different virtual environments of Unreal Engine 5, as provided in the embodiments of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] The first aspect of this invention discloses a virtual reality photography composition optimization method based on hierarchical reinforcement learning. The technical principle involves continuously acquiring real-time rendering frames within a virtual environment (such as an environment built with a high-end engine like Unreal Engine 5) using a custom plugin, and sending the images to an external reinforcement learning client. This external reinforcement learning client loads the hierarchical reinforcement learning model described in this invention. The external reinforcement learning client generates sub-targets and action commands according to a hierarchical structure, and finally maps them back to the virtual environment to control the virtual camera, achieving end-to-end real-time viewpoint optimization. Figure 1 As shown, it includes the following steps:

[0053] S1: Acquire image frames captured in real time by a virtual camera in the virtual environment;

[0054] S2: Input the image frame into the pre-trained hierarchical reinforcement learning model; wherein, the hierarchical reinforcement learning model includes a high-level decision module and a low-level execution module;

[0055] S3: The high-level decision-making module performs high-level feature extraction and abstraction on image frames to generate high-level guidance information;

[0056] S4: The low-level execution module extracts low-level features of the image frame and combines them with high-level guidance information to generate motion control commands for controlling the virtual camera;

[0057] S5: Send motion control commands back to the virtual environment to drive the virtual camera to move, thereby achieving real-time optimization of photographic composition.

[0058] In this embodiment, the hierarchical reinforcement learning model is divided into two stages: a high-level decision-making module and a low-level execution module, in order to achieve efficient and smooth camera perspective optimization. The high-level module focuses on policy planning and sub-objective generation, while the low-level module is responsible for fine-grained action control, reducing the learning difficulty of a single network.

[0059] In one embodiment, the specific execution steps of S1 include: deploying and using the Cosy-Airsim plugin in the UE5 engine, asynchronously capturing a 224×224 resolution frame buffer, sending it to an external reinforcement learning client via a local socket, and mapping the received motion control instructions to six-degree-of-freedom camera commands.

[0060] In one embodiment, during the pre-training process of the hierarchical reinforcement learning model, the training scene is subjected to multi-source perturbations to improve generalization, including but not limited to: global illumination intensity ±ΔI, random perturbations of the main light direction and hue, and key target object scale ±10%.

[0061] In one embodiment, the step of generating high-level guidance information in S3 includes:

[0062] The first feature representation is obtained by using a feature extraction network to extract features from the image frame;

[0063] Channel attention weights are assigned to the first feature representation to perform channel recalibration and obtain the first optimized feature.

[0064] Based on the first optimized feature, feature channel modulation parameters and state sub-target vectors of the virtual camera are generated; wherein, the feature channel modulation parameters are used to perform channel-level conditional modulation of the low-level features; the state sub-target vectors serve as conditional inputs to the low-level execution module to guide the low-level execution module to generate action outputs that conform to the sub-target direction.

[0065] The specific execution steps of the high-level decision-making module Manager in this embodiment are as follows:

[0066] The ResNet-50 network is used to extract the global feature vector (2048 dimensions) of the current frame image (224×224) to obtain the first feature representation.

[0067] The first feature representation is input into the Squeeze-and-Excitation module, which enhances the important channel information through adaptively learned channel weights to obtain the first optimized feature:

[0068]

[0069]

[0070] Where F represents the first feature and GAP represents global average pooling. This is the weight matrix of the first fully connected layer. It is a ReLU nonlinear activation function. This is the weight matrix for the second fully connected layer. is the Sigmoid activation function, ⊙ is the element-wise multiplication of the channels, and s is the channel attention weight vector.

[0071] The first optimized feature is reduced in dimensionality by passing it through a fully connected bottleneck layer (512 dimensions). Bottleneck characteristics (e.g., d=512).

[0072] The characteristic channel modulation parameters are then generated by two multilayer perceptrons (MLPs) with two independent output branches:

[0073]

[0074] in, Scaling the channel; For channel offset; This represents the number of main channels for the lower-level execution module, Worker.

[0075] By utilizing the FiLM (Feature-wise Linear Modulation) layer, bottleneck features are... Mapped to a six-dimensional sub-target vector This indicates the desired coarse-grained position and orientation adjustments for the camera in the next stage.

[0076] It should be noted that the high-level decision-making module Manager synchronizes the six-dimensional sub-target vector generated by the FiLM layer to the low-level execution module Worker; the Worker uses the six-dimensional sub-target vector to perform FiLM conditional modulation on the intermediate features, and then outputs the six-dimensional camera action through LSTM and policy head, so that the sub-target explicitly participates in the temporal decision-making and indirectly constrains the magnitude and direction of the action.

[0077] In one embodiment, the step of generating motion control commands for controlling the virtual camera in S4 includes:

[0078] A feature extraction network is used to extract features from image frames to obtain a second feature representation;

[0079] Channel attention weights are assigned to the second feature representation to perform channel recalibration, resulting in the second optimized feature.

[0080] Using the feature modulation parameters, the second optimized feature is FiLM modulated to obtain the conditional feature;

[0081] The conditional feature input LSTM unit retains the hidden state of the previous time step and outputs motion control commands through smoothing.

[0082] The specific execution steps of the low-level decision-making module Worker in this embodiment are as follows:

[0083] The ResNet-50 network is used to extract the global feature vector (2048 dimensions) of the current frame image (224×224) to obtain the second feature representation.

[0084] The second feature representation is input into the Squeeze-and-Excitation module, which enhances the important channel information through adaptively learned channel weights to obtain the second optimized feature:

[0085]

[0086]

[0087] Where U represents the first feature and GAP represents global average pooling. This is the weight matrix of the first fully connected layer. It is a ReLU nonlinear activation function. This is the weight matrix for the second fully connected layer. is the Sigmoid activation function, ⊙ is the element-wise multiplication of the channels, and s is the channel attention weight vector.

[0088] It is important to note that the Worker also uses the SE module to recalibrate the channels of the original ResNet-50 features in order to focus on the sub-target vectors. The relevant area.

[0089] The high-level sub-target vector is incorporated into the second optimization feature through a FiLM layer, and the second optimization feature is conditionally modulated to make the network focus on achieving the current sub-target vector. Required visual cues. Second optimization feature for the Worker. Perform channel condition modulation:

[0090]

[0091] in, To amplify / suppress channel c; This is the offset translation of channel c.

[0092] Conditionalized features, after conditional modulation, are input into an LSTM unit, preserving historical information and smoothing action output to avoid abrupt changes caused by single-frame decisions.

[0093]

[0094] in As the observation embedding at time t, ( ) represents the hidden state of the previous time step. () indicates the updated hidden state.

[0095] Hidden states of LSTM cells The six-dimensional primitive motion is obtained through a fully connected layer. :

[0096]

[0097] The final camera motion is obtained by cropping the amplitude of the image. :

[0098]

[0099] in, It is used directly to update the camera pose in UE5; Control the maximum step size. Motion amplitude threshold. Set the unit according to the scene (e.g., translation ≤ 0.05 m / step, rotation ≤ 1.5° / step).

[0100] The SE channel attention mechanism combined with FiLM adds only a few parameters, maintaining low computational overhead and making it suitable for real-time operation. The LSTM temporal module, combined with domain randomization training, ensures the robustness of the control strategy when switching between different scenarios.

[0101] In one embodiment, the feature extraction network and channel recalibration mechanism in the low-level execution module are the same as those in the high-level decision module.

[0102] In one embodiment, to balance aesthetic enhancement and control stability, a composite reward of "aesthetic increment + energy regularization + dynamic velocity term" is introduced. The hierarchical reinforcement learning model is trained by maximizing the preset composite reward function; the composite reward function includes at least:

[0103] The aesthetic reward item is positively correlated with the aesthetic evaluation score of the image frame;

[0104] The motion smoothing term is negatively correlated with the amplitude of the motion control command;

[0105] The dynamic speed adjustment feature is configured to adaptively adjust the reward and punishment strategy for the amplitude of motion control commands based on the current aesthetic evaluation score.

[0106] In this embodiment, the composite reward function is:

[0107]

[0108] in, The aesthetic score for the current frame. For the current action, As weight. Incremental aesthetic reward is Smoothing penalty is , This is a dynamic speed adjustment item.

[0109] In this embodiment, the dynamic speed adjustment term is configured as follows:

[0110]

[0111]

[0112] in, A scaling factor that is positively correlated with the aesthetic evaluation score of the current frame, and ∈[0,1], δ is the weighting coefficient, The motion control command vector for the virtual camera. This is the aesthetic threshold.

[0113] When the current aesthetic score is below the preset threshold, the dynamic speed adjustment item penalizes the movement amplitude less to encourage exploration; when the current aesthetic score reaches or exceeds the preset threshold, the dynamic speed adjustment item penalizes the movement amplitude more to encourage fine-tuning. This mechanism encourages rapid exploration when the aesthetic score is low and switches to fine-tuning at higher scores.

[0114] This embodiment also includes an intrinsic reward function, which is only applicable to... Rewards and penalties will not affect sub-target vectors. Rewards and penalties are applied based on whether the actions of lower-level execution modules effectively reduce the gap between the current state and the state described by the sub-target vector. In summary, The next stage is specifically instructed in the form of a six-dimensional directional target. FiLM conditionalization guides the feature and timing decisions of the lower-level execution module Worker, and updates synchronously according to a preset periodic step size, ultimately guiding the generation of the camera action for the next frame. .

[0115] In one embodiment, the high-level decision-making module (Manager) synchronizes high-level guidance information to the low-level execution module (Worker) according to a preset periodic step size. Specifically, during execution, the Manager interacts with the environment every N (N=4) steps, distributing the latest sub-goals to the Worker, and the Manager synchronizes every N=4 steps. To move to the Worker, in order to balance decision-making efficiency and stability.

[0116] It should be noted that the synchronization frequency between Manager and Worker is adjustable, flexibly balancing response speed and action smoothness.

[0117] like Figure 2 As shown, the input image frame (224×224) is fed into the ResNet-50 network to extract global features, and then fed into the hierarchical reinforcement learning model:

[0118] The Manager generates high-level guidance, which is executed by the Worker. After the features are recalibrated by the SE (Squeeze-and-Excitation), the entire hierarchical decision-making process (Policy) is evaluated by the State Value Estimation Critic, and outputs a 6-dimensional sub-objective / adjustment (Δx, Δy, Δz, pitch, yaw, roll).

[0119] Subsequently, Conditioning (FiLM) uses this 6-dimensional sub-target to conditionally modulate the lower-level control. Then, the LSTM unit fuses the timing to output continuous 6-DoF camera motion commands. The final generated output image frame is compared with the input image frame by PK to obtain the aesthetic score and dynamic velocity adjustment term, which are fed back as rewards / constraints to jointly update the Manager / Worker / Policy.

[0120] In one embodiment, the system listens to user voice / gesture commands in real time, supports multimodal interaction such as voice and gestures, and updates high-level sub-target vectors in real time, enabling users to dynamically adjust their compositional intentions during shooting, thereby improving creative efficiency and usability, and enhancing user creative efficiency and experience.

[0121] like Figure 3-4As shown, virtual environments such as indoor corridors, outdoor plazas, and low-light rooms are built in Unreal Engine 5. The system automatically collects the current camera viewpoint through end-to-end data streams and sends image frames to a layered RL agent for decision-making. The high-level Manager sets sub-objectives, and the low-level Worker outputs six-DOF camera motions. The system optimizes the viewpoint in real time and dynamically adjusts the motion speed based on aesthetic scores. The final automatically selected shooting viewpoint outperforms traditional solutions in terms of aesthetic evaluation, stability, and smoothness. This demonstrates the ability of the method provided in the first aspect of this invention to improve aesthetic composition in different scenarios, with significant results. It achieves automated aesthetic composition optimization with low end-to-end latency (≤43 ms), high rendering frame rate (≈34.7 FPS), and interactivity in UE5. The average improvement across the three training scenarios is ≈14%, with no drop in performance (≈6%) observed in the "city street" migration.

[0122] In the VR environment, users specify new shooting intentions via voice or gestures, and the system responds in real time, resetting and optimizing the goals. The system enhances its generalization ability through domain randomization, enabling it to quickly adapt to new, unseen "city street" scenes and achieve automated, aesthetically pleasing photographic composition.

[0123] The second aspect of this invention also discloses a virtual reality photography composition optimization system based on a hierarchical reinforcement learning-based method according to the first aspect of this invention, comprising:

[0124] The data acquisition module is configured to acquire image frames captured in real time by a virtual camera in the virtual environment;

[0125] The hierarchical decision-making module communicates with the data acquisition module and includes a pre-trained high-level decision-making module and a low-level execution module.

[0126] The high-level decision-making module is configured to extract and abstract high-level features from image frames to generate high-level guidance information.

[0127] The low-level execution module is configured to extract low-level features of image frames and combine them with high-level guidance information to generate motion control commands for controlling the virtual camera.

[0128] The control execution module, which communicates with the hierarchical decision module, is configured to send motion control commands back to the virtual environment to drive the virtual camera to move and achieve real-time optimization of photographic composition.

[0129] In summary, this invention enables a photography guidance system that integrates hierarchical reinforcement learning and aesthetic evaluation in high-fidelity environments such as Unreal Engine. This system achieves real-time, continuous, and interactive end-to-end control with an average latency of ≤60 ms and rendering speed of ≥30 FPS, while maintaining low migration loss in optimized photographic composition even in unseen scenes.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0131] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A virtual reality photography composition optimization method based on hierarchical reinforcement learning, characterized in that, Includes the following steps: S1: Acquire image frames captured in real time by a virtual camera in the virtual environment; S2: Input the image frame into a pre-trained hierarchical reinforcement learning model; wherein the hierarchical reinforcement learning model includes a high-level decision-making module and a low-level execution module; S3: The high-level decision-making module performs high-level feature extraction and abstraction on the image frame to generate high-level guidance information, including: The image frame is used to extract features using a feature extraction network to obtain a first feature representation; Channel attention weights are assigned to the first feature representation to perform channel recalibration and obtain the first optimized feature. Based on the first optimized feature, feature channel modulation parameters and the state sub-target vector of the virtual camera are generated; wherein, the feature channel modulation parameters are used to perform channel-level conditional modulation on the low-level features; the state sub-target vector serves as the conditional input of the low-level execution module to guide the low-level execution module to generate action output that conforms to the sub-target direction; S4: The low-level execution module extracts the low-level features of the image frame and combines them with the high-level guidance information to generate motion control commands for controlling the virtual camera, including: The image frame is used to extract features using a feature extraction network to obtain a second feature representation; Channel attention weights are assigned to the second feature representation to perform channel recalibration and obtain the second optimized feature. Using the feature channel modulation parameters, the second optimized feature is subjected to FiLM modulation to obtain conditional features; The conditional features are input into the LSTM unit, the hidden state of the previous time step is retained, and the motion control command is output through smoothing. S5: Send the motion control command back to the virtual environment to drive the virtual camera to move, thereby achieving real-time optimization of the photographic composition.

2. The virtual reality photography composition optimization method based on hierarchical reinforcement learning according to claim 1, characterized in that, The step of setting the feature channel modulation parameters in S3 includes: inputting the first optimized feature into a multilayer perceptron to generate the feature channel modulation parameters in parallel, including scaling factors and bias terms, for linear transformation modulation of each channel of the low-level feature.

3. The virtual reality photography composition optimization method based on hierarchical reinforcement learning according to claim 1, characterized in that, The motion control command for the virtual camera is a continuous six-degree-of-freedom camera motion vector, which corresponds to the instantaneous motion speed or displacement of the virtual camera in six degrees of freedom.

4. The virtual reality photography composition optimization method based on hierarchical reinforcement learning according to claim 1, characterized in that, The hierarchical reinforcement learning model is trained by maximizing a preset composite reward function; The composite reward function includes at least: The aesthetic reward item is positively correlated with the aesthetic evaluation score of the image frame; The motion smoothing term is negatively correlated with the amplitude of the motion control command; A dynamic speed adjustment feature is configured to adaptively adjust the reward and punishment strategy for the amplitude of the motion control command based on the current aesthetic evaluation score.

5. The virtual reality photography composition optimization method based on hierarchical reinforcement learning according to claim 4, characterized in that, The dynamic speed adjustment term is configured as follows: in, A scaling factor that is positively correlated with the aesthetic evaluation score of the current frame, and ∈[0,1], δ These are the weighting coefficients. The motion control command action vector for the virtual camera.

6. The virtual reality photography composition optimization method based on hierarchical reinforcement learning according to claim 1, characterized in that, The high-level decision-making module synchronizes high-level guidance information to the low-level execution module according to a preset cycle step.

7. A virtual reality photography composition optimization system based on a hierarchical reinforcement learning-based virtual reality photography composition optimization method according to any one of claims 1-6, characterized in that, include: The data acquisition module is configured to acquire image frames captured in real time by a virtual camera in the virtual environment; A hierarchical decision-making module, which is communicatively connected to the data acquisition module, includes a pre-trained high-level decision-making module and a low-level execution module; The high-level decision-making module is configured to extract and abstract high-level features from the image frame to generate high-level guidance information. The low-level execution module is configured to extract low-level features of the image frame and combine them with the high-level guidance information to generate motion control commands for controlling the virtual camera. The control execution module, which is communicatively connected to the hierarchical decision module, is configured to send the motion control commands back to the virtual environment to drive the virtual camera to move and achieve real-time optimization of the photographic composition.

Images