A multi-scale end-to-end point cloud rate control method and device based on reinforcement learning

Abstract

The application discloses a multi-scale end-to-end point cloud rate control method based on reinforcement learning, comprising the following steps: acquiring a point cloud sequence, and constructing a corresponding context state representation in units of point clouds of a current frame, wherein the context state representation comprises point cloud geometry information and auxiliary context information; composing a data set by using the point cloud sequence and the context state representations of all point clouds; constructing an initial model, comprising a state feature extraction module, an action decision module and a reward calculation module; training the initial model by using the data set to obtain a global rate-distortion rate control model for multi-scale end-to-end point cloud rate control; and inputting the point cloud sequence into the model to generate an encoding parameter action. The application further provides a multi-scale end-to-end point cloud rate control device. The method provided by the application can solve the difficulty that a current mainstream multi-scale sparse tensor point cloud compression framework cannot achieve a globally optimal rate-distortion performance while meeting a target rate constraint.

Description

Technical Field

[0001] This invention belongs to the field of point cloud geometric compression technology, and particularly relates to a multi-scale end-to-end point cloud bitrate control method and device based on reinforcement learning. Background Technology

[0002] Point clouds, as a mainstream form of 3D data representation, have been widely used in fields such as autonomous driving, virtual and augmented reality, robot navigation, and cultural heritage preservation. Point clouds typically consist of millions of unstructured points, capable of finely depicting the complex geometric details of objects and scenes; each point contains 3D coordinates (x, y, z) and can be supplemented with attribute information such as color, normal vector, or reflectivity. The sheer volume of point cloud data in real-world applications places enormous pressure on network transmission bandwidth and storage resources. To address this challenge, point cloud compression has become a key research direction, with its core objective being to effectively remove redundancy while preserving as much geometric structure and attribute information as possible.

[0003] In traditional methods, Geometry-based Point Cloud Compression (G-PCC) and Video-based Point Cloud Compression (V-PCC) proposed by the Moving Picture Experts Group (MPEG) are representative: G-PCC is mainly for static point clouds, while V-PCC is for dynamic point cloud sequences. Both employ hand-designed tools such as octree decomposition and 2D-3D projection to achieve lossy or lossless compression of point clouds.

[0004] With the rapid development of deep learning, learning-based point cloud compression methods have significantly outperformed traditional solutions in rate-distortion performance. Existing deep learning methods mainly fall into four categories: octree-based, point-based, voxel-based, and sparse tensor-based methods. Among them, octree-based methods construct hierarchical structures through recursive space partitioning, supporting arbitrary precision representations and utilizing node context for probabilistic modeling; point-based methods directly process the original point set, avoiding information loss caused by voxelization; voxel-based methods map point clouds to regular 3D meshes, capturing local geometric correlations through 3D convolution; and sparse tensor-based methods perform computations only at non-empty voxel locations, significantly reducing redundancy and maintaining high reconstruction quality while significantly improving computational efficiency, demonstrating outstanding advantages.

[0005] Building upon this foundation, the multi-scale sparse tensor point cloud compression framework has become the most advanced end-to-end compression paradigm. This framework constructs a multi-scale representation and flexibly adjusts the overall compression intensity by setting the first few scales to lossless compression and the subsequent scales to lossless compression. It has achieved leading rate-distortion performance in both static and dynamic point cloud tasks.

[0006] Patent document CN115103186A discloses a bitrate control method, device, electronic device, and storage medium. The method involves dividing a video frame to be encoded into multiple coding blocks and clustering them into multiple coding block classes. Within each coding block class, multiple sampled coding blocks are sampled and pre-encoded to determine the corresponding compression bitrate and Lagrange multiplier information. Based on the sum of all compression bitrates and the Lagrange multiplier information within each coding block class, block-level bitrate control parameters are determined. Based on the sum of all compression bitrates and the Lagrange multiplier information within the video frame to be encoded, frame-level bitrate control parameters are determined. Finally, based on a preset bitrate control model configured with block-level or frame-level bitrate control parameters, the actual encoded bits of the corresponding video frame to be encoded are controlled to reach a preset target bitrate.

[0007] Patent document CN121033191A discloses a point cloud geometric compression method and system based on transformer at the encoder end, including data preprocessing of acquired point cloud compression data; constructing a transformer-based point cloud compression model; performing point cloud geometric compression using the point cloud compression model based on the preprocessed point cloud compression data, including feature extraction of point cloud compression data at the encoder end; encoding and decoding of feature information; feature restoration at the decoder end; and training of the point cloud compression model based on the PyTorch encoding framework.

[0008] However, the current multi-scale sparse tensor framework combining lossless and lossy scales also introduces new challenges. On the one hand, the overall rate-distortion performance no longer presents the smooth, monotonic curve controlled by a single Lagrange multiplier in traditional video coding, but rather manifests as a series of discontinuous, non-convex, and mutually separated local RD segments depending on the combination of different lossless and lossy scales. On the other hand, to achieve globally optimal rate-distortion performance with a given bit budget, the encoder must simultaneously make joint decisions on the number of lossless scales, the number of lossy scales, and the Lagrange multipliers (used to control the compression intensity of the lossy scales). Adjusting the Lagrange multipliers alone or fixing the number of lossless scales easily leads to local suboptimal solutions, especially in dynamic point cloud sequences, where inter-frame reference further couples the reconstruction quality and bit allocation of each frame, making adaptive rate control particularly complex, and past methods are no longer applicable. Summary of the Invention

[0009] The purpose of this invention is to provide a multi-scale end-to-end cloud rate control method and apparatus based on reinforcement learning to solve the following key problems in the prior art: the rate-distortion performance of the current multi-scale sparse tensor point cloud compression framework exhibits segmented characteristics due to different scale combinations, inter-frame reference causes distortion propagation, and multiple parameters such as the number of lossless scales, the configuration of lossy scales, and the Lagrange multipliers of lossy scales need to be jointly decided simultaneously.

[0010] To achieve the first objective of this invention, the following technical solution is provided: a multi-scale end-to-end cloud code rate control method based on reinforcement learning, comprising the following steps: Obtain the point cloud sequence and construct the corresponding context state representation using the point cloud of the current frame as the unit. The context state representation includes point cloud geometric information and auxiliary context information. The dataset is composed of point cloud sequences and context state representations of all point clouds; Build an initial model, including a state feature extraction module, an action decision module, and a reward calculation module; The state feature extraction module is used to extract the context state representation of the input point cloud and generate a high-dimensional state vector based on the context state representation. The action decision module generates corresponding encoded action parameters based on the input high-dimensional state vector; Simultaneously, compression and reconstruction are performed based on the encoding parameters, and the corresponding performance indicators are calculated. The triple consisting of the high-dimensional state vector, the encoding parameter action, and the corresponding performance indicators is stored in the experience replay buffer for subsequent reward calculation module training. The reward calculation module samples the triplet data from the experience replay buffer to train and predict the corresponding reward value, and optimizes the model parameters based on the prediction results. The initial model was trained using the dataset to obtain a global rate-distortion rate control model for multi-scale end-to-end cloud rate control. When in use, the point cloud frame sequence to be processed is input into the global rate-distortion bitrate control model to generate coding parameter actions for compression and reconstruction.

[0011] This invention constructs a reinforcement learning agent for the entire point cloud sequence, enabling joint control of lossless scale number, lossy scale number, and Lagrange multipliers, thereby approximating the globally optimal rate-distortion performance while satisfying the target bit rate constraint.

[0012] Specifically, the point cloud geometric information includes the point cloud of the current frame to be encoded, the point cloud of the reconstructed reference frame, and the point cloud of the next N frames, where N=3.

[0013] Specifically, the auxiliary context information includes the target bit rate, remaining bit budget, historical coding actions, current frame number, and remaining frame number.

[0014] Specifically, the state feature extraction module is constructed based on a two-level sparse convolutional neural network and a fully connected sub-network; The two-level sparse convolutional neural network is used to extract features from the geometric information of the point cloud layer by layer, and generate geometric feature vectors through global max pooling operation; The fully connected subnetwork is used to extract features from the auxiliary context information to obtain auxiliary feature vectors; Geometric feature vectors and auxiliary feature vectors are concatenated along the channel dimension to form a unified high-dimensional state vector.

[0015] Specifically, the action decision module maps the high-dimensional state vector through a fully connected network, then transforms it into a corresponding discrete probability distribution using a Softmax function, and performs the following steps based on the discrete probability distribution: During the training phase, actions are sampled based on discrete probability distributions; During the testing phase, a greedy strategy is adopted to select the option with the highest probability among each action component and combine them to form a deterministic encoded parameter action.

[0016] Specifically, the encoding parameter action is a discrete triple, including the lossless encoding scale number, the lossy encoding scale number, and the Lagrange multiplier; (1) Number of lossless coding scales, which indicates how many scales are used for lossless compression in multi-scale geometric compression; (2) Number of lossy coding scales, which indicates how many scales are used for lossy compression in multi-scale geometric compression; (3) The Lagrange multiplier λ is used to adjust the trade-off point for the current frame rate distortion optimization.

[0017] Specifically, the rewards include immediate rewards and long-term rewards, as detailed below: ; in, This is the current distortion term, used to measure the reconstructed geometric distortion of the current frame. D represents the geometric distortion of the current frame. >0 represents the distortion weighting coefficient; This is the current bitrate deviation term, used to measure the deviation between the actual number of encoded bits in the current frame and the target bit budget. The total target bitrate of the sequence, Budget for the current remaining bits, >0 is the bitrate control weight, and it is increased in the last frame of the sequence to force strict compliance with the total bitrate constraint; = This is a temporal impact term used to measure the potential contribution of the current frame as a reference frame to the future reconstruction quality; it is a long-term reward. N represents the number of future frames considered, and γ(k)∈(0,1] is the time decay factor. This represents the amount of distortion reduction achieved by using the current frame as a reference for inter-frame prediction of the k-th future frame compared to independent intra-frame coding. This temporal impact term is calculated only during the training phase through auxiliary encoding and does not need to be executed during the inference phase. This allows for global rate-distortion performance optimization while satisfying the overall bitrate constraint and taking into account the reconstruction quality of the current frame and its temporal gain on subsequent frames.

[0018] To achieve the second objective of this invention, the following technical solution is provided: a multi-scale end-to-end cloud rate control device for executing the steps of the above-described reinforcement learning-based multi-scale end-to-end cloud rate control method.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: An end-to-end intelligent control mechanism is designed. Under the premise of satisfying the overall target bitrate constraint, the mechanism dynamically optimizes the sequence-level rate-distortion performance by explicitly modeling the context of the point cloud sequence and jointly selecting the number of lossless coding scales, the number of lossy coding scales, and the corresponding Lagrange multipliers for each frame of point cloud. The deployment and verification were carried out on a mainstream multi-scale point cloud compression framework, and the standard human point cloud sequence Owlii, defined in the international standard general test conditions, was used for testing. In summary, this invention significantly outperforms the current state-of-the-art method (Unicorn v3) in dynamic coding scenarios, achieving BD-BR gains of 12.34% (D1) and 12.83% (D2), with an average bitrate error of 4.07%, and a single frame time of only 0.17 seconds, with almost no increase in system overhead, effectively improving overall compression performance and practicality. Attached Figure Description

[0020] Figure 1 This is a flowchart of a multi-scale end-to-end cloud bitrate control method based on reinforcement learning provided in this embodiment; Figure 2 This is a schematic diagram of the structure of the model provided in this embodiment; Figure 3 This is a schematic diagram of the geometric information feature extraction unit provided in this embodiment; Figure 4 This is a schematic diagram of the auxiliary information feature extraction unit provided in this embodiment; Figure 5This is a schematic diagram of the action probability generation unit provided in this embodiment; Figure 6 This is a schematic diagram of the action decision unit provided in this embodiment; Figure 7 This is a schematic diagram of the multi-scale point cloud encoding / decoding module structure provided in this embodiment; Figure 8 This is a schematic diagram of the instant reward prediction unit structure provided in this embodiment; Figure 9 This is a schematic diagram of the long-term reward prediction unit structure provided in this embodiment; Figure 10 This is a schematic diagram of the experimental results provided in this embodiment. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0022] like Figure 1 As shown in this embodiment, a multi-scale end-to-end cloud bitrate control method based on reinforcement learning is provided, which includes the following steps: Obtain the point cloud sequence and construct the corresponding context state representation using the point cloud of the current frame as the unit. The context state representation includes point cloud geometric information and auxiliary context information. The dataset is composed of point cloud sequences and context state representations of all point clouds; An initial model is constructed and trained using a dataset to obtain a global rate-distortion rate control model for multi-scale end-to-end cloud rate control.

[0023] More specifically, construct the point cloud dataset: The MPEG standard dynamic point cloud test sequence was used as the training and validation dataset, and its content and format conform to the actual point cloud compression application scenario. Simultaneously, the network parameters of the state feature extraction module, action decision module, and reward prediction module were initialized, and the target bitrate, experience playback buffer capacity, and related training hyperparameters were configured to prepare for subsequent training.

[0024] When processing each training frame, a state vector is first constructed. This vector contains the point cloud of the current frame, the reference frame, preview point clouds of several future frames, and auxiliary context information. This information forms the context state representation, and a high-dimensional state vector is generated by the state feature extraction module. Subsequently, the action decision module generates specific encoding actions based on this high-dimensional state vector, including the number of lossless encoding scales, the number of lossy encoding scales, and the corresponding Lagrange multipliers λ. This action is fed into the multi-scale point cloud codec to perform the actual compression and reconstruction operations, and records the reconstruction distortion and actual bit consumption of the current frame.

[0025] Based on distortion and bit consumption, the system calculates an instant reward to measure the current frame’s performance in terms of rate-distortion tradeoff and bitrate control.

[0026] To further model the impact of the current decision on subsequent frames, the system also introduces a long-term reward: by analyzing the contribution of the current frame to the reconstruction quality of subsequent frames in the inter-frame prediction mode, the distortion propagation gain it brings is quantified, and this gain is converted into a positive impact on future performance. Finally, the corresponding high-dimensional state vector, action, and reward information are saved to the experience replay buffer.

[0027] The high-dimensional state vectors, actions, and rewards obtained in the previous step are stored in the experience replay buffer. A batch of data samples is periodically extracted from this buffer, and the reward prediction model is updated using temporal difference error. The action decision module is optimized using an entropy-regularized policy gradient method. In the early stages of training, the state feature extraction module can be fine-tuned; in later stages, its parameters are frozen to increase system stability. This process is repeated until the policy converges, and the optimal model is saved.

[0028] During the inference phase, the pre-trained state feature extraction module and action decision module are loaded.

[0029] For the input point cloud sequence, the state representation is extracted frame by frame, and a greedy strategy is adopted to determine the encoding parameters. Then, a multi-scale point cloud codec is configured according to the selected action to complete the compression and reconstruction of the point cloud.

[0030] like Figure 2 The diagram shown is a structural schematic of the global rate-distortion bitrate control model provided in this embodiment, including a state feature extraction module 100, an action decision module 200, and a reward prediction module 400.

[0031] Strategy optimization during the training phase S 1.1 Initialize the network parameters of the state feature extraction module, action decision module, and reward calculation module; S 1.2. Sample point cloud sequences from the training dataset; S 1.3 Perform the following sub-steps for each frame in the point cloud sequence: S 1.3.1 Construct a high-dimensional state vector using the state feature extraction module; S 1.3.2 Input the high-dimensional state vector into the action decision module to sample and generate coded actions from the discrete action space; S 1.3.3. Input the motion into the multi-scale point cloud codec, perform compression and reconstruction, and obtain the reconstruction distortion and actual bit consumption of the current frame; S 1.3.4. Based on distortion, bit consumption, and the impact on future frames, calculate the immediate reward and long-term reward, and synthesize the comprehensive reward; S 1.3.5 Store the current high-dimensional state vector, action, and reward into the experience replay buffer; S 1.4 Periodically sample a batch of interaction samples from the experience replay buffer and update the reward calculation module; S 1.5. Using the trained reward calculation module, predict the corresponding reward value and update the action decision module; S 1.6 In the early stage of training, the state feature extraction module and the action decision module are jointly optimized, and the state feature extraction module is frozen in the later stage of training to enhance the stability of the learning of the action decision module. S 1.7 Repeat steps s1.2–s1.6 until the policy converges, and save the optimal policy network for inference.

[0032] like Figure 3 and Figure 4 As shown, the state feature extraction module 100 includes a geometric information feature extraction unit 101 and an auxiliary information feature extraction unit 102. The geometric information feature extraction unit 101 takes point cloud information as input and extracts local geometric structure features through two levels of sparse convolutional layers, generating a compact geometric semantic vector through global max pooling. The auxiliary information feature extraction unit 102 receives the target bit rate, remaining bit budget, historical actions, frame number, and remaining frame count, encoding them through a "linear layer – activation function – linear layer" structure to obtain an auxiliary feature vector. Finally, the geometric feature vector and the auxiliary feature vector are concatenated to form a unified state feature vector, which serves as the basis for subsequent action decisions.

[0033] like Figure 5 and Figure 6As shown, the action decision module 200 includes an action probability generation unit 201 and an action decision unit 202. The action probability generation unit 201 takes state features as input and obtains the probability distribution of the corresponding action through a "linear layer – activation function – linear layer – probability normalization" process. The action decision unit 202 performs corresponding sampling based on the probability distribution to obtain the corresponding action information. The action information is then sent to a multi-scale point cloud codec 300 for dynamically configuring the encoding parameters of the current frame. The structure of the multi-scale point cloud codec 300 is shown below. Figure 7 As shown. The codec is configured according to the selected parameters and performs multi-scale compression and reconstruction, outputs the reconstructed point cloud, and records the reconstruction distortion and the actual number of bits consumed in the current frame.

[0034] like Figure 8 and Figure 9 As shown, the reward prediction module 400 calculates a comprehensive reward based on the aforementioned encoding results. This reward consists of two parts: an immediate reward and a long-term reward. Specifically, the immediate reward prediction unit 401 uses the current state features and the selected action as input, and predicts the immediate reward value reflecting the current frame rate distortion performance through a "linear layer-activation function-linear layer" network structure. Simultaneously, to evaluate the impact of the current decision on future frames, the long-term reward prediction unit 402 uses the state and action information of the subsequent three frames, also through a "linear layer-activation function-linear layer" structure, to predict the long-term benefits brought by the current action. Finally, the system weightedly sums the immediate reward and the long-term reward to obtain the comprehensive reward value used for policy updates.

[0035] This embodiment also provides a multi-scale end-to-end cloud rate control device for executing the steps of the reinforcement learning-based multi-scale end-to-end cloud rate control method provided in the above embodiment.

[0036] The bitrate control process of this device is as follows: S 2.1 Load the optimal state feature extraction module and action decision module saved during the training phase; S 2.2 Input the point cloud sequence to be compressed; S 2.3 For each frame, the current encoded state representation is extracted using the state feature extraction module; S 2.4 Input the state vector into the action decision module and generate coded actions using a greedy strategy; S 2.5 Configure the motion to the multi-scale point cloud codec to complete the compression and reconstruction of the current frame; S 2.6 Repeat steps s2.3–s2.5 until the end of the sequence, and output the complete compressed bitstream and the reconstructed point cloud sequence.

[0037] This embodiment was deployed and validated on a mainstream multi-scale point cloud compression framework. The standard human point cloud sequence Owlii, defined in the international standard general test conditions, was used for testing. The experimental results are as follows: Figure 10 As shown.

[0038] In dynamic coding scenarios, this technical solution demonstrates significant advantages over the current state-of-the-art method (Unicorn v3): BD-BR rate-distortion performance gains reach 12.34% (D1) and 12.83% (D2), respectively, with an average bit rate control error of only 4.07%, and a single-frame decision time of only 0.17 seconds, introducing almost no additional system overhead, effectively balancing compression efficiency, reconstruction quality, and practicality.

[0039] Example 2 For compression scenarios where future frame information cannot be obtained, a simplified reward mechanism based solely on immediate rewards can be chosen.

[0040] Specifically, the reward signal consists of the reconstruction distortion and bit consumption deviation of the current frame: reconstruction distortion is quantized using point cloud geometry metrics, while bit consumption deviation is measured as the difference between the actual number of coded bits in the current frame and the bit budget allocated according to the target bit rate of the sequence. The final reward value is obtained by weighted summation of these two metrics.

[0041] This mechanism does not rely on future frame information and still achieved a significant improvement in rate-distortion performance in experiments, with BD-BR gains reaching 8.35% (D1) and 8.71% (D2) and an average bit rate error of only 3.88%, effectively guiding the encoder to achieve a good balance between rate-distortion optimization and bit rate control accuracy.

[0042] Example 3 For applications sensitive to decision-making latency or with limited computing resources, the input point cloud geometric information is pre-downsampled, and the downsampled point cloud is then fed into the state feature extraction module to generate a high-dimensional state representation. This strategy significantly reduces the computational overhead of feature extraction and drastically shortens the processing time per frame. Although rate-distortion performance decreases slightly, its advantages are evident in scenarios with high real-time requirements or limited hardware resources. For example, with two-stage downsampling, the BD-BR gain can still reach 9.63% (D1) and 9.75% (D2), with an average bitrate error of 4.33%, while the decision time per frame is reduced from 0.163 seconds to 0.095 seconds. Users can flexibly adjust the downsampling ratio according to actual application needs, achieving an adaptive balance between coding efficiency and computational complexity.

[0043] Furthermore, the terms "upper," "lower," "inner," "outer," "front," and "rear" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0044] Of course, the above description is only a specific embodiment of the present invention and is not intended to limit the scope of the present invention. All equivalent changes or modifications made to the structure, features and principles described in the claims of the present invention should be included in the scope of the claims of the present invention.

[0045] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-scale end-to-end cloud rate control method based on reinforcement learning, characterized in that, Includes the following steps: Obtain the point cloud sequence and construct the corresponding context state representation using the point cloud of the current frame as the unit. The context state representation includes point cloud geometric information and auxiliary context information. The dataset is composed of point cloud sequences and context state representations of all point clouds; Build an initial model, including a state feature extraction module, an action decision module, and a reward calculation module; The state feature extraction module is used to extract the context state representation of the input point cloud and generate a high-dimensional state vector based on the context state representation. The action decision module generates corresponding encoded action parameters based on the input high-dimensional state vector; Compression and reconstruction are performed based on the encoding parameters, and the corresponding performance indicators are calculated. The triple consisting of the high-dimensional state vector, the encoding parameter action and the corresponding performance indicators are stored in the experience replay buffer for subsequent reward calculation module training. The reward calculation module samples the triplet data from the experience replay buffer to train and predict the corresponding reward value, and optimizes the model parameters based on the prediction results. The initial model was trained using the dataset to obtain a global rate-distortion rate control model for multi-scale end-to-end cloud rate control. During testing, the point cloud sequence to be processed is input into the global rate-distortion bitrate control model to generate encoding parameter actions for compression and reconstruction.

2. The multi-scale end-to-end cloud code rate control method based on reinforcement learning according to claim 1, characterized in that, The point cloud geometric information includes the point cloud of the current frame to be encoded, the point cloud of the reconstructed reference frame, and the point cloud of the next N frames, where N=3.

3. The multi-scale end-to-end cloud code rate control method based on reinforcement learning according to claim 1, characterized in that, The auxiliary context information includes the target bit rate, remaining bit budget, historical coding actions, current frame number, and remaining frame number.

4. The multi-scale end-to-end cloud code rate control method based on reinforcement learning according to claim 1, characterized in that, The state feature extraction module is constructed based on a two-level sparse convolutional neural network and a fully connected sub-network. The two-level sparse convolutional neural network is used to extract features from the geometric information of the point cloud layer by layer, and generate geometric feature vectors through global max pooling operation; The fully connected subnetwork is used to extract features from the auxiliary context information to obtain auxiliary feature vectors; Geometric feature vectors and auxiliary feature vectors are concatenated along the channel dimension to form a unified high-dimensional state vector.

5. The multi-scale end-to-end cloud code rate control method based on reinforcement learning according to claim 1, characterized in that, The action decision module maps the high-dimensional state vector through a fully connected network, then transforms it into a corresponding discrete probability distribution using a Softmax function, and performs the following steps based on the discrete probability distribution: During the training phase, actions are sampled based on discrete probability distributions; During the testing phase, a greedy strategy is adopted to select the option with the highest probability in each action component and combine them to form deterministic encoded parameter actions.

6. The multi-scale end-to-end cloud code rate control method based on reinforcement learning according to claim 1, characterized in that, The encoding parameters are discrete triples, including lossless encoding scales, lossy encoding scales, and Lagrange multipliers.

7. The multi-scale end-to-end cloud code rate control method based on reinforcement learning according to claim 1, characterized in that, The rewards include immediate rewards and long-term rewards, specifically as follows: ； in, This is the current distortion term, used to measure the reconstructed geometric distortion of the current frame. D represents the geometric distortion of the current frame. >0 represents the distortion weighting coefficient; This is the current bitrate deviation term, used to measure the deviation between the actual number of encoded bits in the current frame and the target bit budget. The total target bitrate of the sequence, Budget for the current remaining bits, >0 is the bitrate control weight, and it is increased in the last frame of the sequence to force strict compliance with the total bitrate constraint; = This is a temporal impact term used to measure the potential contribution of the current frame as a reference frame to the future reconstruction quality; it is a long-term reward. N represents the number of future frames considered, and γ(k)∈(0,1] is the time decay factor. This represents the amount of distortion reduction achieved by using the current frame as a reference for inter-frame prediction in the k-th future frame compared to independent intra-frame coding.

8. A multi-scale end-to-end cloud rate control device, characterized in that, The steps are for performing the reinforcement learning-based multi-scale end-to-end cloud bitrate control method as described in claims 1 to 7.

Images