Immersive image transmission method and system for AR cinema

Abstract

The invention discloses an immersive image transmission method and system for an AR cinema, and relates to the technical field of AR image transmission, and the method comprises the steps: optimizing a stereo unit set and a NeRF radiation field unit set through a variable weight joint optimization method, predicting the fusion probability of a stereo unit and a NeRF radiation field unit through a space-time Transform fusion method, carrying out the unit fusion according to the fusion probability, and carrying out the unit fusion. And mapping the fused and non-fused units to a three-dimensional grid structure to generate a stereoscopic NeRF hybrid grid. A variable weight joint optimization method is adopted, weights are dynamically calculated and optimized, priority optimization of a high-interaction area, resource allocation efficiency improvement and special effect area detail expression precision improvement are achieved, a space-time Transform fusion method is introduced, the fusion probability is predicted by utilizing a learnable weight matrix, semantic association of a stereoscopic unit and a NeRF unit is enhanced through space-time feature modeling, and the fusion efficiency of the stereoscopic unit and the NeRF unit is improved. Light field breakage and geometric distortion are eliminated, and the rendering quality of the AR cinema scene in the dynamic special effect area is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of AR image transmission, and particularly to an immersive image transmission method and system for an AR cinema. Background Art

[0002] In recent years, the application of augmented reality (AR) technology in the field of immersive cinemas has significantly promoted the development of real-time interaction and high-quality rendering technologies for virtual scenes. Existing technologies mainly rely on the processing of point cloud data, geometric modeling, and light field rendering to construct virtual scenes; among them, the discrete representation of the stereoscopic unit of point cloud data, combined with the neural radiance field (NeRF) technology to generate continuous light field information, has become the core method for constructing AR cinema scenes; there are significant deficiencies in the optimization and integration of the stereoscopic unit and the NeRF radiance field unit in the existing technology, which limits the rendering quality and real-time interaction experience of dynamic special effect areas (such as flames and magic light effects). The existing technology usually uses an optimization method with a fixed weight (such as standard gradient descent or mean aggregation) to adjust the parameters of the stereoscopic unit and the NeRF unit, ignoring the differences in interaction frequency and complexity among different units. This results in the optimization process being unable to dynamically adapt to the error distribution in high-interaction areas (such as special effect areas), causing uneven resource allocation. Static areas (such as walls) may be over-optimized, while the detail expression in dynamic areas is insufficient; existing fusion methods mostly rely on simple feature stitching or weighted averaging (such as feature fusion based on Euclidean distance), and do not fully consider the semantic relevance between the stereoscopic unit (discrete geometric structure) and the NeRF unit (continuous light field information). This leads to unnatural transitions in the dynamic special effect areas of the fused hybrid mesh, resulting in broken light field information or geometric distortion; when mapping the optimized unit to a three-dimensional mesh structure in the existing technology, uniform mesh division or static weighted mapping is usually adopted, ignoring the differences in interaction frequency and complexity in different areas of the scene. This results in insufficient mesh resolution in high-interaction areas (such as special effect areas), making it difficult to capture complex dynamic light field changes, while low-interaction areas (such as walls) may be allocated too many computing resources, reducing the overall efficiency; in addition, the existing method lacks a hierarchical division mechanism based on spectral clustering and dynamic weight allocation (such as focus layer, transition layer, background layer), and cannot achieve adaptive mesh optimization based on interaction frequency and complexity, affecting the rendering quality of the stereoscopic-NeRF hybrid mesh. Summary of the Invention

[0003] In view of the above existing problems, the present invention is proposed.

[0004] Therefore, the present invention provides an immersive image transmission method and system for an AR cinema, which solves the limitations of the unit optimization method, the deficiencies of the unit fusion strategy, and the inefficiency of mesh mapping and hierarchical allocation in the existing technology.

[0005] To solve the above technical problems, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides an immersive image transmission method for an AR cinema, which includes loading AR cinema virtual scene data by a cloud server, extracting a point cloud data set therefrom, and using a weighted K-means algorithm to synthesize each point in the point cloud data set into a stereoscopic unit set;

[0007] Based on the stereoscopic unit set, using a pre-trained GNN model to generate a dynamic point cloud subset, using an octree partitioning method to generate an initial grid from the point cloud subset, calculating the complexity characteristics of each grid region, and generating a NeRF radiation field unit set according to the complexity characteristics;

[0008] Optimizing the stereoscopic unit set and the NeRF radiation field unit set through a variable weight joint optimization method, using a spatio-temporal Transformer fusion method to predict the fusion probability of the stereoscopic unit and the NeRF radiation field unit, performing unit fusion according to the fusion probability, and mapping the fused and unfused units to a three-dimensional grid structure to generate a stereoscopic-NeRF hybrid grid;

[0009] Calculating the unit priority according to the collected real-time interaction data and the stereoscopic-NeRF hybrid grid, configuring a priority queue for the data within the unit using the QUIC protocol, and efficiently transmitting the data according to the QUIC priority queue.

[0010] As a preferred solution of the immersive image transmission method for an AR cinema according to the present invention, wherein: the step of loading AR cinema virtual scene data by a cloud server, extracting a point cloud data set therefrom, and using a weighted K-means algorithm to synthesize each point in the point cloud data set into a stereoscopic unit set means that the cloud server loads the AR cinema virtual scene and extracts a point cloud data set P = {Q1, Q2,..Q i}, and each point Q i = (x i , y i , z i , s i , g i , b i ) is each point cloud point Q i Calculate the interaction weight w i , randomly select a point from the point cloud data set P as the first center μ1 of the stereoscopic unit, use the weighted K-means++ algorithm to generate the nth stereoscopic unit center, calculate the minimum weighted distance from each point Q i to the selected stereoscopic unit center, and according to the weighted distance and the interaction weight w i , calculate the probability Y(Q i ) of each point being selected as the new center, use weighted random sampling, and according to the probability Y(Q iSelect a point from the set P as the new center U n , and repeat the iteration of the new center U n Generate steps to obtain a set of stereoscopic unit centers, and assign the point Q i to the stereoscopic unit V closest to the center point n . Form a set W = {V1, V2…, V n}, and calculate the point density ρ n inside the unit n .

[0011] As a preferred solution of the immersive image transmission method for an AR cinema described in the present invention, wherein: generating a dynamic point cloud subset using a pre-trained GNN model based on the stereoscopic unit set, generating an initial grid from the point cloud subset using an octree partitioning method, calculating the complexity features of each grid region, generating a NeRF radiation field unit based on the complexity features. Construct a point cloud map based on the stereoscopic unit set W, with the center point U n of each stereoscopic unit V n as a node, and the node feature vector is U n = [x n , y n , z n , r n , g n , b n , w i , ρ n . Use the k-nearest neighbor method to construct an edge of the point cloud map based on the Euclidean space distance between the stereoscopic unit centers. Use the pre-trained GNN model to predict the dynamic label l n for each node U n in the point cloud map. Generate a dynamic point cloud subset P ι according to the dynamic special effect area. Perform octree partitioning on the dynamic point cloud subset P ι to generate an initial grid G τ . Calculate the complexity feature c τ of each grid region. Calculate the color feature [s τ , g τ , b τ of each grid G τ by the principal component analysis method. Use the feature binding method to bind the complexity feature c τ and the color feature [s τ , g τ , b τ to the center coordinates (x τ , y τ , z τ , c τ , s τ , g τ , bτ ), to form a spatial point set \(\{(x τ , y τ , z τ , c τ )\}, and use the K-means clustering method to cluster the point set, set the number of clusters K, and each cluster represents a NeRF unit R n . Combine the R n units to obtain the NeRF unit set F = \{R1, R2, …, R n \}.

[0012] As a preferred solution of the immersive image transmission method for an AR cinema described in the present invention, wherein: the stereoscopic unit set and the NeRF radiation field unit set are optimized by the variable-weight joint optimization method, the spatio-temporal Transformer fusion method is used to predict the fusion probability of the stereoscopic unit and the NeRF radiation field unit, unit fusion is performed according to the fusion probability, and the fused and unfused units are mapped to a three-dimensional grid structure to generate a stereoscopic-NeRF hybrid grid, which refers to calculating the node feature vector U n average value of each dimension to generate the NeRF unit feature vector F j , use the Gaussian kernel similarity method to calculate the similarity J n between U j and the F nj unit feature vectors, construct the similarity matrix J based on the feature vector similarity J nj , construct the Laplacian matrix L norm , apply spectral clustering to the Laplacian matrix L norm to cluster into three layers, divide the units in the set A into three natural clusters through eigenvalue decomposition, set the high interaction frequency threshold H, the medium interaction frequency threshold C, and the complexity threshold O, and assign levels to each class according to the interaction frequency I a and the complexity c τ ;

[0013] According to the interaction frequency I a and the complexity c τ , calculate the average interaction frequency and complexity of each layer, dynamically assign the weight w layer,i , combine the stereoscopic unit set and the NeRF radiation field unit set to form the set A, A = \{W n , F j |n = 1, …, N, j = 1, …, N\} unit set, calculate the optimized weights w n and w j for each unit in the set A, minimize the error and regularization term of the stereoscopic unit and the NeRF radiation field unit through the variable-weight joint optimization method, use the Adam optimizer to iteratively adjust the unit parameters multiple times to generate the optimized units W n ' and Fj ’;

[0014] Predicting the fusion probability of the stereoscopic unit and the NeRF radiation field unit using a spatio-temporal Transformer fusion method Set the fusion probability threshold T. If the fusion probability is greater than the threshold T, then fuse the stereoscopic unit and the NeRF radiation field unit. If the fusion probability is less than the threshold T, then the stereoscopic unit and the NeRF radiation field unit are not fused, the original features are retained, and the fused and unfused units are mapped to a three-dimensional grid structure to generate a stereoscopic-NeRF hybrid grid.

[0015] As a preferred solution of the immersive image transmission method for an AR cinema according to the present invention, wherein: the calculating the unit priority according to the collected real-time interaction data and the stereoscopic-NeRF hybrid grid refers to collecting the real-time interaction data and the units in the hybrid grid and calculating the unit priority Set the unit priority threshold Ψ. If then it is a high-priority unit A high , otherwise it is a low-priority unit A low .

[0016] As a preferred solution of the immersive image transmission method for an AR cinema according to the present invention, wherein: the configuring the priority queue for the data within the unit using the QUIC protocol refers to configuring the TLS1.3 protocol, using the session parameters in TLS to initialize the QUIC connection, and calculating the priority w of the data stream according to the hierarchical unit features, combined with the interaction changes, the amount of compressed data, the network quality, and the device performance priority , and the priority w of the data priority is input into the QUIC implementation library. As the basis for the priority mapping configuration, the QUIC library performs pairing according to the TLS configuration and the real-time network performance, so that the high-interaction units are bound to the high-priority data streams, and the low-interaction units are assigned to the low-priority data streams of QUIC to fill the remaining bandwidth, optimizing the throughput.

[0017] As a preferred solution of the immersive image transmission method for an AR cinema according to the present invention, wherein: the efficient transmission of data according to the QUIC priority queue means compressing high-priority and low-priority data respectively by the Brotli and Gzip algorithms through the QUIC priority queue, encapsulating them into QUIC data packets and marking priorities. The QUIC protocol dynamically allocates bandwidth based on BBR congestion control and priority scheduling, preferentially transmits high-priority data streams, and adaptively adjusts according to network quality and interaction change rate. During the transmission process, the client receives data through QUIC, preferentially decodes high-priority data streams to render the focus area in real time, combines with the Unity fusional stereoscopic-NeRF grid to render the scene. The client receives QUIC data, preferentially decodes high-priority data streams, maps them to the stereoscopic-NeRF hybrid grid, and generates an AR scene by fusing in Unity. The rendering resolution is adjusted by dynamic resolution scaling according to the device performance. At the same time, the Prometheus tool is used to monitor latency, packet loss rate, and frame rate, and optimize the priority and scheduling strategy in real-time feedback.

[0018] In a second aspect, the present invention provides an immersive image transmission system for an AR cinema, including a data loading and point cloud extraction module for loading AR scene data from a cloud server and extracting a point cloud data set;

[0019] A stereoscopic unit generation module for generating a set of stereoscopic units, providing a discrete geometric structure of the point cloud, and supporting dynamic area division;

[0020] A NeRF radiation field unit generation module for generating an initial grid and a set of NeRF units, supporting fine expression of special effect areas;

[0021] A unit optimization and fusion module for optimizing unit parameters, fusing stereoscopic and NeRF radiation field units, and generating a high-quality hybrid grid;

[0022] A unit and data stream priority configuration and transmission module for allocating priorities to units and data streams, optimizing transmission efficiency, and ensuring that high-priority data streams arrive first.

[0023] In a third aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and wherein: when the computer program is executed by the processor, it implements any step of the immersive image transmission method for an AR cinema as described in the first aspect of the present invention.

[0024] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and wherein: when the computer program is executed by the processor, it implements any step of the immersive image transmission method for an AR cinema as described in the first aspect of the present invention.

[0025] The beneficial effects of the present invention are as follows: By adopting the variable-weight joint optimization method, the optimization weight is dynamically calculated, and combined with the Adam optimizer to iteratively adjust the parameters, the priority optimization of the high-interaction area is achieved, the resource allocation efficiency is improved, and the detail expression accuracy of the special effect area is enhanced; Secondly, aiming at the problem of insufficient fusion strategy in the prior art, this solution introduces the spatio-temporal Transformer fusion method, uses the learnable weight matrix to predict the fusion probability, enhances the semantic association between the stereoscopic unit and the NeRF unit through spatio-temporal feature modeling, eliminates light field breakage and geometric distortion, and improves the visual consistency of the fused grid; Aiming at the problems of inefficient grid mapping and hierarchical allocation, through the spectral clustering method and the dynamic weight allocation method, the focus layer, transition layer and background layer are partitioned based on the interaction frequency and complexity, and the grid resolution is dynamically adjusted, significantly improving the rendering quality, real-time interaction experience and computing efficiency of the AR cinema scene in the dynamic special effect area, providing users with a smooth and immersive viewing experience, and at the same time maintaining high-efficiency cloud-to-client transmission performance in complex interaction scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0027] Figure 1 It is a flowchart of an immersive image transmission method for an AR cinema in Embodiment 1.

[0028] Figure 2 It is a schematic structural diagram of an immersive image transmission system for an AR cinema in Embodiment 1.

[0029] Figure 3 It is a flowchart of the generation of a stereoscopic-NeRF hybrid grid for an immersive image transmission method for an AR cinema in Embodiment 1.

[0030] Figure 4 It is a flowchart of the QUIC priority transmission for an immersive image transmission method for an AR cinema in Embodiment 1. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] In order to make the above objects, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings of the specification.

[0032] In the following description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, the present invention may be practiced in other ways than those specifically described herein, and those skilled in the art can make similar extensions without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0033] Secondly, as used herein, "an embodiment" or "embodiments" refer to specific features, structures, or characteristics that may be included in at least one implementation of the present invention. The phrase "in an embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it an isolated or alternative embodiment that is mutually exclusive with other embodiments.

[0034] Example 1, referring to Figures 1 to 4 , is the first embodiment of the present invention. This embodiment provides an immersive image transmission method for an AR cinema, including the following steps:

[0035] S1. Load the AR cinema virtual scene data through a cloud server, extract the point cloud data set therefrom, and use the weighted K-means algorithm to synthesize each point in the point cloud data set into a stereoscopic unit set;

[0036] Based on the stereoscopic unit set, use a pre-trained GNN model to generate a dynamic point cloud subset, use the octree partitioning method to generate an initial grid from the point cloud subset, calculate the complexity features of each grid region, and generate a NeRF radiation field unit set according to the complexity features;

[0037] Specifically, loading the AR cinema virtual scene data through a cloud server and extracting the point cloud data set therefrom, and using the weighted K-means algorithm to synthesize each point in the point cloud data set into a stereoscopic unit set means that the cloud server loads the AR cinema virtual scene, including 3D models (OBJ format), 4K textures (PNG format), and PBR lighting effects (material parameters), and extracts the point cloud data set P = {Q1, Q2,..Q i}, each point Q i = (x i , y i , z i , s i , g i , b i )(x i , y i , z i is the 3D position, s i , g i , b i is the RGB color (the RGB color represents the visual attribute of the point and does not participate in the distance calculation), Q i represents the i-th point in the point cloud set P), and for each point cloud point Q iCalculate the interaction weight w i :

[0038]

[0039] where Q f is the f-th focus coordinate. The perspective focus coordinate (provided by the eye tracker) is converted into a 3D space coordinate Q f =(x f , y f , z f ), is the Gaussian kernel width, which controls the focus area range. The optimal one is selected by using cross-validation and evaluating the impact of different Gaussian kernel widths on the model performance β is the interaction frequency factor, which is automatically adjusted by optimizing the objective function (such as the accuracy of the interaction effect). I t is the interaction frequency of the t-th point, which is obtained through multi-sensor data fusion;

[0040] Randomly select a point from the point cloud dataset P as the first center μ1 of the stereoscopic unit, and use the weighted K-means++ algorithm to generate the n-th stereoscopic unit center (n = 2, 3,..., N). Calculate the minimum weighted distance from each point Q i to the selected stereoscopic unit centers {μ1,..., μ n -1}. According to the weighted distance and the interaction weight w i , calculate the probability Y(Q i ) that each point is selected as the new center:

[0041]

[0042] where Q j is the j-th point in the point cloud set P, Q j is also a vector of the same dimension, and its structure is the same as that of π i . w j is the weight corresponding to Q j , which is used to calculate the weighted distance of each point in the denominator. min n ||Q i -μ n || 2 is the closest distance from the point to the n-th stereoscopic unit center. High-weight points (such as special effect areas) are more likely to be selected as the center to ensure a dense center distribution;

[0043] Use weighted random sampling to select a point from the set P as the new center U i according to the probability Y(Q n(The probability value is proportional to the square of the interaction weight and the minimum weighted distance, making the probability of points with high interaction weights (such as water flow and flame effect areas) farther from the existing centers higher. Normalize the probabilities of all points to form a probability distribution and accumulate it to the interval [0, 1]. Generate a random number uniformly distributed within [0, 1], and select the corresponding point Q according to the probability interval into which the random number falls i as the new center U n , ensuring that the new centers are distributed in high interaction areas (such as effect areas) and not too densely (avoiding overlap through the minimum weighted distance). Repeat the iteration of the new center U n to generate the steps (calculate distances, probabilities, sampling, and generate new centers each time), obtaining the set of stereoscopic unit centers {U1, U2, …, U n};

[0044] Assign the point Q i to the stereoscopic unit V that is closest to the center point n :

[0045] V n = {Q i | n = arg min n ||Q i - U n || 2},

[0046] where argmin is a mathematical operator representing finding the stereoscopic unit that minimizes the weighted distance among all stereoscopic units;

[0047] Form the set of stereoscopic units V n into the set W = {V1, V2, …, V n}. The set of stereoscopic units provides the discrete geometric structure of the point cloud, and calculate the point density ρ n :

[0048]

[0049] where {Q i ∈ V n} is the number of point cloud points in the unit V n , refers to the volume of the Ball sphere with the center U n as the center of the sphere and the radius r n , and r n = max Q i ∈ V n ||Q i - U n ||, which refers to the maximum distance from the points in the unit to the center.

[0050] By combining interaction weights, this method preferentially selects points in high-interaction regions as the centers of stereoscopic units, making the center distribution denser, improving the geometric structure expression accuracy. The K-means++ algorithm is used to ensure that the new centers are distributed in high-interaction regions and avoid overlap through weighted random sampling and the minimum weighted distance probability, significantly enhancing the detail capture ability of the special effect region. Compared with traditional K-means clustering, the consistency of unit division in dynamic regions is improved, and the point density within the computing unit further optimizes resource allocation, reducing the computational overhead in static regions, and significantly enhancing the visual realism and immersive interaction experience in the dynamic special effect region of the AR cinema scene.

[0051] Furthermore, based on the set of stereoscopic units, a pre-trained GNN model is used to generate a dynamic point cloud subset. The octree partitioning method is used to generate an initial mesh from the point cloud subset, and the complexity characteristics of each mesh region are calculated. According to the complexity characteristics, a NeRF radiation field unit is generated. A point cloud map is constructed based on the set of stereoscopic units W, with the center point U n of each stereoscopic unit V n as nodes, and the node feature vector is The node feature vector is used as the stereoscopic unit feature vector. The k-nearest neighbor (such as k-NN, k = 8) method is used to construct the edges of the point cloud map based on the Euclidean space distance (the straight-line distance between three-dimensional coordinates) between the centers of the stereoscopic units. The pre-trained GNN model is used to predict the dynamic label l n for each node U in the point cloud map n :

[0052]

[0053] where w is the GNN weight, optimized through the pre-trained GNN model, and Aggregate is the aggregation function. Among them, is the edge weight, calculated by applying the Gaussian kernel function. is the set of neighbor nodes of node n (determined by the k-NN (k = 8) method, including the k other center points with the smallest Euclidean distance from the center point U n in three-dimensional space), is the th unit node feature, σ is the Sigmoid function, which maps the output of the GNN model to [0, 1]. The dynamic label l n ∈[0, 1], θ is a predefined threshold, determined by the cross-validation method, and Π is the indicator function. If ≥θ, then l n = 1 indicates a dynamic special effect region (such as flames, magic light effects, composed of all stereoscopic units V where l n = 1) nSet definition, the area range is determined by the geometric boundary of the point cloud points within the unit), otherwise l n = 0 indicates a static area (such as a wall);

[0054] According to the dynamic special effect area (l n = 1), generate a dynamic point cloud subset:

[0055] P l = {Q i |l n = 1, Q i ∈V n},

[0056] Perform octree partitioning on the dynamic point cloud subset P ι to generate the initial grid G τ , (Input the point cloud data into the octree, determine the spatial boundary, construct a cubic bounding box containing all points, recursively divide the bounding box into eight sub-cubes to form an octree structure, each sub-cube corresponds to a node, if the number of points in the node exceeds the preset threshold, continue to subdivide, otherwise, the node becomes a leaf node, and the point cloud data in the leaf node generates the initial grid), calculate the complexity characteristics of each grid area:

[0057]

[0058] Among them, E τ is the volume of the grid G τ , Z τ is the side length of the cubic grid, α τ is the τ-th grid interaction factor, c τ is the complexity of the τ-th quantization area;

[0059] Calculate the color characteristics [s τ , g τ , b τ of each grid G τ by the principal component analysis method, and use the feature binding method to bind the complexity characteristic c τ and the color characteristic [s τ , g τ , b τ to the center of the grid (referring to the geometric center of the grid area G τ (a cube), represented as a three-dimensional coordinate point) coordinates (x τ , y τ , z τ , c τ , s τ , g τ , b τ ), forming a spatial point set {(x τ , yτ , z τ , c τ )}, use the K-means clustering method to cluster the point set, set the number of clusters K (such as 3, representing low, medium, and high complexity intervals) through the silhouette coefficient method, and each cluster represents a NeRF unit R n , and combine the R n units to obtain the NeRF unit set F = {R1, R2, …, R n}. The NeRF radiation field unit provides continuous scene light field information (such as color, density).

[0060] Aiming at the problems of insufficient recognition accuracy of dynamic special effect areas, low grid division efficiency, and lack of continuous light field expression in the generation of NeRF units in the existing AR cinema image transmission technology, a method for generating dynamic point cloud subsets and constructing NeRF units based on a pre-trained GNN model and octree division method is proposed. Predict dynamic labels through the pre-trained GNN model, calculate edge weights by combining the k-nearest neighbor method and Gaussian kernel function, accurately identify dynamic special effect areas, improve the generation accuracy of dynamic point cloud subsets, use the octree division method to generate initial grids, recursively subdivide to ensure the grid resolution of high-interaction areas, reduce the calculation of static areas, generate a NeRF unit set by calculating and binding color features and complexity features, and combining the K-means clustering method, provide continuous scene light field information, and improve the rendering quality, significantly enhancing the detail expression and rendering consistency of dynamic special effect areas in the AR cinema scene.

[0061] S2. Optimize the stereoscopic unit set and NeRF radiation field unit set through the variable weight joint optimization method, use the spatio-temporal Transformer fusion method to predict the fusion probability of the stereoscopic unit and the NeRF radiation field unit, perform unit fusion according to the fusion probability, and map the fused and unfused units to a three-dimensional grid structure to generate a stereoscopic-NeRF hybrid grid;

[0062] Calculate the unit priority according to the collected real-time interaction data and the stereoscopic-NeRF hybrid grid;

[0063] Specifically, optimize the stereoscopic unit set and NeRF radiation field unit set through the variable weight joint optimization method, use the spatio-temporal Transformer fusion method to predict the fusion probability of the stereoscopic unit and the NeRF radiation field unit, perform unit fusion according to the fusion probability, and map the fused and unfused units to a three-dimensional grid structure to generate a stereoscopic-NeRF hybrid grid refers to calculating the node feature vector U n The average value of each dimension to generate the NeRF unit feature vector F j :

[0064]

[0065] Calculate U using the Gaussian kernel similarity method n with F j the similarity J of the unit feature vectors nj :

[0066]

[0067] where, ||U n - F j || 2 is the Euclidean distance between the feature vectors, is the ζ-th similarity kernel width, obtained by the cross-validation method, and J nj is the similarity between the n-th stereoscopic unit and the feature vectors of the j-th NeRF radiation field unit;

[0068] Based on the feature vector similarity J nj construct the similarity matrix J and construct the Laplacian matrix L norm :

[0069]

[0070] L norm = D -1 / 2 (D - J)D -1 / 2 ,

[0071] where, is the -th element of the degree matrix D, represent the row index and column index of the degree matrix D respectively;

[0072] Apply spectral clustering to the Laplacian matrix L norm , cluster into three layers, divide the units in set A into three natural clusters through eigenvalue decomposition, set the hierarchical threshold, high interaction frequency threshold H, medium interaction frequency threshold C and complexity threshold O according to the interaction frequency I a (obtain the interaction frequency I of each unit through the kernel density weighting method a ) and complexity c τ Assign levels to each class as follows:

[0073] The focus layer (high interaction area such as special effects) is for units where I a > H and c τ > O;

[0074] The transition layer (medium interaction area) is for units where C < I a ≤ H;

[0075] The background layer (low interaction area such as walls) is for units where I a ≤ C;

[0076] According to the interaction frequency I a and complexity c τ , calculate the average interaction frequency and complexity of each layer, and dynamically allocate the weight w layer,i , (calculate the average interaction frequency I of all units in each layer by the dynamic weight allocation method layer , use the clustering weighting method to calculate the average complexity c of all units in each layer layer , add the I layer and c layer of each layer to obtain the dynamic importance score of this layer, and divide the scores of each layer by the sum of the scores of all layers to obtain the normalized weight w layer,i , the focus layer is assigned the highest weight due to the highest interaction frequency and complexity, the transition layer has the second highest weight, and the background layer has the lowest weight. The weights are normalized (the sum is 1));

[0077] Combine the stereoscopic unit set and the NeRF radiation field unit set to form a set A, A = {W n , F j ∣n = 1,..., N, j = 1,..., N} (N is the total number of stereoscopic units and NeRF radiation field units respectively), and calculate the optimized weight of each unit in set A:

[0078]

[0079] Among them, i is the error term in the stereoscopic unit, referring to the point cloud reprojection error, which is obtained through the point cloud reprojection error calculation method, is the error term in the NeRF unit, referring to the rendering color error, which is obtained through the radiation field rendering error integration method, δ = 10 -3 is a small positive number, obtained by the numerical stability setting method to prevent the denominator from being zero, is the interaction factor, obtained through the hyperparameter tuning method, w n is the optimized weight of the nth stereoscopic unit, w j is the jth NeRF radiation field unit;

[0080] Minimize the errors and regularization terms of the stereoscopic unit and the NeRF radiation field unit through the variable weight joint optimization method, and use the Adam optimizer to iteratively adjust the unit parameters multiple times to generate the optimized unit:

[0081]

[0082] Among them, w vg is the weight mean, obtained through the mean aggregation method, λ is the regularization weight (obtained by tuning through the cross-validation method) W n ’ is the nth optimized stereoscopic unit, F j’ is the j-th optimized NeRF radiation field unit;

[0083] Use the spatio-temporal Transformer fusion method to predict the fusion probability between the stereoscopic unit and the NeRF radiation field unit:

[0084]

[0085] Among them, is the fusion probability between the n-th stereoscopic unit and the j-th NeRF radiation field unit, is a learnable weight matrix, which is trained by minimizing the cross-entropy loss;

[0086] Set the fusion probability threshold T through cosine similarity. If the fusion probability is greater than the threshold T, then fuse the stereoscopic unit and the NeRF radiation field unit;

[0087] If the fusion probability is less than the threshold T, then the stereoscopic unit and the NeRF radiation field unit are not fused, and the original features are retained. Map the fused and unfused units to a three-dimensional grid structure. (Map the fused units (weighted average of point cloud and radiation field parameters) and unfused units (original features) to a three-dimensional grid through the weighted mapping method. If multiple units fall into the same grid, fuse the features through weighted average) to generate a stereoscopic-NeRF hybrid grid.

[0088] Aiming at the problems of insufficient dynamic adaptability of unit optimization, lack of fusion semantics, inefficient grid allocation, and high computational complexity in the existing AR cinema image transmission technology, through the variable-weight joint optimization method, using the point cloud reprojection error and interaction factors, dynamically adjust the optimization weights, and preferentially enhance the detail expression in high-interaction regions, improving the rendering quality and resource allocation efficiency; adopt the spatio-temporal Transformer fusion method, predict the fusion probability through a learnable weight matrix, fuse the units in high-interaction regions to optimize the grid resolution allocation, achieve semantic consistency fusion between the stereoscopic unit and the NeRF unit, enhance the detail expression in the special effect region, eliminate the light field break, improve the visual consistency, use the spectral clustering method and the dynamic weight allocation method, divide the focus layer, transition layer and background layer based on the interaction frequency and complexity, optimize the grid resolution allocation, enhance the details in high-interaction regions, reduce the computational overhead in static regions, and generate a stereoscopic-NeRF hybrid grid through optimizing the construction of the similarity matrix and cooperating with the weighted mapping method to ensure a significant improvement in the rendering realism and real-time interaction experience in the AR cinema scene.

[0089] Furthermore, calculate the unit priority according to the collected real-time interaction data and the stereoscopic-NeRF hybrid grid. The collected real-time interaction data includes interaction frequency, sensor data, network status, device performance, and units in the hybrid grid. Calculate the unit priority:

[0090]

[0091] Among them, is the priority weight of the th unit, Q f is the focus of the max th perspective, obtained through eye tracking, I f is the maximum interaction frequency, obtained through the max-pooling method, d(Q f ,U n ) is the distance from the stereoscopic center to the focus, obtained through geometric calculation, η ∈ [0, 1] is the network state factor, such as width, latency, and packet loss rate, quantifying the real-time performance of the network, γ ∈ [0, 1] is the device performance factor, obtained through the weighted average of frame rate, bandwidth, computing power, and memory;

[0092] Set the unit priority threshold Ψ through the analytic hierarchy process. If then it is a high-priority unit A high , otherwise it is a low-priority unit A low .

[0093] Aiming at the problems of lack of dynamic adaptability in data priority allocation, inability to effectively cope with real-time interaction changes and network fluctuations in the existing AR cinema video transmission technology, by comprehensively considering interaction frequency, perspective focus, geometric distance, network state factor, and device performance factor, using the analytic hierarchy process to calculate the unit priority weight, realizing the priority processing of high-interaction areas, improving the resource allocation efficiency. Compared with the traditional static priority rules, this method dynamically adapts to changes in network bandwidth, latency, and device performance, ensuring a reduction in the transmission latency of high-priority units and an improvement in the stability of the rendering frame rate. By using the max-pooling method and eye-tracking method to accurately identify the user interaction focus, optimizing the data scheduling in the special effect area, significantly enhancing the immersive interaction experience of the AR cinema scene, improving the overall smoothness, and providing efficient support for high-quality viewing in complex interaction environments.

[0094] S3. Use the QUIC protocol to configure a priority queue for the data within the unit, and perform efficient data transmission according to the QUIC priority queue;

[0095] Specifically, using the QUIC protocol to configure a priority queue for the data within the unit means configuring the TLS1.3 protocol, using the session parameters (such as session keys) in TLS to initialize the QUIC connection, ensuring that the priority queue configuration is executed on a secure QUIC connection, and calculating the priority of the data stream according to the hierarchical unit characteristics, combined with interaction changes, compressed data volume, network quality, and device performance:

[0096]

[0097] Among them, σ represents the Sigmoid function, and w1, w2, and w3 are dynamically adjusted weights (obtained through adaptive weight optimization). is the τ-th grid feature vector (during the generation of the stereo-NeRF hybrid grid, a feature vector assigned to each three-dimensional grid cell, (such as coordinates, color, interaction factor, density, complexity), generated by the weighted mapping method). is the th compressed data volume (obtained through data stream compression and statistical methods). is the th network quality (such as bandwidth fluctuation and delay jitter. Bandwidth fluctuation calculates the standard deviation of the time series through continuous bandwidth tests (such as iPerf), and delay jitter extracts the delay variance from network analysis tools (such as Wireshark)) γ e is the e-th device performance factor.

[0098] The priority w of the data priority is input into the QUIC implementation library as the basis for priority mapping configuration. The QUIC library performs pairing according to the TLS configuration and real-time network performance (such as bandwidth, delay), enabling high-interaction units (focus layers) to bind high-priority data streams, and low-interaction units (transition layers and background layers) to be assigned to QUIC low-priority data streams to fill the remaining bandwidth, optimizing throughput.

[0099] Aiming at the problems of static data transmission priority allocation, inability to adapt to network fluctuations and interaction changes in the existing AR cinema video transmission technology, by combining hierarchical unit features, interaction changes, compressed data volume, network quality, and device performance, using the Sigmoid function and dynamically adjusting weights, the data stream priority is calculated to achieve the priority transmission of high-interaction area data streams, reducing the transmission delay. Initialize a secure QUIC connection using the TLS1.3 protocol, and dynamically pair high-priority data streams through the QUIC implementation library to optimize throughput. The data streams in the low-interaction area fill the remaining bandwidth, improving resource utilization. Compared with the traditional TCP protocol or static scheduling, through BBR congestion control and real-time network performance feedback, the bandwidth allocation is adaptively adjusted to ensure the improvement of the interaction fluency in the AR cinema scenario under complex network environments, significantly enhancing the immersive viewing experience.

[0100] Furthermore, efficient data transmission according to the QUIC priority queue means compressing high-priority and low-priority data through the Brotli and Gzip algorithms respectively in the QUIC priority queue, encapsulating them into QUIC data packets and marking the priorities. The QUIC protocol dynamically allocates bandwidth based on BBR congestion control and priority scheduling, preferentially transmitting high-priority data streams, and adaptively adjusts according to network quality and interaction change rate. During the transmission process, the client receives data through QUIC, preferentially decodes high-priority data streams for real-time rendering of the focus area, and combines Unity's stereoscopic-NeRF grid to render the scene (in the Unity engine, using the stereoscopic-NeRF rendering pipeline, mapping the decoded data to the stereoscopic-NeRF hybrid grid, and generating an AR scene through the fusion of stereoscopic point cloud rendering (Open3D) and NeRF light field rendering (PyTorch), and determining the rendering resolution according to the device performance k). The client receives QUIC data, preferentially decodes high-priority data streams, maps them to the stereoscopic-NeRF hybrid grid (rendering the stereoscopic point cloud through Open3D and the NeRF light field through PyTorch), and generates an AR scene by fusing in Unity. The rendering resolution is adjusted through dynamic resolution scaling according to the device performance (CPU / GPU computing power collected by sensors) (range 0.5x - 1.0x). At the same time, the Prometheus tool monitors latency, packet loss rate, and frame rate in real time, provides real-time feedback to optimize priorities and scheduling strategies, forms a closed-loop optimization, and ensures a smooth experience of the interactive scene.

[0101] Aiming at the problems of high transmission latency, lack of adaptability in priority scheduling, and limited rendering efficiency in the existing AR cinema image transmission technology, by compressing high-priority and low-priority data through the Brotli and Gzip algorithms respectively, combining the BBR congestion control and priority scheduling of the QUIC protocol, dynamically allocating bandwidth, preferentially transmitting data streams in high-interaction areas, reducing transmission latency and increasing throughput. Using the Unity stereoscopic-NeRF rendering pipeline, mapping it to the stereoscopic-NeRF hybrid grid in real time, improving the detail accuracy of the focus area rendering, adapting to device performance through dynamic resolution scaling, improving frame rate stability, and significantly improving the fluency of low-end devices. The Prometheus tool monitors latency, packet loss rate, and frame rate in real time, forms a closed-loop optimization mechanism, dynamically adjusts priorities and scheduling strategies, improves interactive fluency, and significantly enhances the immersive viewing experience of the AR cinema scene.

[0102] This embodiment also provides an immersive image transmission system for an AR cinema, including:

[0103] A data loading and point cloud extraction module, used to load AR scene data from a cloud server and extract a point cloud data set;

[0104] A stereoscopic unit generation module, which is used to generate a set of stereoscopic units, provide a discrete geometric structure of the point cloud, and support dynamic region division;

[0105] A NeRF radiation field unit generation module, which is used to generate an initial mesh and a set of NeRF units, and support the fine expression of special effect areas;

[0106] A unit optimization and fusion module, which is used to optimize unit parameters, fuse stereoscopic and NeRF radiation field units, and generate a high-quality hybrid mesh;

[0107] A unit and data stream priority configuration and transmission module, which is used to assign priorities to units and data streams, optimize transmission efficiency, and ensure that high-priority data streams arrive first.

[0108] This embodiment also provides a computer device, which is applicable to a situation of an immersive image transmission method for an AR cinema, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement an immersive image transmission method for an AR cinema as proposed in the above embodiment.

[0109] This computer device may be a terminal. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0110] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the immersive image transmission method and system for an AR cinema as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM for short), electrically erasable programmable read-only memory (EEPROM for short), erasable programmable read-only memory (EPROM for short), programmable read-only memory (PROM for short), read-only memory (ROM for short), magnetic memory, flash memory, magnetic disk or optical disc.

[0111] In summary, the present invention adopts a variable-weight joint optimization method, dynamically calculates and optimizes weights, and combines the Adam optimizer to iteratively adjust parameters to achieve priority optimization of high-interaction areas, improve resource allocation efficiency, and enhance the accuracy of detail expression in special effect areas; secondly, in response to the problem of insufficient fusion strategies in the prior art, this solution introduces a spatio-temporal Transformer fusion method, uses a learnable weight matrix to predict the fusion probability, enhances the semantic association between the stereoscopic unit and the NeRF unit through spatio-temporal feature modeling, eliminates light field breaks and geometric distortions, and improves the visual consistency of the fusion grid; for the problem of inefficient grid mapping and hierarchical allocation, through spectral clustering and dynamic weight allocation methods, the focus layer, transition layer, and background layer are divided based on interaction frequency and complexity, and the grid resolution is dynamically adjusted, significantly improving the rendering quality, real-time interaction experience, and computing efficiency of the AR cinema scene in the dynamic special effect area, providing users with a smooth and immersive viewing experience, and maintaining high-efficiency cloud-to-client transmission performance in complex interaction scenarios.

[0112] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. An immersive video transmission method for an AR cinema, characterized in that: including loading AR cinema virtual scene data through a cloud server, extracting a point cloud data set therefrom, and using the weighted K-means algorithm to synthesize each point in the point cloud data set into a stereoscopic unit set; generating a dynamic point cloud subset using a pre-trained GNN model based on the stereoscopic unit set, generating an initial grid using the octree partitioning method for the point cloud subset, calculating the complexity features of each grid region, and generating a NeRF radiation field unit set according to the complexity features; optimizing the stereoscopic unit set and the NeRF radiation field unit set through a variable-weight joint optimization method, predicting the fusion probability of the stereoscopic unit and the NeRF radiation field unit using a spatio-temporal Transformer fusion method, performing unit fusion according to the fusion probability, and mapping the fused and unfused units to a three-dimensional grid structure to generate a stereoscopic-NeRF hybrid grid; calculating the unit priority according to the collected real-time interaction data and the stereoscopic-NeRF hybrid grid, configuring a priority queue for the data within the unit using the QUIC protocol, and efficiently transmitting the data according to the QUIC priority queue.

2. The immersive video transmission method for an AR cinema according to claim 1, wherein: The AR cinema virtual scene data is loaded through the cloud server, and the point cloud data set is extracted from it. The weighted K-means algorithm is used to synthesize each point in the point cloud data set into a stereoscopic unit set, which means that the cloud server loads the AR cinema virtual scene and extracts the point cloud data set P = {Q1, Q2,..Q i}, each point Q i = (x i , y i , z i , s i , g i , b i ) is each point cloud point Q i Calculate the interaction weight w i , randomly select a point from the point cloud data set P as the first center μ1 of the stereoscopic unit, use the weighted K-means++ algorithm to generate the nth stereoscopic unit center, and calculate the minimum weighted distance from each point Q i to the selected stereoscopic unit center. According to the weighted distance and the interaction weight w i , calculate the probability Y(Q i ) that each point is selected as the new center. Use weighted random sampling, and select a point from the set P as the new center U i according to the probability Y(Q n ). Repeat the iteration of the new center U n generation step to obtain the stereoscopic unit center set, and assign the point Q i to the stereoscopic unit V n with the closest distance to the center point. The stereoscopic unit V n forms a set W = {V1, V2…, V n}, and calculate the point density ρ n .

3. The immersive video transmission method for an AR cinema according to claim 2, wherein: Generate a dynamic point cloud subset using a pre-trained GNN model based on a set of stereoscopic units. Generate an initial mesh from the point cloud subset using an octree partitioning method. Calculate the complexity features of each mesh region. Generate a NeRF radiation field unit based on the complexity features, referring to constructing a point cloud map based on the set of stereoscopic units W, with each stereoscopic unit V n The center point U n As a node, the node feature vector is U n =[x n , y n , z n , r n , g n , b n , w i , ρ n . Use the k-nearest neighbor method to construct the edges of the point cloud map based on the Euclidean spatial distance between the centers of the stereoscopic units. Use the pre-trained GNN model to predict the dynamic label l n for each node U n in the point cloud map. Generate a dynamic point cloud subset P i according to the dynamic special effect area. Perform octree partitioning on the dynamic point cloud subset P ι to generate an initial mesh G τ . Calculate the complexity feature c τ of each mesh region. Calculate the color feature [s τ , g τ , b τ of each mesh G τ using the principal component analysis method. Use the feature binding method to bind the complexity feature c τ and the color feature [s τ , g τ , b τ to the center coordinates (x τ , y τ , z τ , c τ , s τ , g τ , b τ ) of the mesh to form a spatial point set {(x τ , y τ , z τ , c τ ). Use the K-means clustering method to cluster the point set, set the number of clusters K, and each cluster represents a NeRF unit R n . Combine the R n units to obtain the NeRF unit set F = {R1, R2, …, R n}.

4. The immersive video transmission method for an AR cinema according to claim 3, characterized in that: The stereoscopic unit set and the NeRF radiation field unit set are optimized by the variable-weight joint optimization method. The spatio-temporal Transformer fusion method is used to predict the fusion probability of the stereoscopic unit and the NeRF radiation field unit. Unit fusion is performed according to the fusion probability, and the fused and unfused units are mapped to a three-dimensional grid structure to generate a stereoscopic-NeRF hybrid grid, which refers to calculating the node feature vector U by the mean aggregation method n The average value of each dimension to generate the NeRF unit feature vector F j , and the Gaussian kernel similarity method is used to calculate U n and F j The similarity J of the unit feature vectors nj , based on the feature vector similarity J nj Construct the similarity matrix J and construct the Laplacian matrix L norm , and apply spectral clustering to the Laplacian matrix L norm Cluster into three layers, divide the units in the set A into three natural clusters by eigenvalue decomposition, set the high interaction frequency threshold H, the medium interaction frequency threshold C, and the complexity threshold O, and according to the interaction frequency I of the unit a and complexity c τ Assign levels to each class; According to the interaction frequency I a and complexity c τ , calculate the average interaction frequency and complexity of each layer, dynamically allocate the weight w layer,i , form a set A by combining the stereoscopic unit set and the NeRF radiation field unit set, A = {W n , F j |n = 1,..., N, j = 1,..., N}, calculate the optimized weights w n and w j for each unit in set A, minimize the error and regularization term of the stereoscopic unit and the NeRF radiation field unit through the variable-weight joint optimization method, use the Adam optimizer to iteratively adjust the unit parameters multiple times, and generate the optimized units W n ' and F j '; Predicting the Fusion Probability of Stereo Units and NeRF Radiance Field Units Using a Spatiotemporal Transformer Fusion Method Set the fusion probability threshold T. If the fusion probability is greater than the threshold T, then fuse the stereo unit and the NeRF radiance field unit. If the fusion probability is less than the threshold T, then the stereo unit and the NeRF radiance field unit are not fused, the original features are retained, and the fused and unfused units are mapped to a three-dimensional grid structure to generate a stereo-NeRF hybrid grid.

5. The immersive video transmission method for an AR cinema according to claim 4, characterized in that: The priority of the volumetric-NeRF hybrid grid calculation unit based on the collection of real-time interaction data refers to collecting real-time interaction data and the units in the hybrid grid, calculating the unit priority wθ, setting the unit priority threshold Ψ. If wθ > Ψ, it is a high-priority unit A high , otherwise it is a low-priority unit A low .

6. The immersive video transmission method for an AR cinema according to claim 5, wherein: The configuration of the priority queue for data within the unit using the QUIC protocol refers to the configuration of the TLS1.3 protocol. The session parameters in TLS are used to initialize the QUIC connection. According to the hierarchical unit characteristics, combined with interaction changes, compressed data volume, network quality, and device performance, the priority w of the data stream is calculated. priority , and the priority w of the data priority is input into the QUIC implementation library as the basis for priority mapping configuration. The QUIC library performs pairing according to the TLS configuration and real-time network performance, enabling high-interaction units to be bound to high-priority data streams, and low-interaction units to be assigned to low-priority QUIC data streams to fill the remaining bandwidth, optimizing throughput.

7. The immersive video transmission method for an AR cinema according to claim 6, wherein: The efficient transmission of data according to the QUIC priority queue means compressing the high-priority and low-priority data through the Brotli and Gzip algorithms respectively for the QUIC priority queue, encapsulating them into QUIC data packets and marking the priorities. The QUIC protocol dynamically allocates bandwidth based on BBR congestion control and priority scheduling, preferentially transmits the high-priority data stream, and adaptively adjusts according to the network quality and interaction change rate. During the transmission process, the client receives data through QUIC, preferentially decodes the high-priority data stream for real-time rendering of the focus area, combines with the Unity to fuse the stereoscopic-NeRF grid to render the scene. The client receives QUIC data, preferentially decodes the high-priority data stream, maps it to the stereoscopic-NeRF hybrid grid, fuses in Unity to generate an AR scene, and adjusts the rendering resolution according to the device performance through dynamic resolution scaling. At the same time, monitors the latency, packet loss rate, and frame rate through the Prometheus tool, and provides real-time feedback to optimize the priority and scheduling strategy.

8. An immersive image transmission system for an AR cinema, based on the immersive image transmission method for an AR cinema according to any one of claims 1 to 7, characterized in that: including a data loading and point cloud extraction module, configured to load AR scene data from a cloud server and extract a point cloud data set; a stereoscopic unit generation module, configured to generate a stereoscopic unit set, provide a discrete geometric structure of the point cloud, and support dynamic region partitioning; a NeRF radiation field unit generation module, configured to generate an initial grid and a NeRF unit set, and support fine expression of special effect regions; a unit optimization and fusion module, configured to optimize unit parameters, fuse the stereoscopic and NeRF radiation field units, and generate a high-quality hybrid grid; a unit and data stream priority configuration and transmission module, configured to assign priorities to units and data streams, optimize the transmission efficiency, and ensure that high-priority data streams arrive first.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of an immersive image transmission method for an AR cinema according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of an immersive image transmission method for an AR cinema according to any one of claims 1 to 7.

Images