3D Scene Rendering Method, Device and Equipment Based on Curved Screen

Abstract

The present invention relates to a 3D scene rendering method, apparatus and device based on a curved screen. The method: performs frequency domain decomposition on 3D scene data to obtain scene frequency domain feature data; constructs a surface mapping matrix according to the curvature parameter and radian parameter of the curved screen; performs 3D scene space feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a space mapping feature map; inputs the space mapping feature map into a multi-curvature convolution kernel for surface feature processing to obtain a surface enhanced feature map; divides the initial rendering parameter set into a surface sensitive parameter set and a surface insensitive parameter set, and performs rendering parameter optimization to obtain a target rendering parameter set; performs 3D scene rendering through the target rendering parameter set and the surface enhanced feature map to obtain a curved screen display image. The present invention improves the display quality of the rendering result in different regions of the curved screen.

Description

Technical Field

[0001] The present invention relates to the technical field of 3D scene rendering, and particularly to a 3D scene rendering method, apparatus and device based on a curved screen. Background Art

[0002] Due to its unique geometric characteristics and immersive viewing experience, the curved screen is gradually becoming an important development direction of the new generation of display devices. Compared with traditional flat displays, the curved screen can provide users with a wider viewing range and a more natural viewing experience. However, existing 3D scene rendering technologies are mainly designed for flat display devices, and directly applying these rendering methods on the curved screen often leads to problems such as image distortion and poor spatial consistency.

[0003] The main challenge in current 3D scene rendering on curved screens lies in how to accurately handle the geometric transformation problems brought by curved surface display. Traditional rendering methods are prone to problems such as edge distortion and depth distortion on the curved screen, and it is difficult to maintain the spatial continuity of the scene. Especially in the large curvature display scenario, due to the non-linear characteristics of the projection transformation, conventional rendering parameter optimization methods are difficult to simultaneously take into account the geometric accuracy and visual quality of the scene. Summary of the Invention

[0004] The main purpose of the present invention is to provide a 3D scene rendering method, apparatus and device based on a curved screen, so as to improve the display quality of the rendering results in different regions of the curved screen.

[0005] To achieve the above object, the present invention provides a 3D scene rendering method based on a curved screen, including the following steps:

[0006] Perform frequency domain decomposition on the 3D scene data to obtain scene frequency domain feature data;

[0007] Construct a surface mapping matrix according to the curvature parameter and radian parameter of the curved screen;

[0008] Perform 3D scene spatial feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map;

[0009] Input the spatial mapping feature map into a multi-curvature convolution kernel for surface feature processing to obtain a surface enhanced feature map;

[0010] Divide the initial rendering parameter set into a surface-sensitive parameter set and a surface-insensitive parameter set, and perform rendering parameter optimization to obtain a target rendering parameter set;

[0011] Perform 3D scene rendering through the target rendering parameter set and the surface enhanced feature map to obtain a curved screen display image.

[0012] The present invention also provides a 3D scene rendering device based on a curved screen, including:

[0013] A frequency domain decomposition module for performing frequency domain decomposition on 3D scene data to obtain scene frequency domain feature data;

[0014] A construction module for constructing a surface mapping matrix according to the curvature parameter and radian parameter of the curved screen;

[0015] A feature mapping module for performing 3D scene space feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map;

[0016] A feature processing module for inputting the spatial mapping feature map into a multi-curvature convolution kernel for surface feature processing to obtain a surface enhanced feature map;

[0017] A rendering parameter optimization module for dividing an initial rendering parameter set into a surface-sensitive parameter set and a surface-insensitive parameter set, and performing rendering parameter optimization to obtain a target rendering parameter set;

[0018] A 3D scene rendering module for performing 3D scene rendering through the target rendering parameter set and the surface enhanced feature map to obtain a curved screen display image.

[0019] The present invention also provides a computer device, including a memory and a processor, where a computer program is stored in the memory, and when the processor executes the computer program, the steps of the method described in any one of the above are implemented.

[0020] In summary, the technical solution provided by the present invention realizes the effective separation and extraction of high-frequency details and low-frequency structure information through the design of frequency domain decomposition and a bidirectional feature extraction network, improving the detail fidelity and structural integrity of the rendering result; adopts a surface mapping matrix combined with a non-linear correction mechanism to effectively solve the geometric deformation problem in curved screen rendering and ensure the spatial consistency of the rendered scene during curved surface display; introduces a multi-scale feature pyramid structure and a dual attention mechanism to enhance the accuracy of feature extraction, enabling the rendering result to better maintain the local details and global structure of the scene; realizes adaptive feature enhancement for different curvature regions through the design of a multi-curvature convolution kernel and a graph convolution network, improving the display quality of the rendering result in different regions of the curved screen; designs a dual-track optimization strategy based on parameter classification, adopting different optimization methods for surface-sensitive parameters and insensitive parameters respectively, improving the accuracy and efficiency of rendering parameter optimization; adopts a curvature adaptive sampling and feature injection strategy in the final rendering stage, significantly improving the edge sharpness and detail expressiveness of the rendered image and reducing display distortion. Description of the Drawings

[0021] Figure 1 It is a schematic diagram of the steps of a 3D scene rendering method based on an arc screen in an embodiment of the present invention;

[0022] Figure 2 It is a structural block diagram of a 3D scene rendering device based on an arc screen in an embodiment of the present invention;

[0023] Figure 3 It is a structural schematic block diagram of a computer device in an embodiment of the present invention.

[0024] The realization, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. Specific Embodiments

[0025] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0026] Referring to Figure 1 , this embodiment provides a 3D scene rendering method based on an arc screen, including the following steps:

[0027] S1. Perform frequency-domain decomposition on the 3D scene data to obtain scene frequency-domain feature data;

[0028] Among them, the 3D scene data is decomposed by a 5-layer Laplacian pyramid. Through the 5-layer Laplacian pyramid decomposition, the construction of multi-level scene data is realized. The Laplacian pyramid decomposition is an effective image processing method that generates image levels with different resolutions through Gaussian blur and downsampling, so as to extract information at different scales. The input 3D scene data is subjected to Gaussian blur processing to obtain an image with a lower resolution, and then the high-level data of the pyramid is constructed by successive downsampling layer by layer. The data of each layer is subtracted from the Gaussian-blurred version of the previous layer, and the residual information obtained is the Laplacian pyramid data of that layer. This step can effectively separate low-frequency and smooth information while retaining the edge and detail features of the scene. Through 5-layer decomposition, it is ensured that information at different scales is captured. The multi-level scene data is subjected to wavelet transform processing to extract frequency information in different directions. The wavelet transform is a signal processing method that provides both time-domain and frequency-domain localization characteristics. The two-dimensional wavelet transform is used to decompose the scene data of each layer to obtain horizontal component data, vertical component data, and diagonal component data. These three components represent the horizontal details, vertical details, and diagonal details information in the scene respectively. The horizontal component data, vertical component data, and diagonal component data are input into the first feature extraction branch. In the first feature extraction branch, the horizontal component data, vertical component data, and diagonal component data are respectively input into a 3×3 convolutional kernel, and three consecutive feature extraction operations are performed in combination with the ReLU activation function. The selection of the 3×3 convolutional kernel is based on its good local perception ability, enabling it to effectively capture local texture information, while the ReLU activation function enhances the non-linear feature expression ability and prevents the problem of gradient disappearance. Through three consecutive convolutional operations, this branch fully extracts the high-frequency information in the image, such as edges, textures, and rapidly changing regions. At the same time, in the second feature extraction branch, the horizontal component data, vertical component data, and diagonal component data are also used as inputs. This branch uses a 7×7 convolutional kernel and a linear activation function for feature extraction. The 7×7 convolutional kernel can perceive a larger area range compared to the 3×3 convolutional kernel and is more suitable for extracting low-frequency information, that is, the global structure and gradual change information in the scene. The use of the linear activation function ensures the smoothness and consistency of the low-frequency features and does not introduce non-linear transformations during the feature extraction process. This branch can effectively extract the low-frequency information of the scene, ensuring that global features such as lighting and shadows can be correctly retained during the rendering process, thereby avoiding oversharpening or distortion. The high-frequency feature data obtained from the first feature extraction branch and the low-frequency feature data obtained from the second feature extraction branch are combined to construct the scene frequency-domain feature data.

[0029] S2. Construct a surface mapping matrix according to the curvature parameter and radian parameter of the arc screen;

[0030] Specifically, a parametric surface equation of the curved screen is constructed according to the curvature parameter and the radian parameter of the curved screen. Since the shape of the curved screen is approximately represented as a surface with a certain radius, the parametric surface is used to represent its geometric characteristics. Let the curvature parameter of the curved screen be K and the radian parameter be θ. Then, a parametric surface equation of the curved screen in three-dimensional space is defined. For example, the cylindrical coordinate system or the spherical coordinate system is used to describe the position of any point on the screen, so as to establish its mathematical expression. The parametric surface equation of the curved screen is analyzed and transformed to construct a 4×4 basic transformation matrix including translation, rotation, and scaling components. Since the projection of the 3D scene is usually carried out in the Cartesian coordinate system, and the surface of the curved screen is a curved surface, a conversion relationship is established between different coordinate systems. Considering the translation component in the basic transformation, it is used to adjust the center position of the scene on the screen to align it with the geometric center of the curved screen. Then, the rotation component is calculated to ensure that the direction of the scene conforms to the curved surface characteristics of the curved screen, while the scaling component is used to adjust the spatial scale of the scene under different radian conditions to avoid image distortion caused by different screen shapes. Combining these three components, a standard 4×4 transformation matrix is constructed to make the 3D scene initially meet the display requirements of the curved screen. A non-linear mapping function is constructed according to the parametric surface equation of the curved screen, and a surface deformation correction term is introduced into it to obtain a non-linear correction matrix. By analyzing the local curvature changes at different positions of the curved screen, a corresponding non-linear function is established to make the coordinate transformation adapt to the actual shape of the curved screen. For example, correction terms of quadratic or cubic curve fitting are introduced in different screen areas to compensate for distortion and ensure the geometric accuracy of the final rendered image. Through this step, the errors that occur in the curved screen by the traditional linear transformation method are effectively reduced, making the 3D scene more naturally adapt to the curvature characteristics of the screen. The 4×4 basic transformation matrix and the non-linear correction matrix are multiplied to obtain the initial mapping matrix. The mapping parameter correction amount is calculated according to the frequency domain feature data of the scene to construct a parameter adjustment matrix. Based on the frequency domain feature data analysis of the scene, the projection errors of different frequency components on the curved screen are analyzed. For example, the error distribution in the local space is calculated through Fourier transform or wavelet transform, and a correction amount is constructed using this error information, so that the mapping matrix can be dynamically adjusted to adapt to different scene characteristics. The initial mapping matrix and the parameter adjustment matrix are combined and calculated to obtain the final surface mapping matrix.

[0031] S3. Based on the frequency domain feature data of the scene and the surface mapping matrix, perform 3D scene spatial feature mapping to obtain a spatial mapping feature map;

[0032] It should be noted that for the deep feature extraction of the scene frequency domain feature data, the attention mechanism unit is used to process the scene frequency domain feature data, making the feature extraction process more focused on the key areas. In the attention mechanism unit, a 3×3 deformable convolution is used to adaptively extract spatial features. Since the deformable convolution can dynamically adjust the receptive field and is more adaptable to the complex geometric shape of the curved screen, compared with the traditional fixed convolution operation, it can more effectively capture the feature changes caused by the surface deformation. At the same time, in the attention mechanism unit, a 1×1 convolution is used to extract channel attention features, enabling the information of different channels to be adaptively weighted and enhancing the feature expression ability. Through the combination of these two methods, a dual attention feature map is obtained, which more accurately represents the spatial features of the scene and adapts to the special shape of the curved screen. The dual attention feature map is processed by skip connection to fuse features at different levels to establish the connection between global information and local information. Skip connection can enhance the semantic information of low-level features while retaining the detailed information of high-level features, making the feature expression more complete. After completing the multi-level feature association, considering the depth information of the scene, the contributions of features at different depths to the final rendering effect are uneven. Based on the scene depth information, the depth importance coefficient is calculated to obtain the depth weight coefficient. The calculation of the depth weight coefficient is achieved by analyzing the gradient changes in different depth regions of the scene or by learning through a depth perception network to adaptively allocate the feature weights of different regions, ensuring that the spatial structure of the scene can be more realistically restored during the final rendering. The multi-level associated features are weighted and calculated according to the depth weight coefficient, making the feature expression more consistent with the depth distribution of the scene, and obtaining a weighted feature map. A weighted sum operation or a dot product operation is used to make the depth weight directly act on the feature distribution to ensure more accurate feature mapping. In order to correctly map the features to the curved surface coordinate space of the curved screen, a surface mapping matrix is used for transformation. However, since this matrix contains both geometric transformation information and non-linear correction information, it is decomposed into two parts to make the transformation process more refined. The geometric transformation part of the mapping is performed on the weighted feature map. This step mainly involves linear transformation operations such as coordinate conversion, rotation, and scaling, so as to project the features into the initial mapping space of the curved screen. The non-linear correction part is used to adjust the mapping result to compensate for the non-linear deformation caused by the screen curvature, making the final mapped feature data more accurately match the geometric shape of the screen. A curvature-based adaptive bilinear interpolation operation is performed on the mapped feature data. A curvature adaptive mechanism is introduced to adjust the weight distribution during the interpolation process, making the interpolation calculation adapt to the morphological changes in different curvature regions. For example, in areas with a large screen curvature, the non-uniformity of the interpolation weight is appropriately increased to reduce geometric distortion, while in areas with a small curvature, the standard bilinear interpolation method is used to ensure computational efficiency. Through this step, the final spatial mapped feature map is obtained.

[0033] S4. Input the spatial mapping feature map into a multi-curvature convolution kernel for surface feature processing to obtain a surface-enhanced feature map;

[0034] Specifically, construct a multi-curvature convolution kernel that adapts to different curvature regions according to the geometric characteristics of the curved screen, and set the curvature values to be , and , ensuring that the most suitable spatial features can be extracted within different curvature ranges. Due to the changes in light projection, perspective distortion, and texture mapping caused by different curvature regions, each group of curvature convolution kernels contains two convolutional layers and one batch normalization layer to ensure the stability of feature extraction. The first two convolutional layers are respectively used to learn local spatial features and adapt to the influence of different curvatures, while the batch normalization layer is used to normalize the feature distribution, prevent gradient vanishing or explosion, and at the same time improve the convergence speed of training, making the final surface features smoother and more generalizable. After constructing the multi-curvature convolution kernel, input the spatial mapping feature map into these multi-curvature convolution kernels respectively for feature calculation to obtain three groups of feature data with different curvature perspectives. The working ranges of each convolution kernel are different, where is mainly used for feature extraction in the low-curvature region and is suitable for the part close to the plane, while is suitable for the medium-curvature region and can capture the bending characteristics of the screen to a certain extent, while It is mainly used for high-curvature regions, enabling feature representation to better adapt to the screen edge or regions with a large degree of curvature. In this way, feature calculations are performed within different curvature ranges to ensure that features in different curvature regions can be extracted and represented in an optimal manner during final rendering, avoiding the distortion and blurring problems that occur when traditional convolutions process curved screens. After obtaining the feature data of three different curvature perspectives, the spatial structure of the curved screen is divided to more precisely match the feature representation. The entire curved screen is divided into n equal arc segments, and independent feature enhancement parameters are assigned to each arc segment to form an arc segment parameter set. Different parameters are set in different curvature regions, enabling the feature enhancement process to adaptively adjust the feature representation to reduce the error caused by the global parameterization method. To establish the relationship between different arc segments, the n arc segments are constructed into a graph structure, where each arc segment is regarded as a graph node and the relationship between adjacent arc segments is regarded as a graph edge, resulting in an arc segment relationship graph. Graph convolution operations are performed on the feature data of three different curvature perspectives and the arc segment relationship graph to establish the spatial dependence relationship between features. Since traditional CNN methods have limitations in processing non-Euclidean space data and the data distribution on the curved screen is non-uniform, graph convolution can better learn the feature interaction between different arc segments, enabling the sharing of feature information in adjacent regions and strengthening the global feature representation. The graph convolution operation aggregates the features of different regions through the adjacency matrix and the normalized Laplacian matrix, enabling the final spatial dependence features to accurately reflect the mutual influence between different arc segments, improving the spatial consistency of rendering, and obtaining the spatial dependence features. Feature enhancement operations are performed based on the spatial dependence features and the arc segment parameter set to obtain the surface enhancement feature map. The feature enhancement process is based on the feature parameters of different arc segments and performs weighted summation of the feature data by adjusting the weights, enabling the features of each region to better conform to its curvature characteristics.

[0035] S5. Divide the initial rendering parameter set into a surface-sensitive parameter set and a surface-insensitive parameter set, and perform rendering parameter optimization to obtain the target rendering parameter set;

[0036] Among them, the initial rendering parameters are classified to distinguish the parameters that are greatly affected by the curved screen and the parameters that are relatively independent of the surface shape. The set of initial rendering parameters is divided into a surface-sensitive parameter set and a surface-insensitive parameter set. Among them, the surface-sensitive parameter set includes geometric transformation parameters and perspective projection parameters, which determine the projection method of the 3D scene on the curved screen and are directly affected by the screen curvature. The surface-insensitive parameter set mainly contains lighting parameters and material parameters. Although these parameters affect the visual style and rendering effect of the scene, they do not change directly due to the geometric shape of the screen. After the parameter classification is completed, optimization functions are constructed for these two types of parameters respectively to ensure that different types of parameters can be optimally adjusted. For the surface-sensitive parameter set, its optimization function takes into account both surface distortion and feature preservation, so it includes two key terms. One is the surface distortion metric term, which is used to measure the projection error of the 3D scene on the curved screen. For example, it calculates the deviation of the original geometric information after surface projection to ensure that the optimized parameters can minimize distortion to the greatest extent. The other is the feature preservation term, which is used to ensure that the important geometric features of the scene will not be lost due to optimization to avoid structural distortion of the 3D model on the curved screen. Through the combined action of these two parts, it is ensured that the optimized parameters can reduce distortion and maintain the integrity of spatial features. For the surface-insensitive parameter set, the mean square error is used as the optimization objective to ensure that the optimized lighting and material parameters can be closest to the best visual effect to the greatest extent and maintain stability in different rendering environments. After constructing the optimization objective, the optimization calculation is then performed, and the gradient descent operation is carried out on the surface-sensitive parameter set. Since the curvature of the curved screen varies in different regions, during the optimization process, the optimization step size is dynamically adjusted according to the screen curvature, reducing the step size in the high-curvature region to ensure that the parameter changes will not be too drastic, while appropriately increasing the step size in the low-curvature region to speed up the optimization. This effectively prevents over-optimization in areas with drastic curvature changes, thereby improving the stability of the final rendering result. After multiple rounds of iterative optimization, the surface-sensitive parameter set can gradually converge, making the projection of the 3D scene more accurate and reducing the visual distortion caused by the screen curvature. At the same time, for the surface-insensitive parameter set, an adaptive optimization method is used for optimization to ensure that the adjustment of lighting and material parameters can be more efficient and accurate. During the optimization process, different lighting conditions and material reflection characteristics are dynamically adjusted so that the optimized parameters can maintain good visual performance in different scenes, while avoiding lighting distortion or loss of material details caused by optimization. Through this step, it is ensured that the final rendering effect is more stable, so that regardless of how the curvature of the screen changes, the lighting and materials can still maintain a natural and consistent performance. After the surface-sensitive parameter set and the surface-insensitive parameter set are respectively optimized, a combined operation is performed on the two to form the final target rendering parameter set.During the merging process, ensure the matching of geometric transformation parameters with lighting and material parameters to guarantee that there are no inconsistent phenomena during the rendering process. For example, after adjusting the optimized geometric parameters, the projection method that affects lighting is affected. Therefore, corresponding compensation is carried out during the merging process to ensure that the final rendering effect can maintain the best visual performance on the curved screen.

[0037] S6. Perform 3D scene rendering through the target rendering parameter set and the surface enhancement feature map to obtain the curved screen display image.

[0038] Specifically, vertex transformation operations are performed on the vertex data of the 3D scene using the geometric transformation parameters in the target rendering parameter set. This process involves operations such as coordinate transformation, rotation, and scaling, adjusting the vertex positions of the original 3D scene to fit the spatial coordinate system of the curved screen. Since the curved screen has a specific curvature distribution, when performing vertex transformation, the influence of the curvature of different regions on the vertex positions is considered to ensure that the transformed vertex coordinates can be accurately mapped to the display area of the curved screen, maintaining the consistency of the spatial structure of the entire scene. After completing the vertex transformation, a projection matrix transformation is performed on the transformed vertex coordinates to ensure that the 3D scene can be correctly projected onto the curved screen. In this process, using the perspective projection parameters in the target rendering parameter set, the transformed vertex coordinates are converted to the screen coordinate system, enabling the 3D model to be correctly projected onto the surface of the curved screen. Since the traditional perspective projection method is applicable to flat screens and the projection relationship of the curved screen is more complex, a surface projection correction method is introduced during the projection transformation process to compensate for the perspective distortion caused by the screen curvature, so that the final projected vertex data can accurately match the geometric characteristics of the curved screen. After obtaining the projected vertex data, triangular mesh partitioning is performed on it, and curvature adaptive sampling is carried out on this basis to ensure that optimization processing can be performed for different curvature regions during the rendering process. The purpose of triangular mesh partitioning is to discretize the entire 3D scene, enabling the rendering calculation to be processed in units of polygon fragments. During the partitioning process, the density of the triangular meshes is dynamically adjusted according to the curvature characteristics of the screen to ensure that finer geometric details can be obtained in high-curvature regions, while maintaining a lower sampling density in low-curvature regions to improve computational efficiency. During the curvature adaptive sampling process, the sampling density is proportional to the curvature value, that is, the number of sampling points is increased in regions with larger curvature to enhance the detail expression ability, while the number of sampling points is reduced in regions with smaller curvature to avoid waste of computational resources, obtaining optimized sampling point data. After obtaining the sampling point data, based on the lighting parameters and material parameters in the target rendering parameter set, lighting shading calculations are performed on the sampling point data to generate initial pixel data. During the lighting calculation process, factors such as the light source position, reflection characteristics of the material, and normal information are combined to calculate the lighting intensity and color distribution of each sampling point. Since the lighting effect of the curved screen is affected by the surface characteristics, during the calculation process, the influence of the screen curvature on the lighting is considered. For example, a light focusing effect appears in high-curvature regions, resulting in an increase in local brightness, while the light distribution is relatively uniform in low-curvature regions. Therefore, corresponding lighting compensation is required to ensure that the final lighting effect can be balanced across the entire screen range. After obtaining the initial pixel data, combined with the surface enhancement feature map, feature injection operations are performed to enhance the expressiveness of the pixel data.The surface enhancement feature map contains surface feature information extracted during the previous calculation process, such as curvature features, spatial dependence features, texture enhancement information, etc. Therefore, during the feature injection process, these features are fused with the initial pixel data to enhance the spatial consistency of the final rendering result. For example, the surface enhancement features are used to optimize the lighting calculation, making the shadows and highlights in high-curvature areas appear more natural, or the texture enhancement information in the feature map is used to make the material details of the scene more clearly presented, improving the quality of the final rendering result. Curvature-based anti-aliasing processing is performed on the enhanced pixel data to reduce the edge aliasing effect caused by pixelation. A curvature-based adaptive anti-aliasing method is adopted to adjust the interpolation calculation method according to the local curvature, thereby reducing the edge aliasing in high-curvature areas and ensuring that the final rendering result can maintain a smooth and natural visual effect on the entire curved screen. After calculation, a display image suitable for the curved screen is generated, enabling the 3D scene to be presented on the curved screen in high quality and maintaining the best visual performance under different curvature conditions.

[0039] In one example, the 3D scene data is decomposed in the frequency domain to obtain the scene frequency domain feature data, including:

[0040] The 3D scene data is decomposed into a 5-layer Laplacian pyramid to obtain multi-level scene data;

[0041] The multi-level scene data is processed by wavelet transform to obtain the horizontal component data, vertical component data, and diagonal component data of each level;

[0042] The horizontal component data, vertical component data, and diagonal component data are input into the first feature extraction branch, and three consecutive feature extraction operations are performed through the 3×3 convolution kernel and ReLU activation function of the first feature extraction branch to obtain high-frequency feature data;

[0043] The horizontal component data, vertical component data, and diagonal component data are input into the second feature extraction branch, and feature extraction operations are performed through the 7×7 convolution kernel and linear activation function of the second feature extraction branch to obtain low-frequency feature data;

[0044] The high-frequency feature data and low-frequency feature data are used as the scene frequency domain feature data.

[0045] In this example, the 3D scene data is decomposed into a 5-layer Laplacian pyramid. The Laplacian pyramid decomposition gradually decomposes the detailed information of an image or data into multiple scales through layer-by-layer filtering and downsampling, obtaining the low-frequency and high-frequency information of each layer. Each layer of the Laplacian pyramid contains scene data at different scales, where the low-frequency part contains the overall contour information of the scene, while the high-frequency part contains more detailed textures and details. Each layer of the Laplacian pyramid decomposition is represented as:

[0046] ;

[0047] Among them, represents the Laplacian pyramid data of the th layer, represents the Gaussian blurred image of the th layer, represents the next-level pyramid layer, usually obtained by Gaussian blurring and downsampling. In this way, the Laplacian pyramid effectively extracts the feature information of the scene at different scales. For the Laplacian pyramid data of each layer, wavelet transform processing is performed to extract frequency domain features. Wavelet transform effectively analyzes the local features of the signal at multiple scales and decomposes the signal into multiple frequency sub-bands. Two-dimensional wavelet transform is performed on the Laplacian pyramid data of each layer to obtain horizontal component data, vertical component data, and diagonal component data. The two-dimensional wavelet transform is expressed as:

[0048] ;

[0049] Among them, represents the input scene data, and are the filter coefficients of the wavelet transform, is the position coordinate. The output after wavelet transform contains multiple components. Among them, the horizontal component data captures the detailed information in the horizontal direction, the vertical component data captures the detailed information in the vertical direction, and the diagonal component data captures the diagonal features in the image. These frequency domain components effectively describe the texture information of the scene in different directions and at different scales. The horizontal, vertical, and diagonal component data of each layer are input into two different feature extraction branches for feature processing. The first feature extraction branch uses a 3×3 convolutional kernel and combines the ReLU activation function to perform three consecutive feature extraction operations to extract the high-frequency features in the scene data, representing the detailed and texture information. In the convolutional operation, each convolution extracts specific local features and introduces non-linearity through the ReLU activation function to enhance the expression ability of the network. The specific convolutional operation is expressed as:

[0050] ;

[0051] Among them, represents the convolution result, is the convolutional kernel, is the input data, is the bias term, Denote the convolution operation, and ReLU denotes the activation function. Through three consecutive convolution operations, high-frequency features in the input data are gradually extracted. The second feature extraction branch uses a 7×7 convolution kernel and a linear activation function for feature extraction. The main task of this branch is to extract low-frequency features, which represent the overall structure and large-scale information of the scene. Different from the first branch, the linear activation function preserves the original numerical values of the feature data, enabling the low-frequency information to be extracted in its original form. The convolution operation in this process is similar to the aforementioned one, but the activation function is different, and the formula is:

[0052] ;

[0053] Denote the convolution result, is the convolution kernel, is the input data, is the bias term, but the ReLU activation function is not introduced because the system focuses on the linear expression of low-frequency features. Through the processing of the first branch and the second branch, high-frequency feature data and low-frequency feature data are obtained respectively. The high-frequency feature data captures detailed information, while the low-frequency feature data retains the overall structure of the scene. Combining these two types of feature data, the complete scene frequency domain feature data is obtained.

[0054] In this embodiment, the scene frequency domain feature data is used to construct the surface mapping matrix after feature alignment processing. The feature alignment processing includes: inputting the scene frequency domain feature data into parallel convolutional neural network branches and Transformer branches, extracting local fine-grained features through the convolutional neural network branches, and extracting long-range context features through the Transformer branches to obtain dual-branch feature data; constructing a multi-scale feature pyramid for the dual-branch feature data, performing feature extraction at different scale levels respectively to obtain a multi-scale feature sequence; inputting the multi-scale feature sequence into a feature interaction module, performing cross-branch attention calculation on the features between different branches to obtain interaction attention weights; weighted fusing the dual-branch features according to the interaction attention weights and introducing a residual connection to obtain primary semantic alignment features; performing self-attention calculation on the primary semantic alignment features to establish long-range dependencies within the features to obtain a self-attention feature map; inputting the self-attention feature map into a multi-layer perceptron for non-linear transformation to obtain high-level semantic alignment features; performing a residual self-interaction operation on the high-level semantic alignment features to fuse global information and local information to obtain the final semantic alignment features; using the final semantic alignment features as the optimized scene frequency domain feature data.

[0055] In an example, constructing the surface mapping matrix according to the curvature parameter and radian parameter of the arc screen includes:

[0056] Construct a parametric surface equation of the arc screen according to the curvature parameter and radian parameter of the arc screen;

[0057] Perform transformation analysis on the parametric surface equation of the arc screen to obtain a 4×4 basic transformation matrix including translation components, rotation components, and scaling components;

[0058] Construct a non-linear mapping function according to the parametric surface equation of the arc screen, introduce the surface deformation correction term into the non-linear mapping function, and obtain the non-linear correction matrix;

[0059] Perform matrix multiplication on the basic transformation matrix and the non-linear correction matrix to obtain the initial mapping matrix;

[0060] Calculate the mapping parameter correction amount according to the scene frequency domain feature data, obtain the parameter adjustment matrix, and perform a combined operation on the initial mapping matrix and the parameter adjustment matrix to obtain the surface mapping matrix.

[0061] In this example, a parametric surface equation of the arc screen is constructed according to the curvature parameter and radian parameter of the arc screen. The curvature parameter of the arc screen represents the degree of curvature of the surface of the arc screen, while the radian parameter defines the bending angle of the arc screen. In a parametric way, it is expressed as a function in a two-dimensional coordinate system, and the polar coordinate system is used to represent the surface characteristics therein. Assuming that the center of the arc screen is located at the origin, the parametric surface equation is written as:

[0062] ;

[0063] ;

[0064] ;

[0065] Among them, is the radius of the arc screen, and are the angle parameters respectively, represents the angle change in the horizontal direction, represents the angle change in the vertical direction. In this way, the spatial position of each point on the surface of the arc screen is described. Perform transformation analysis on the parametric surface equation of the arc screen to obtain a 4 transformation matrix. The installation position and angle of the arc screen will be offset, so a spatial transformation is performed on the surface equation. The transformation process includes translation, rotation, and scaling operations. These transformation operations are uniformly represented as a 4x4 transformation matrix. Assuming that the translation component is , the rotation component is (the rotation angle around a certain axis), and the scaling component is , then the transformation matrix is expressed as:

[0066] ;

[0067] Among them, are scaling factors respectively, is the translation amount. By performing matrix transformation on the surface equation with this matrix, the transformed surface coordinates are obtained. According to the parametric surface equation of the curved screen, a non-linear mapping function is constructed, and the surface deformation correction term is introduced into the non-linear mapping function to obtain the non-linear correction matrix. When displaying on the curved screen, due to the curved nature of the display device, the rendered scene will undergo geometric distortion, especially in the edge area of the screen. To correct these deformations, a non-linear mapping function is introduced, which makes necessary deformation adjustments to the geometric shape of the scene. The non-linear correction matrix is realized by introducing the surface deformation correction term, and the correction term is dynamically adjusted according to the curvature and bending degree. The non-linear mapping matrix is expressed as:

[0068] ;

[0069] Among them, is the curvature correction function, represents the curvature value. This function adjusts the corresponding deformation degree according to the local curvature of the scene, and usually enhances the deformation as the curvature increases, so as to correct the distortion in the edge area. Perform matrix multiplication on the basic transformation matrix and the non-linear correction matrix to obtain the initial mapping matrix. By multiplying the basic transformation matrix and the non-linear correction matrix, an initial mapping matrix containing all transformations (translation, rotation, scaling, correction) is obtained:

[0070] ;

[0071] The initial mapping matrix contains the geometric transformation and surface correction information of the curved screen, and is the basic matrix for mapping scene data in the entire rendering process. Calculate the mapping parameter correction amount according to the frequency domain characteristic data of the scene to obtain the parameter adjustment matrix. Since the frequency domain characteristics of the scene (such as texture, lighting, etc.) are different in different curved areas, additional corrections are made according to the specific frequency domain characteristics of the scene. This process obtains the correction amount of the mapping parameters by analyzing the frequency domain characteristic data of the scene (such as the characteristics obtained through Laplacian pyramid decomposition or wavelet transform). Assume that the mapping parameter correction amount is , then the parameter adjustment matrix is expressed as:

[0072] ;

[0073] Among them, is the adjusted scaling factor, is the adjusted translation amount. These parameters are dynamically calculated according to the specific frequency domain characteristics of the scene. Combine and operate the initial mapping matrix and the parameter adjustment matrix to obtain the final surface mapping matrix:

[0074] ;

[0075] This mapping matrix is used to map all geometries in the 3D scene to the display area of the curved screen, ensuring that the rendering result can accurately reflect the curvature characteristics of the curved screen and the spatial transformation of the scene. Through the mapping process, the 3D scene displayed on the curved screen obtains the correct geometric mapping, thus avoiding distortion caused by the screen curvature.

[0076] In this embodiment, after obtaining the surface mapping matrix and before performing the spatial feature mapping on the 3D scene, it further includes: constructing a non - linear affine model for the surface mapping matrix, decomposing the surface mapping matrix into a linear mapping part and a non - linear perturbation part to obtain an initial affine model; designing a non - linear perturbation observer based on the initial affine model to perform state estimation on the non - linear perturbation part to obtain a perturbation state estimation value; constructing an integral sliding - mode dynamic switching surface according to the perturbation state estimation value, establishing a perturbation compensation function through the dynamic change of the switching surface to obtain compensation parameters; performing homogeneous second - order quasi - continuous optimization on the surface mapping matrix based on the compensation parameters to eliminate the influence of non - linear perturbation and obtain an optimized surface mapping matrix; performing Lyapunov stability analysis on the optimized surface mapping matrix, constructing stability constraint conditions to obtain stability parameters; adaptively adjusting the curvature parameters of the optimized surface mapping matrix according to the stability parameters to obtain adjusted curvature parameters; updating the non - linear mapping function of the optimized surface mapping matrix using the adjusted curvature parameters to obtain a compensated surface mapping matrix; performing curvature self - adaptation verification on the compensated surface mapping matrix to ensure that the mapping accuracy meets the preset threshold requirements, obtaining the final surface mapping matrix, and using the final surface mapping matrix as the input for subsequent spatial feature mapping.

[0077] In one example, performing 3D scene spatial feature mapping based on the scene frequency - domain feature data and the surface mapping matrix to obtain a spatial mapping feature map, including:

[0078] Inputting the scene frequency - domain feature data into the attention mechanism unit, and respectively extracting spatial attention features and channel attention features through the 3×3 deformable convolution and 1×1 convolution in the attention mechanism unit to obtain a dual - attention feature map;

[0079] Performing skip - connection processing on the dual - attention feature map, establishing the association between features at different levels to obtain multi - level associated features, and calculating the depth importance of the multi - level associated features based on the scene depth information to obtain depth weight coefficients;

[0080] Performing weighted calculation on the multi - level associated features according to the depth weight coefficients to obtain a weighted feature map;

[0081] Decompose the surface mapping matrix into a geometric transformation part and a non-linear correction part, and perform step-by-step mapping on the weighted feature map to obtain the mapped feature data;

[0082] Perform curvature-based adaptive bilinear interpolation operation on the mapped feature data to obtain the spatial mapping feature map.

[0083] In this example, the scene frequency domain feature data is input into the attention mechanism unit. The scene frequency domain feature data is obtained by performing frequency domain decomposition on the 3D scene, which contains different frequency components of the scene and reflects different spatial information. The attention mechanism automatically adjusts its weights dynamically according to the characteristics of the input data, so as to strengthen the expression of important features and suppress irrelevant features. Two convolutional operations are used inside the attention mechanism unit: 3×3 deformable convolution and 1×1 convolution. The 3×3 deformable convolution is used to extract spatial attention features. Through the convolutional operation, the network adaptively adjusts the shape of the convolutional kernel at different positions and spatial scales to better capture the spatial information of the scene. The formula is as follows:

[0084] ;

[0085] where, represents the extracted spatial attention features, represents the 3×3 deformable convolution operation, is the input frequency domain feature data. The 1×1 convolution is used to extract channel attention features. Different from the spatial attention features, the channel attention features focus on weighting between different feature channels and dynamically adjust according to the importance of different channels to the rendering result. The formula is as follows:

[0086] ;

[0087] where, represents the extracted channel attention features, is the convolution operation. Through these two convolutional operations, the dual attention feature maps of space and channel are obtained. Perform skip connection processing on the dual attention feature maps. Skip connection is a technique in deep learning that can effectively connect feature maps at different levels, enabling the network to utilize the correlation information of low-level and high-level features. Connect the spatial attention feature map and the channel attention feature map through skip connection to obtain multi-level associated features. The skip connection processing is achieved through simple addition or splicing operations. The formula is as follows:

[0088] ;

[0089] Through this operation, low-level and high-level features are effectively fused, enhancing the feature expression ability. The multi-level associated features are weighted based on the depth information of the scene. The depth information of the scene provides weights for different depth positions, helping to determine the importance of features at different levels. The depth information is normalized by the depth value of each pixel to obtain the depth weight coefficient. Let the depth value be , and its normalized depth weight coefficient is expressed as:

[0090] ;

[0091] Among them, represents the depth weight coefficient, is the depth value of the th pixel, is the maximum depth value in the scene. Through this step, different weights are assigned to each pixel according to its depth, realizing the calculation of the importance of depth. Based on the depth weight coefficient, the multi-level associated features are weighted and calculated to obtain the weighted feature map. The weighted calculation is realized by applying the depth weight coefficient to the feature value of each pixel. The formula is as follows:

[0092] ;

[0093] Among them, is the weighted feature map, is the multi-level feature map after skip connection, is the depth weight coefficient. Based on the geometric transformation and non-linear correction part of the surface mapping matrix, the weighted feature map is mapped step by step. The geometric transformation part of the surface mapping matrix includes scaling, rotation and translation operations, while the non-linear correction part finely adjusts the features by introducing a curvature correction term. The mapping operation is realized by matrix multiplication or other transformation techniques to map the weighted feature map from the 3D space to the display space of the curved screen. The mapping formula is expressed as:

[0094] ;

[0095] Among them, is the mapped feature data, is the geometric transformation matrix. For the mapped feature data, an adaptive bilinear interpolation operation based on curvature is performed to optimize the mapping result. The curvature value is obtained by calculating the curvature of each point on the surface, reflecting the degree of curvature of the surface at that point. To cope with the curvature change of the curved screen, an adaptive bilinear interpolation method is used to dynamically adjust the interpolation strategy according to the curvature values at different positions. The bilinear interpolation formula is expressed as:

[0096] ;

[0097] Among them, is the interpolated eigenvalue, and are the coordinates of two pixel points used for interpolation, is the interpolation weight, which depends on the curvature value. Through the above steps, the finally obtained spatial mapping feature map can accurately reflect the frequency domain features of the scene and is optimized through geometric and non-linear corrections of the curved screen to ensure the realism and visual effects of the scene.

[0098] Among them, before the spatial mapping feature map is input into the multi-curvature convolution kernel, it also includes feature error correction for the spatial mapping feature map, specifically including: uniformly dividing the spatial mapping feature map into grids, setting feature sampling points within each grid, constructing a feature sampling network to obtain a feature sampling point distribution map; extracting feature data from multiple perspectives of the spatial mapping feature map to obtain multiple groups of perspective feature data; calculating the feature mapping error for each sampling point in the feature sampling point distribution map to obtain a feature error distribution map; allocating weights to the multiple groups of perspective feature data according to the feature error distribution map to obtain weighted perspective feature data; constructing a feature error correction model based on the weighted perspective feature data to obtain feature correction parameters; using the feature correction parameters to perform iterative optimization operations on the feature error distribution map to obtain optimized feature error data; compensating the spatial mapping feature map according to the optimized feature error data to obtain an error-corrected feature map; evaluating the feature consistency of the error-corrected feature map to generate a corrected spatial mapping feature map.

[0099] In one example, inputting the spatial mapping feature map into a multi-curvature convolution kernel for curved surface feature processing to obtain a curved surface enhanced feature map, including:

[0100] Constructing a multi-curvature convolution kernel with curvature values of k1, k2, and k3 according to the geometric characteristics of the curved screen. Each group of curvature convolution kernels in the multi-curvature convolution kernel contains two convolutional layers and one batch normalization layer;

[0101] Inputting the spatial mapping feature map into the multi-curvature convolution kernel respectively for feature calculation to obtain three groups of feature data with different curvature perspectives;

[0102] Dividing the curved screen into n equal arc segments and assigning independent feature enhancement parameters to each arc segment to obtain an arc segment parameter set;

[0103] Constructing the n arc segments into a graph structure, where each arc segment is a graph node and the relationship between adjacent arc segments is a graph edge to obtain an arc segment relationship graph;

[0104] Performing graph convolution operations on the three groups of feature data with different curvature perspectives and the arc segment relationship graph to establish the spatial dependence relationship between features to obtain spatial dependence features;

[0105] Perform feature enhancement operations based on spatial dependence characteristics and arc segment parameter sets to obtain a surface enhancement feature map.

[0106] In this example, multi-curvature convolution kernels with curvature values of , and are constructed according to the geometric characteristics of the curved screen. The curvature of the curved screen is a key factor affecting the display effect and determines the degree of curvature of each area on the screen. To adapt to areas with different curvatures, three convolution kernels with different curvature values are designed, corresponding to different areas on the screen respectively. For example, represents the curvature of the central area of the screen, represents the curvature of the area close to the edge, represents the extreme curvature area (such as both ends of the screen). Each convolution kernel consists of two convolutional layers and a batch normalization layer, so as to better extract local features and improve the training stability of the model. Each convolutional layer of the convolution kernel uses different kernel sizes and strides to capture spatial information at different scales through local feature extraction. The batch normalization layer is used to normalize the output and reduce the overfitting phenomenon during training.

[0107] Conv2D(Conv2D with BatchNorm;

[0108] Among them, is the feature obtained through the multi-curvature convolution kernel, Conv2D represents the convolution operation, is the input feature map. The convolution kernels corresponding to each curvature value perform convolution operations on the spatial mapping feature map respectively to obtain three sets of feature data from different curvature perspectives, that is, , corresponding to the features of different curvature regions respectively. The curved screen is equally divided into arc segments, and the screen is divided into multiple regions. Each region corresponds to an independent feature enhancement parameter to ensure that the display effects of different regions can be independently optimized according to their curvature characteristics. The parameter set of each arc segment contains all the parameters required for feature enhancement, such as the brightness, contrast, color enhancement, etc. of a specific region. Through this division, the rendering effect can be adjusted more flexibly, so that the details of different regions can be better presented.

[0109] ;

[0110] Among them, represents the th arc segment, is the total number of divided arc segments. Each arc segment corresponds to a set of feature enhancement parameter sets , that is, , these parameters are optimized independently for each arc segment. After the arc segment division is completed, arc segments are constructed into a graph structure, where each arc segment serves as a graph node, and graph edges are established between adjacent arc segments, thus obtaining an arc segment relationship graph, where each node represents an arc segment and the edge represents the relationship between adjacent arc segments. This graph structure enables the system to capture the spatial dependence relationships between different arc segments through a graph convolutional network (GCN) and process the features of each arc segment from different curvature perspectives.

[0111] ;

[0112] Among them, is the arc segment relationship graph, is the set of nodes in the graph, representing each arc segment, is the set of edges in the graph, representing the relationship between adjacent arc segments. Through the graph convolutional network, the spatial dependence features of each arc segment are calculated to establish the relationships between various arc segments. For the feature data from three groups of different curvature perspectives and the arc segment relationship graph

[0113] ;

[0114] Among them, is the output feature after graph convolution, is the set of neighbor nodes of the th node, is the adjacency matrix of the graph, representing the connection strength between node and node , is the convolution kernel weight, is the activation function. Through graph convolution operations, the spatial relationships between different arc segments are captured and these relationships are incorporated into the features of each arc segment to obtain the spatial dependence features , which contain the interaction information between different arc segments in space. According to the spatial dependence features and the arc segment parameter set , a feature enhancement operation is performed to obtain a surface enhancement feature map. Through feature enhancement, the visual effects of each arc segment are adjusted to achieve more refined rendering. Feature enhancement is carried out through weighted calculation, non-linear transformation, etc., and the finally obtained surface enhancement feature map contains the final features after being processed by spatial dependence relationships and feature enhancement:

[0115] ;

[0116] Among them, is the surface enhancement feature map, is the arc segment parameter set. Through this step, the feature map optimized according to the curvature characteristics and spatial dependence relationship of the arc-shaped screen is obtained.

[0117] In an example, the initial rendering parameter set is divided into a surface-sensitive parameter set and a surface-insensitive parameter set, and rendering parameter optimization is performed to obtain the target rendering parameter set, including:

[0118] The initial rendering parameter set is divided into a surface-sensitive parameter set and a surface-insensitive parameter set according to the parameter type. The surface-sensitive parameter set includes geometric transformation parameters and perspective projection parameters, and the surface-insensitive parameter set includes lighting parameters and material parameters;

[0119] A surface-sensitive optimization function including a surface distortion metric term and a feature preservation term is constructed for the surface-sensitive parameter set. At the same time, an optimization objective function based on the mean square error is constructed for the surface-insensitive parameter set to obtain the surface-insensitive optimization function;

[0120] Perform gradient descent operation on the surface-sensitive parameter set according to the surface-sensitive optimization function, and dynamically adjust the update step size through the curvature value to obtain the optimized surface-sensitive parameter set;

[0121] Perform Adam optimizer operation on the surface-insensitive parameter set according to the surface-insensitive optimization function to obtain the optimized surface-insensitive parameter set;

[0122] Perform a merging operation on the optimized surface-sensitive parameter set and the optimized surface-insensitive parameter set to obtain the target rendering parameter set.

[0123] In this example, the initial set of rendering parameters is divided by parameter type to classify the rendering parameters into a set of surface-sensitive parameters and a set of surface-insensitive parameters. The set of surface-sensitive parameters includes geometric transformation parameters and perspective projection parameters related to the screen curvature, and these parameters have a strong impact on the rendering effect of the surface. Geometric transformation parameters include rotation matrices, translation matrices, and scaling matrices, etc., which are used to describe the transformation of the scene or object on the surface. Perspective projection parameters include viewing distance, viewing angle, etc., which directly affect the spatial perspective effect of the scene. The set of surface-insensitive parameters includes lighting parameters and material parameters. Lighting parameters are used to control the position, brightness, and type of light sources, while material parameters affect the reflection, refraction, texture, and other characteristics of the object surface. Through this division, it is identified which parameters are sensitive to surface changes and which parameters have less impact on surface changes. After completing the parameter type division, optimization functions are constructed for the set of surface-sensitive parameters and the set of surface-insensitive parameters respectively. For the set of surface-sensitive parameters, an optimization function containing a surface distortion metric term and a feature preservation term is constructed. During the optimization process, the surface distortion metric term is used to measure the geometric distortion caused by surface transformation, while the feature preservation term ensures the preservation of the features of the original scene or object during the rendering process. For example, the surface distortion metric term is expressed by the following formula:

[0124] ;

[0125] where, is the surface distortion metric term, is the transformation matrix of the th point, is the ideal transformation matrix. By minimizing this metric term, the geometric distortion during the surface transformation is reduced. The feature preservation term is used to ensure that the geometric features (such as edges, textures, etc.) of each object in the scene are well preserved during the transformation process. The feature preservation term is achieved through a certain similarity metric (such as the structural similarity index). For the set of surface-insensitive parameters, an optimization objective function based on the mean square error is constructed. The mean square error is one of the most commonly used optimization objective functions and can effectively measure the difference between the rendered image and the target image. During the optimization process, the goal is to minimize the error between the rendering result and the real effect to achieve better lighting and material effects. The formula for the mean square error is:

[0126] ;

[0127] where, is the mean square error, is the value of the rendered image of the th pixel, is the value of the target image of the th pixel, is the total number of pixels in the image. Minimizing this loss function can optimize the lighting and material effects and reduce the errors generated during the rendering process. Optimize the set of surface-sensitive parameters. Use the gradient descent algorithm to optimize these parameters. The gradient descent algorithm calculates the gradient of the loss function with respect to the parameters and updates the parameters along the gradient direction to minimize the loss function. During the optimization process, introduce the curvature value to dynamically adjust the update step size to avoid overly drastic parameter updates in areas with large curvatures, which may lead to unstable rendering effects. For example, assume there is a set of surface-sensitive parameters , and the corresponding optimization function is , then the update rule is:

[0128] ;

[0129] where is the learning rate (step size), and is the gradient of the loss function. On this basis, adjust the learning rate according to the curvature value, using a smaller learning rate in areas with larger curvatures to ensure the stability of parameter updates. For the set of surface-insensitive parameters, use the Adam optimizer for optimization. The Adam optimizer is an optimization algorithm that combines the momentum method and adaptive learning rate and can converge to a better solution in a shorter time. By using the Adam optimizer, efficiently adjust the lighting and material parameters to ensure that the details of the rendering effect are more realistic. The update rule of the Adam optimizer is:

[0130] ;

[0131] ;

[0132] ;

[0133] where and are the first-order moment estimate and second-order moment estimate of the gradient respectively, and are the momentum parameters, is the learning rate, is a constant to prevent division by zero errors. After iterative optimization by the gradient descent and Adam optimizer, the optimized sets of surface-sensitive parameters and surface-insensitive parameters are obtained respectively. Merge these two optimized parameter sets to obtain the final target rendering parameter set.

[0134] In an example, perform 3D scene rendering using the target rendering parameter set and the surface enhancement feature map to obtain an arc screen display image, including:

[0135] Perform vertex transformation operations on the vertex data of the 3D scene according to the geometric transformation parameters in the target rendering parameter set to obtain the transformed vertex coordinates;

[0136] Perform projection matrix transformation on the transformed vertex coordinates according to the perspective projection parameters in the target rendering parameter set to obtain the projected vertex data;

[0137] Perform triangular patch division on the projected vertex data and perform curvature adaptive sampling on the triangular patches, where the sampling density is proportional to the curvature value, to obtain the sampled point data;

[0138] Perform lighting shading calculation on the sampled point data according to the lighting parameters and material parameters in the target rendering parameter set to obtain the initial pixel data;

[0139] Perform feature injection operation on the surface enhancement feature map and the initial pixel data to obtain the enhanced pixel data, and perform curvature-based anti-aliasing processing on the enhanced pixel data to obtain the curved screen display image.

[0140] In this example, the geometric transformation parameters in the target rendering parameter set include rotation, translation, and scaling operations, and the spatial position of the vertices of the 3D scene is adjusted through the transformation matrix. Suppose there is a vertex data set in a 3D scene , where each vertex is the spatial coordinate of an object in the scene. According to the geometric transformation parameters in the target rendering parameter set, a 4×4 transformation matrix is used to perform vertex transformation to obtain the new vertex coordinates , where the transformed coordinates are obtained through matrix operations:

[0141] ;

[0142] where, is the transformation matrix, including operations such as rotation, translation, and scaling, is the original vertex coordinate, is the transformed vertex coordinate. Through this step, the objects in the entire 3D scene are transformed to the new spatial position according to the geometric transformation requirements in the target rendering parameters. Perform projection matrix transformation on the transformed vertex coordinates using the perspective projection parameters in the target rendering parameter set. The projection matrix is to convert points in 3D space into points on a 2D image plane, including two methods: perspective projection and orthographic projection. Suppose the transformed vertex coordinates are , and the projection matrix is used to map these coordinates to the perspective projection plane to obtain the projected vertex data :

[0143] ;

[0144] Among them, is the perspective projection matrix, which is adjusted according to parameters such as the position of the camera and the viewing angle. is the transformed vertex coordinate, is the projected vertex coordinate. The projection process is realized through the perspective projection matrix. The perspective distortion is calculated according to parameters such as the viewing distance and focal length to obtain the final 2D image coordinates. Triangular patch division is performed based on the projected vertex data. In 3D graphics processing, a triangular patch is the most basic geometric unit, defined by three vertices. For a complex 3D scene, multiple vertices are divided into multiple triangular patches according to certain rules, and there may be different sampling densities on each patch. To cope with the influence brought by the curvature of the curved screen, curvature adaptive sampling is adopted. Curvature adaptive sampling means dynamically adjusting the sampling density according to the curvature value of each triangular patch. Suppose the curvature value of a triangular patch is , and the density of the sampling points is proportional to the curvature value:

[0145] ;

[0146] Among them, is the constant coefficient, is the curvature of the patch, is the sampling density. The greater the curvature, the higher the sampling density, and vice versa. In this way, those regions with larger curvature changes are more precisely controlled to improve the rendering accuracy, especially in complex curved regions. The lighting parameters and material parameters are applied to these sampling point data to calculate the lighting shading value of each sampling point. The basic goal of lighting shading calculation is to determine the color value of each sampling point according to factors such as the light source, material properties, and viewing angle. Suppose the normal vector of the sampling point is , the intensity of the light source is , the viewing angle is , and the material reflection coefficient is . Then, according to the Phong reflection model, the lighting shading value is calculated by the following formula:

[0147] ;

[0148] Among them, is the light source direction vector, is the reflection direction vector, is the specular exponent, is the ambient light reflection coefficient, is the diffuse reflection coefficient, is the specular reflection coefficient, and They are the normal vector and the viewing direction vector of the sampling point respectively. Through the above calculations, initial pixel data is generated for each sampling point. Perform feature injection operation on the surface enhancement feature map and the initial pixel data. Combine the surface feature map previously extracted by deep learning or other methods with the initial pixel data calculated during the rendering process to enhance the rendering effect. Assume the surface enhancement feature map is , and the initial pixel data is . Combine these two through weighted operation:

[0149] ;

[0150] where, is the weighting coefficient, which controls the fusion degree of the initial pixel data and the enhancement feature map, is the enhanced pixel data. Through feature injection, the quality of the rendering effect is effectively improved, making the surface details and lighting effects of the object more realistic. Perform curvature-based anti-aliasing processing on the enhanced pixel data to reduce the jagged phenomenon in the image caused by insufficient pixel resolution. During the curvature-based anti-aliasing processing, dynamically adjust the anti-aliasing intensity according to the curvature value of each pixel's location. Assume the curvature value of each pixel is , then the intensity of the anti-aliasing processing is proportional to the curvature value:

[0151] ;

[0152] where, is a constant coefficient, is the anti-aliasing intensity, is the curvature value. In areas with larger curvature, the anti-aliasing processing intensity is larger, thus effectively reducing the jagged phenomenon generated in these areas. Through the above steps, a smooth and clear curved screen display image is finally obtained.

[0153] Referring to Figure 2 , this embodiment provides a 3D scene rendering device based on a curved screen, including:

[0154] Frequency domain decomposition module 1, used to perform frequency domain decomposition on the 3D scene data to obtain scene frequency domain feature data;

[0155] Construction module 2, used to construct a surface mapping matrix according to the curvature parameter and radian parameter of the curved screen;

[0156] Feature mapping module 3, used to perform 3D scene space feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a space mapping feature map;

[0157] A feature processing module 4, configured to input a spatial mapping feature map into a multi-curvature convolution kernel for surface feature processing to obtain a surface enhanced feature map;

[0158] A rendering parameter optimization module 5, configured to divide an initial rendering parameter set into a surface-sensitive parameter set and a surface-insensitive parameter set, and perform rendering parameter optimization to obtain a target rendering parameter set;

[0159] A 3D scene rendering module 6, configured to perform 3D scene rendering through the target rendering parameter set and the surface enhanced feature map to obtain an arc screen display image.

[0160] In this embodiment, for the specific implementation of each unit in the above device embodiment, please refer to the description in the above method embodiment, and details will not be repeated here.

[0161] Refer to Figure 3 , in the embodiment of the present invention, a computer device is further provided. The computer device may be a server, and its internal structure may be as Figure 3 shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0162] Those skilled in the art can understand that Figure 3 the structure shown in

[0163] is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0164] The above are only the preferred embodiments of the present invention, and do not thereby limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall similarly be included within the patent protection scope of the present invention.

Claims

1. A 3D scene rendering method based on a curved screen, characterized in that: The following steps are involved: Perform frequency domain decomposition on 3D scene data to obtain scene frequency domain feature data; Constructing a surface mapping matrix according to the curvature parameters and radian parameters of the curved screen; Performing 3D scene spatial feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map; Input the spatial mapping feature map into the multi-curvature convolution kernel for surface feature processing to obtain a surface enhancement feature map; specifically including: inputting the spatial mapping feature map into the multi-curvature convolution kernel for feature calculation to obtain three groups of feature data with different curvature viewing angles; dividing the curved screen into n equal arc segments, and assigning independent feature enhancement parameters to each arc segment to obtain an arc segment parameter set; constructing the n arc segments into a graph structure, wherein each arc segment is a graph node, and the relationship between adjacent arc segments is a graph edge, to obtain an arc segment relationship graph; performing a graph convolution operation on the feature data of the three groups with different curvature viewing angles and the arc segment relationship graph, establishing a spatial dependency relationship between features, and obtaining a spatial dependency feature; performing a feature enhancement operation according to the spatial dependency feature and the arc segment parameter set to obtain the surface enhancement feature map; The initial rendering parameter set is divided into a surface-sensitive parameter set and a surface-insensitive parameter set, and rendering parameter optimization is performed to obtain a target rendering parameter set; specifically comprising: dividing the initial rendering parameter set by parameter type to obtain a surface-sensitive parameter set and a surface-insensitive parameter set, wherein the surface-sensitive parameter set includes geometric transformation parameters and perspective projection parameters, and the surface-insensitive parameter set includes lighting parameters and material parameters; constructing a surface-sensitive optimization function including a surface distortion measurement item and a feature preservation item for the surface-sensitive parameter set, and at the same time, optimizing the surface-insensitive The surface-sensitive parameter set is constructed by constructing an optimization objective function based on mean square error to obtain a surface-insensitive optimization function; a gradient descent operation is performed on the surface-sensitive parameter set according to the surface-sensitive optimization function, and the update step size is dynamically adjusted through the curvature value to obtain an optimized surface-sensitive parameter set; an Adam optimizer operation is performed on the surface-insensitive parameter set according to the surface-insensitive optimization function to obtain an optimized surface-insensitive parameter set; the optimized surface-sensitive parameter set and the optimized surface-insensitive parameter set are merged to obtain a target rendering parameter set; 3D scene rendering is performed using the target rendering parameter set and the curved surface enhancement feature map to obtain a curved screen display image.

2. The 3D scene rendering method based on a curved screen according to claim 1, characterized in that: The step of performing frequency domain decomposition on the 3D scene data to obtain scene frequency domain feature data includes: Performing a 5-layer Laplace pyramid decomposition on the 3D scene data to obtain multi-level scene data; Performing wavelet transform processing on the multi-level scene data to obtain horizontal component data, vertical component data and diagonal component data of each level; Input the horizontal component data, the vertical component data, and the diagonal component data into a first feature extraction branch, and perform three consecutive feature extraction operations through a 3×3 convolution kernel and a ReLU activation function of the first feature extraction branch to obtain high-frequency feature data; Inputting the horizontal component data, the vertical component data and the diagonal component data into a second feature extraction branch, performing a feature extraction operation through a 7×7 convolution kernel and a linear activation function of the second feature extraction branch, and obtaining low-frequency feature data; The high-frequency feature data and the low-frequency feature data are used as scene frequency domain feature data.

3. The 3D scene rendering method based on a curved screen according to claim 1, characterized in that: The step of constructing a curved surface mapping matrix according to the curvature parameters and radian parameters of the curved screen includes: Constructing the parameterized surface equation of the curved screen according to the curvature parameter and the radian parameter of the curved screen; Performing transformation analysis on the curved screen parameterized surface equation to obtain a 4×4 basic transformation matrix including a translation component, a rotation component, and a scaling component; Constructing a nonlinear mapping function according to the curved screen parameterized surface equation, introducing the surface deformation correction term into the nonlinear mapping function, and obtaining a nonlinear correction matrix; Performing a matrix multiplication operation on the basic transformation matrix and the nonlinear correction matrix to obtain an initial mapping matrix; The mapping parameter correction amount is calculated according to the scene frequency domain feature data to obtain a parameter adjustment matrix, and the initial mapping matrix and the parameter adjustment matrix are combined to obtain a surface mapping matrix.

4. The 3D scene rendering method based on a curved screen according to claim 1, characterized in that: The performing 3D scene spatial feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map includes: The scene frequency domain feature data is input into the attention mechanism unit, and the spatial attention feature and the channel attention feature are extracted respectively through the 3×3 deformable convolution and the 1×1 convolution in the attention mechanism unit to obtain a dual attention feature map; Performing skip connection processing on the dual attention feature map, establishing associations between features at different levels, obtaining multi-level associated features, and performing depth importance calculation on the multi-level associated features based on scene depth information to obtain a depth weight coefficient; Performing weighted calculation on the multi-level associated features according to the depth weight coefficient to obtain a weighted feature map; Decomposing the surface mapping matrix into a geometric transformation part and a nonlinear correction part, mapping the weighted feature map step by step to obtain mapping feature data; An adaptive bilinear interpolation operation based on curvature is performed on the mapping feature data to obtain a spatial mapping feature map.

5. The 3D scene rendering method based on a curved screen according to claim 1, characterized in that: The step of performing 3D scene rendering using the target rendering parameter set and the curved surface enhancement feature map to obtain a curved screen display image includes: Performing vertex transformation operations on vertex data of the 3D scene according to the geometric transformation parameters in the target rendering parameter set to obtain transformed vertex coordinates; Performing a projection matrix transformation on the transformed vertex coordinates according to the viewing angle projection parameters in the target rendering parameter set to obtain projection vertex data; Divide the projection vertex data into triangular patches, and perform curvature adaptive sampling on the triangular patches, wherein the sampling density is proportional to the curvature value, to obtain sampling point data; According to the illumination parameters and material parameters in the target rendering parameter set, performing illumination shading calculation on the sampling point data to obtain initial pixel data; The curved surface enhancement feature map and the initial pixel data are subjected to feature injection operation to obtain enhanced pixel data, and curvature-based anti-aliasing processing is performed on the enhanced pixel data to obtain a curved screen display image.

6. A 3D scene rendering device based on a curved screen, characterized in that: For implementing the steps of the method according to any one of claims 1 to 5, the device comprises: The frequency domain decomposition module is used to perform frequency domain decomposition on 3D scene data to obtain scene frequency domain feature data; A construction module, used for constructing a surface mapping matrix according to the curvature parameters and radian parameters of the curved screen; A feature mapping module, used to perform 3D scene spatial feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map; A feature processing module, used for inputting the spatial mapping feature map into a multi-curvature convolution kernel to perform surface feature processing to obtain a surface enhancement feature map; A rendering parameter optimization module, used to divide the initial rendering parameter set into a surface-sensitive parameter set and a surface-insensitive parameter set, and perform rendering parameter optimization to obtain a target rendering parameter set; The 3D scene rendering module is used to perform 3D scene rendering through the target rendering parameter set and the curved surface enhancement feature map to obtain a curved screen display image.

7. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

Images