3D scene rendering method, device and equipment based on arc-shaped screen

Abstract

The invention relates to a 3D scene rendering method, device and equipment based on an arc-shaped screen. The method comprises the following steps: performing frequency domain decomposition on 3D scene data to obtain scene frequency domain feature data; constructing a curved surface mapping matrix according to the curvature parameter and the radian parameter of the arc-shaped screen; performing 3D scene spatial feature mapping based on the scene frequency domain feature data and the curved surface mapping matrix to obtain a spatial mapping feature map; inputting the spatial mapping feature map into a multi-curvature convolution kernel to perform curved surface feature processing to obtain a curved surface enhanced feature map; dividing the initial rendering parameter set into a curved surface sensitive type parameter set and a curved surface insensitive type parameter set, and executing rendering parameter optimization to obtain a target rendering parameter set; and performing 3D scene rendering through the target rendering parameter set and the curved surface enhancement feature map to obtain an arc screen display image. The display quality of the rendering result in different areas of the arc-shaped screen is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of 3D scene rendering, and in particular to a 3D scene rendering method, device and equipment based on a curved screen. Background Art

[0002] Curved screens are becoming an important development direction for the next generation of display devices due to their unique geometric characteristics and immersive viewing experience. Compared with traditional flat-panel displays, curved screens can provide users with a wider field of view and a more natural viewing experience. However, existing 3D scene rendering technologies are mainly designed for flat-panel display devices. Directly applying these rendering methods on curved screens often leads to problems such as image distortion and poor spatial consistency.

[0003] The main challenge facing the current curved screen 3D scene rendering is how to accurately handle the geometric transformation problems caused by curved display. Traditional rendering methods are prone to edge distortion and depth distortion on curved screens, and it is difficult to maintain the spatial continuity of the scene. Especially in large curvature display scenes, due to the nonlinear characteristics of projection transformation, conventional rendering parameter optimization methods are difficult to take into account both the geometric accuracy and visual quality of the scene. Summary of the invention

[0004] The main purpose of the present invention is to provide a 3D scene rendering method, device and equipment based on a curved screen, so as to improve the display quality of the rendering results in different areas of the curved screen.

[0005] To achieve the above object, the present invention provides a 3D scene rendering method based on a curved screen, comprising the following steps: Perform frequency domain decomposition on 3D scene data to obtain scene frequency domain feature data; Constructing a surface mapping matrix according to the curvature parameters and radian parameters of the curved screen; Performing 3D scene spatial feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map; Inputting the spatial mapping feature map into a multi-curvature convolution kernel to perform surface feature processing to obtain a surface enhancement feature map; The initial rendering parameter set is divided into a surface-sensitive parameter set and a surface-insensitive parameter set, and rendering parameter optimization is performed to obtain a target rendering parameter set; 3D scene rendering is performed using the target rendering parameter set and the curved surface enhancement feature map to obtain a curved screen display image.

[0006] The present invention also provides a 3D scene rendering device based on a curved screen, comprising: The frequency domain decomposition module is used to perform frequency domain decomposition on 3D scene data to obtain scene frequency domain feature data; A construction module, used for constructing a surface mapping matrix according to the curvature parameters and radian parameters of the curved screen; A feature mapping module, used to perform 3D scene spatial feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map; A feature processing module, used for inputting the spatial mapping feature map into a multi-curvature convolution kernel to perform surface feature processing to obtain a surface enhancement feature map; A rendering parameter optimization module, used to divide the initial rendering parameter set into a surface-sensitive parameter set and a surface-insensitive parameter set, and perform rendering parameter optimization to obtain a target rendering parameter set; The 3D scene rendering module is used to perform 3D scene rendering through the target rendering parameter set and the curved surface enhancement feature map to obtain a curved screen display image.

[0007] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0008] In summary, the technical solution provided by the present invention realizes the effective separation and extraction of high-frequency details and low-frequency structural information through the design of frequency domain decomposition and bidirectional feature extraction network, thereby improving the detail fidelity and structural integrity of the rendering result; the surface mapping matrix combined with the nonlinear correction mechanism is adopted to effectively solve the geometric deformation problem in the curved screen rendering, and ensure the spatial consistency of the rendered scene when it is displayed on the curved surface; the multi-scale feature pyramid structure and the dual attention mechanism are introduced to enhance the accuracy of feature extraction, so that the rendering result can better maintain the local details and global structure of the scene; through the design of multi-curvature convolution kernels and graph convolution networks, adaptive feature enhancement of different curvature regions is realized, and the display quality of the rendering result in different regions of the curved screen is improved; a dual-track optimization strategy based on parameter classification is designed, and different optimization methods are adopted for surface sensitive parameters and insensitive parameters, respectively, to improve the accuracy and efficiency of rendering parameter optimization; in the final rendering stage, the curvature adaptive sampling and feature injection strategy are adopted to significantly improve the edge clarity and detail expression of the rendered image, and reduce display distortion. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a schematic diagram of the steps of a 3D scene rendering method based on a curved screen in one embodiment of the present invention; Figure 2 It is a structural block diagram of a 3D scene rendering device based on a curved screen in one embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0010] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0012] Reference Figure 1 This embodiment provides a 3D scene rendering method based on a curved screen, comprising the following steps: S1, performing frequency domain decomposition on the 3D scene data to obtain scene frequency domain feature data; Among them, the 3D scene data is decomposed into 5 layers of Laplacian pyramids, and the construction of multi-level scene data is realized through the 5-layer Laplacian pyramid decomposition. Laplacian pyramid decomposition is an effective image processing method. It generates image levels of different resolutions through Gaussian blurring and downsampling, so as to extract information of different scales. The input 3D scene data is Gaussian blurred to obtain a lower resolution image, and then the high-level data of the pyramid is constructed by downsampling layer by layer. The data of each layer is subtracted from the Gaussian blurred version of the previous layer. The residual information obtained is the Laplacian pyramid data of this layer. This step can effectively separate low-frequency and smooth information while retaining the edge and detail features of the scene. Through the 5-layer decomposition, it is ensured that information of different scales is captured. Wavelet transform is performed on the multi-level scene data to extract frequency information in different directions. Wavelet transform is a signal processing method that provides localization characteristics in both time domain and frequency domain. Two-dimensional wavelet transform is used to decompose the scene data of each layer to obtain horizontal component data, vertical component data and diagonal component data. These three components represent the horizontal details, vertical details and diagonal details in the scene respectively. The horizontal component data, vertical component data, and diagonal component data are input into the first feature extraction branch. In the first feature extraction branch, the horizontal component data, vertical component data, and diagonal component data are respectively input into the 3×3 convolution kernel, and the ReLU activation function is combined to perform three consecutive feature extraction operations. The 3×3 convolution kernel is selected based on its good local perception ability, which enables it to effectively capture local texture information, while the ReLU activation function enhances the nonlinear feature expression ability and prevents the gradient disappearance problem. Through three consecutive convolution operations, this branch fully extracts high-frequency information in the image, such as edges, textures, and rapidly changing areas. At the same time, in the second feature extraction branch, the horizontal component data, vertical component data, and diagonal component data are also used as inputs. This branch uses a 7×7 convolution kernel and uses a linear activation function for feature extraction. Compared with the 3×3 convolution kernel, the 7×7 convolution kernel can perceive a larger area range and is more suitable for extracting low-frequency information, that is, the global structure and gradient information in the scene, while the use of the linear activation function ensures the smoothness and consistency of the low-frequency features and does not introduce nonlinear transformations in the feature extraction process. This branch can effectively extract the low-frequency information of the scene, ensuring that global features such as lighting and shadows can be correctly preserved during the rendering process, thereby avoiding over-sharpening or distortion. The high-frequency feature data obtained by the first feature extraction branch and the low-frequency feature data obtained by the second feature extraction branch are combined to construct the scene frequency domain feature data.

[0013] S2, constructing a surface mapping matrix according to the curvature parameters and radian parameters of the curved screen; Specifically, the parameterized surface equation of the curved screen is constructed according to the curvature parameter and the radian parameter of the curved screen. Since the shape of the curved screen is approximately represented as a curved surface with a certain radius, the parameterized surface is used to represent its geometric characteristics. Assuming that the curvature parameter of the curved screen is K and the radian parameter is θ, the parameterized surface equation of the curved screen in three-dimensional space is defined, for example, a cylindrical coordinate system or a spherical coordinate system is used to describe the position of any point on the screen, thereby establishing its mathematical expression. The parameterized surface equation of the curved screen is transformed and analyzed to construct a 4×4 basic transformation matrix containing translation, rotation and scaling components. Since the projection of 3D scenes is usually performed in a Cartesian coordinate system, and the surface of the curved screen is a curved surface, a conversion relationship is established between different coordinate systems. The translation component in the basic transformation is considered to adjust the center position of the scene on the screen so that it is aligned with the geometric center of the curved screen. Then the rotation component is calculated to ensure that the direction of the scene conforms to the surface characteristics of the curved screen, and the scaling component is used to adjust the spatial scale of the scene under different radians to avoid image distortion caused by different screen shapes. Combining these three components, a standard 4×4 transformation matrix is ​​constructed to make the 3D scene initially adapt to the display requirements of the curved screen. A nonlinear mapping function is constructed based on the parameterized surface equation of the curved screen, and the surface deformation correction term is introduced into it to obtain a nonlinear correction matrix. By analyzing the local curvature changes at different positions of the curved screen, a corresponding nonlinear function is established so that the coordinate transformation adapts to the actual form of the curved screen, for example, a correction term of quadratic or cubic curve fitting is introduced in different screen areas to compensate for distortion and ensure the geometric accuracy of the final rendered image. Through this step, the error of the traditional linear transformation method on the curved screen is effectively reduced, so that the 3D scene can adapt to the curvature characteristics of the screen more naturally. The 4×4 basic transformation matrix and the nonlinear correction matrix are matrix multiplied to obtain the initial mapping matrix. The mapping parameter correction amount is calculated according to the scene frequency domain feature data to construct a parameter adjustment matrix. The projection error of different frequency components on the curved screen is analyzed based on the frequency domain feature data of the scene, for example, the error distribution in the local space is calculated by Fourier transform or wavelet transform, and a correction amount is constructed using the error information so that the mapping matrix can be dynamically adjusted to adapt to different scene characteristics. The initial mapping matrix and the parameter adjustment matrix are combined to obtain the final surface mapping matrix.

[0014] S3, performing 3D scene spatial feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map; It should be noted that the scene frequency domain feature data is subjected to deep feature extraction, and the scene frequency domain feature data is processed by the attention mechanism unit, so that the feature extraction process is more focused on the key areas. In the attention mechanism unit, a 3×3 deformable convolution is used to adaptively extract spatial features. Since the deformable convolution can dynamically adjust the receptive field, it is more adaptable to the complex geometric form of the curved screen. Compared with the traditional fixed convolution operation, it can more effectively capture the feature changes caused by surface deformation. At the same time, in the attention mechanism unit, a 1×1 convolution is used to extract channel attention features, so that the information of different channels can be adaptively weighted to enhance the feature expression ability. By combining these two methods, a dual attention feature map is obtained, which more accurately represents the spatial features of the scene and adapts to the special form of the curved screen. The dual attention feature map is processed by jump connection, and the features of different levels are fused to establish the connection between global information and local information. Jump connection can enhance the semantic information of low-level features while retaining the detailed information of high-level features, making the feature expression more complete. After completing the multi-level feature association, considering the depth information of the scene, the contribution of features at different depths to the final rendering effect is uneven. The depth importance coefficient is calculated based on the scene depth information to obtain the depth weight coefficient. The depth weight coefficient is calculated by analyzing the gradient changes of different depth areas in the scene, or learning through a depth perception network, so as to adaptively allocate the feature weights of different areas to ensure that the spatial structure of the scene can be restored more realistically in the final rendering. The multi-level associated features are weighted according to the depth weight coefficient, so that the expression of the features is more consistent with the depth distribution of the scene, and a weighted feature map is obtained. The depth weight is directly applied to the feature distribution by weighted sum operation or dot product operation to ensure that the feature mapping is more accurate. In order to correctly map the features to the curved coordinate space of the curved screen, the surface mapping matrix is ​​used for transformation, but since the matrix contains both geometric transformation information and nonlinear correction information, it is decomposed into two parts to make the transformation process more refined. The weighted feature map is mapped in the geometric transformation part. This step mainly involves linear transformation operations such as coordinate conversion, rotation, and scaling, so that the features are projected into the initial mapping space of the curved screen. The nonlinear correction part is used to adjust the mapping result to compensate for the nonlinear deformation caused by the screen curvature, so that the final mapping feature data can more accurately match the geometric shape of the screen. Perform a curvature-based adaptive bilinear interpolation operation on the mapping feature data. Introduce a curvature adaptive mechanism to adjust the distribution of weights during the interpolation process so that the interpolation calculation can adapt to the morphological changes in different curvature areas. For example, in areas with larger screen curvature, the non-uniformity of the interpolation weights is appropriately increased to reduce geometric distortion, while in areas with smaller curvature, a standard bilinear interpolation method is used to ensure computational efficiency. Through this step, the final spatial mapping feature map is obtained.

[0015] S4, inputting the spatial mapping feature map into the multi-curvature convolution kernel for surface feature processing to obtain a surface enhanced feature map; Specifically, a multi-curvature convolution kernel adapted to different curvature regions is constructed according to the geometric characteristics of the curved screen, and the curvature values ​​are set as , and , ensuring that the most suitable spatial features can be extracted within different curvature ranges. Since different curvature areas lead to changes in light projection, perspective distortion, and texture mapping, each set of curvature convolution kernels contains two convolution layers and a batch normalization layer to ensure the stability of feature extraction. The first two convolution layers are used to learn local spatial features and adapt to the influence of different curvatures, while the batch normalization layer is used to normalize the feature distribution to prevent gradient disappearance or explosion, while improving the convergence speed of training, making the final surface features smoother and more generalizable. After constructing the multi-curvature convolution kernels, the spatial mapping feature maps are respectively input into these multi-curvature convolution kernels for feature calculation to obtain three sets of feature data from different curvature perspectives. Each convolution kernel has a different range of action, among which It is mainly used for feature extraction in low curvature areas and is suitable for parts close to the plane. Suitable for medium curvature areas, it can capture the curved characteristics of the screen to a certain extent. It is mainly used in high curvature areas, so that the feature expression can better adapt to the edge of the screen or the area with a large degree of curvature. In this way, feature calculations are performed within different curvature ranges to ensure that the features of different curvature areas can be extracted and expressed in the best way during the final rendering, avoiding the distortion and blurring problems that occur when processing curved screens with traditional convolution. After obtaining three sets of feature data with different curvature perspectives, the curved screen is divided into spatial structures to more accurately match the feature expression. The entire curved screen is divided into n equal arc segments, and each arc segment is assigned an independent feature enhancement parameter to form an arc segment parameter set. Different parameters are set in different curvature areas so that the feature enhancement process can adaptively adjust the feature expression to reduce the error caused by the global parameterization method. In order to construct the relationship between different arc segments, n arc segments are constructed as a graph structure, in which each arc segment is regarded as a graph node, and the relationship between adjacent arc segments is regarded as a graph edge, and an arc segment relationship graph is obtained. Graph convolution operations are performed on the feature data of three sets of different curvature perspectives and the arc segment relationship graph to establish the spatial dependency relationship between features. Since traditional CNN methods have limitations when processing non-Euclidean spatial data, and the data distribution on the curved screen is non-uniform, graph convolution can better learn the feature interactions between different arc segments, so that the feature information of adjacent areas can be shared and the global feature expression can be strengthened. The graph convolution operation aggregates the features of different regions through the adjacency matrix and the normalized Laplace matrix, so that the final spatially dependent features can accurately reflect the mutual influence between different arc segments, improve the spatial consistency of the rendering, and obtain spatially dependent features. Feature enhancement operations are performed based on the spatially dependent features and the arc parameter set to obtain a surface enhanced feature map. The feature enhancement process is based on the feature parameters of different arc segments, and the feature data is weighted and summed by weight adjustment, so that the features of each area can be more consistent with its curvature characteristics.

[0016] S5, dividing the initial rendering parameter set into a surface-sensitive parameter set and a surface-insensitive parameter set, and performing rendering parameter optimization to obtain a target rendering parameter set; Among them, the initial rendering parameters are classified to distinguish between parameters that are greatly affected by the curved screen and parameters that are relatively independent of the surface morphology. The initial rendering parameter set is divided into a surface-sensitive parameter set and a surface-insensitive parameter set. Among them, the surface-sensitive parameter set includes geometric transformation parameters and perspective projection parameters. These parameters determine the projection method of the 3D scene on the curved screen and are directly affected by the curvature of the screen. The surface-insensitive parameter set mainly includes lighting parameters and material parameters. Although these parameters affect the visual style and rendering effect of the scene, they will not directly change due to the geometric morphology of the screen. After the parameter classification is completed, optimization functions are constructed for these two types of parameters to ensure that different types of parameters can be optimally adjusted. For the surface-sensitive parameter set, its optimization function considers both surface distortion and feature preservation, so it includes two key items. One is the surface distortion metric, which is used to measure the projection error of the 3D scene on the curved screen, such as calculating the deviation of the original geometric information after the surface projection, to ensure that the optimized parameters can minimize the distortion; the other is the feature preservation item, which is used to ensure that the important geometric features of the scene will not be lost due to optimization, so as to avoid the structural distortion of the 3D model on the curved screen. Through the joint action of these two parts, it is ensured that the optimized parameters can reduce distortion and maintain the integrity of spatial features. For the surface-insensitive parameter set, the mean square error is used as the optimization target to ensure that the optimized lighting and material parameters can be as close to the optimal visual effect as possible and maintain stability under different rendering environments. After constructing the optimization target, the optimization calculation is performed next, and the gradient descent operation is performed on the surface-sensitive parameter set. Since the curvature of the curved screen varies in different areas, during the optimization process, the optimization step size is dynamically adjusted according to the curvature of the screen, and the step size is reduced in the high curvature area to ensure that the parameter changes are not too drastic, while the step size is appropriately increased in the low curvature area to speed up the optimization speed. It effectively prevents over-optimization in places where the curvature changes drastically, thereby improving the stability of the final rendering results. After multiple rounds of iterative optimization, the surface-sensitive parameter set can gradually converge, making the projection of the 3D scene more accurate and reducing the visual distortion caused by the screen curvature. At the same time, for the surface-insensitive parameter set, an adaptive optimization method is used for optimization to ensure that the adjustment of lighting and material parameters can be more efficient and accurate. During the optimization process, different lighting conditions and material reflection characteristics are dynamically adjusted so that the optimized parameters can maintain good visual performance in different scenes, while avoiding lighting distortion or loss of material details due to optimization. Through this step, the final rendering effect is ensured to be more stable, so that no matter how the curvature of the screen changes, the lighting and material can still maintain a natural and consistent performance. After the surface-sensitive parameter set and the surface-insensitive parameter set are optimized respectively, the two are merged to form a final target rendering parameter set.During the merging process, ensure that the geometric transformation parameters match the lighting and material parameters to ensure that there will be no inconsistency during the rendering process. For example, after the optimized geometric parameters are adjusted, the projection method of the lighting is affected, so corresponding compensation is made during the merging process to ensure that the final rendering effect can maintain the best visual performance on the curved screen.

[0017] S6, performing 3D scene rendering using a target rendering parameter set and a surface enhancement feature map to obtain a curved screen display image.

[0018] Specifically, the vertex data of the 3D scene are transformed using the geometric transformation parameters in the target rendering parameter set. This process involves operations such as coordinate transformation, rotation, and scaling to adjust the vertex positions of the original 3D scene to a spatial coordinate system adapted to the curved screen. Since the curved screen has a specific curvature distribution, when performing vertex transformation, the influence of the curvature of different regions on the vertex position is considered to ensure that the transformed vertex coordinates can be accurately mapped to the display area of ​​the curved screen, so that the spatial structure of the entire scene remains consistent. After the vertex transformation is completed, the projection matrix transformation is performed on the transformed vertex coordinates to ensure that the 3D scene can be correctly projected onto the curved screen. In this process, the perspective projection parameters in the target rendering parameter set are used to convert the transformed vertex coordinates to the screen coordinate system, so that the 3D model can be correctly projected onto the curved screen surface. Since the traditional perspective projection method is suitable for flat screens, and the projection relationship of the curved screen is more complicated, a curved surface projection correction method is introduced in the projection transformation process to compensate for the perspective distortion caused by the screen curvature, so that the final projection vertex data can accurately match the geometric characteristics of the curved screen. After obtaining the projection vertex data, it is divided into triangles, and curvature adaptive sampling is performed on this basis to ensure that different curvature areas can be optimized during the rendering process. The purpose of triangle division is to discretize the entire 3D scene so that the rendering calculation can be processed in units of polygonal fragments. During the division process, the density of the triangles is dynamically adjusted according to the curvature characteristics of the screen to ensure that finer geometric details can be obtained in high curvature areas, while maintaining a lower sampling density in low curvature areas to improve calculation efficiency. In the curvature adaptive sampling process, the sampling density is proportional to the curvature value, that is, the number of sampling points is increased in areas with larger curvatures to enhance the ability to express details, while the number of sampling points is reduced in areas with smaller curvatures to avoid wasting computing resources and obtain optimized sampling point data. After obtaining the sampling point data, the sampling point data is subjected to illumination shading calculation based on the illumination parameters and material parameters in the target rendering parameter set to generate initial pixel data. In the illumination calculation process, the light source position, the reflection characteristics of the material, the normal information and other factors are combined to calculate the illumination intensity and color distribution of each sampling point. Since the lighting effect of the curved screen is affected by the characteristics of the curved surface, the influence of the screen curvature on the lighting is considered during the calculation process. For example, the light focusing effect occurs in the high curvature area, resulting in increased local brightness, while the light distribution is more uniform in the low curvature area. Therefore, corresponding lighting compensation is required to ensure that the final lighting effect can be balanced across the entire screen. After obtaining the initial pixel data, the surface enhancement feature map is combined to perform feature injection operations to enhance the expressiveness of the pixel data.The surface enhancement feature map contains the surface feature information extracted in the previous calculation process, such as curvature features, spatial dependency features, texture enhancement information, etc. Therefore, in the feature injection process, these features are fused with the initial pixel data to enhance the spatial consistency of the final rendering result. For example, the surface enhancement feature is used to optimize the lighting calculation, so that the shadows and highlights in the high curvature area are more natural, or the texture enhancement information in the feature map is used to make the material details of the scene more clearly displayed, thereby improving the quality of the final rendering result. Curvature-based anti-aliasing processing is performed on the enhanced pixel data to reduce the edge jagged effect caused by pixelation. A curvature-based adaptive anti-aliasing method is used to adjust the interpolation calculation method according to the local curvature, thereby reducing the edge jagged in the high curvature area and ensuring that the final rendering result can maintain a smooth and natural visual effect on the entire curved screen. After calculation, a display image suitable for the curved screen is generated, so that the 3D scene can be presented on the curved screen in a high-quality form and can maintain the best visual performance under different curvature conditions.

[0019] In one example, the 3D scene data is decomposed in the frequency domain to obtain scene frequency domain feature data, including: Perform 5-layer Laplacian pyramid decomposition on the 3D scene data to obtain multi-level scene data; Perform wavelet transform processing on multi-level scene data to obtain horizontal component data, vertical component data and diagonal component data of each level; The horizontal component data, the vertical component data, and the diagonal component data are input into the first feature extraction branch, and the feature extraction operations are performed three times in a row through the 3×3 convolution kernel and the ReLU activation function of the first feature extraction branch to obtain high-frequency feature data; The horizontal component data, the vertical component data, and the diagonal component data are input into the second feature extraction branch, and a feature extraction operation is performed through the 7×7 convolution kernel and the linear activation function of the second feature extraction branch to obtain low-frequency feature data; The high-frequency feature data and the low-frequency feature data are used as scene frequency domain feature data.

[0020] In this example, a 5-layer Laplacian pyramid decomposition is performed on the 3D scene data. The Laplacian pyramid decomposition gradually decomposes the detailed information of the image or data into multiple scales through layer-by-layer filtering and downsampling, and obtains low-frequency and high-frequency information of each layer. Each layer of the Laplacian pyramid contains scene data of different scales, where the low-frequency part contains the overall outline information of the scene, and the high-frequency part contains more detailed textures and details. Each layer of the Laplacian pyramid decomposition is represented as: ; in, Indicates The Laplacian pyramid data of the layer, Indicates Gaussian blurred image of the layer, The pyramid layer representing the next level is usually obtained by Gaussian blurring and downsampling. In this way, the Laplacian pyramid effectively extracts the feature information of the scene at different scales. For each layer of the Laplacian pyramid data, wavelet transform processing is performed to extract frequency domain features. Wavelet transform effectively analyzes the local characteristics of the signal at multiple scales and decomposes the signal into multiple frequency sub-bands. A two-dimensional wavelet transform is performed on each layer of the Laplacian pyramid data to obtain horizontal component data, vertical component data, and diagonal component data. The two-dimensional wavelet transform is expressed as: ; in, Represents the input scene data, and are the filter coefficients of the wavelet transform, is the position coordinate. The output after wavelet transformation contains multiple components, among which the horizontal component data captures the detail information in the horizontal direction, the vertical component data captures the detail information in the vertical direction, and the diagonal component data captures the diagonal features in the image. These frequency domain components effectively describe the texture information of the scene in different directions and at different scales. The horizontal, vertical and diagonal component data of each layer are input into two different feature extraction branches for feature processing. The first feature extraction branch uses a 3×3 convolution kernel and combines the ReLU activation function to perform three consecutive feature extraction operations to extract high-frequency features in the scene data to represent details and texture information. In the convolution operation, each convolution extracts specific local features and introduces nonlinear factors through the ReLU activation function to enhance the expression ability of the network. The specific convolution operation is expressed as: ; in, represents the convolution result, is the convolution kernel, is the input data, is the bias term, Represents the convolution operation, and ReLU represents the activation function. Through three consecutive convolution operations, high-frequency features in the input data are gradually extracted. The second feature extraction branch uses a 7×7 convolution kernel and a linear activation function for feature extraction. The main task of this branch is to extract low-frequency features, which represent the overall structure and large-scale information of the scene. Unlike the first branch, the linear activation function retains the original value of the feature data, so that low-frequency information can be extracted in its original form. The convolution operation of this process is similar to the above, but the activation function is different. The formula is: ; represents the convolution result, is the convolution kernel, is the input data, It is a bias term, but the ReLU activation function is not introduced because the system focuses on the linear expression of low-frequency features. Through the processing of the first branch and the second branch, high-frequency feature data and low-frequency feature data are obtained respectively. High-frequency feature data captures detailed information, while low-frequency feature data retains the overall structure of the scene. Combining these two feature data, we can get the complete scene frequency domain feature data.

[0021] In this embodiment, the scene frequency domain feature data is used to construct a surface mapping matrix after feature alignment processing, and the feature alignment processing includes: inputting the scene frequency domain feature data into a parallel convolutional neural network branch and a Transformer branch, extracting local fine-grained features through the convolutional neural network branch, and extracting long-distance context features through the Transformer branch to obtain dual-branch feature data; constructing a multi-scale feature pyramid for the dual-branch feature data, performing feature extraction at different scale levels, and obtaining a multi-scale feature sequence; inputting the multi-scale feature sequence into a feature interaction module, performing cross-branch attention calculation on features between different branches, and obtaining interactive attention weights; performing weighted fusion on the dual-branch features according to the interactive attention weights, and introducing residual connections to obtain primary semantic alignment features; performing self-attention calculation on the primary semantic alignment features, establishing long-distance dependencies within the features, and obtaining a self-attention feature map; inputting the self-attention feature map into a multi-layer perceptron for nonlinear transformation to obtain high-level semantic alignment features; performing residual self-interaction operations on the high-level semantic alignment features, fusing global information and local information, and obtaining final semantic alignment features; and using the final semantic alignment features as the optimized scene frequency domain feature data.

[0022] In one example, a surface mapping matrix is ​​constructed according to the curvature parameters and radian parameters of the curved screen, including: Constructing the parameterized surface equation of the curved screen according to the curvature parameter and the radian parameter of the curved screen; Perform transformation analysis on the curved screen parameterized surface equation to obtain a 4×4 basic transformation matrix containing translation, rotation and scaling components; A nonlinear mapping function is constructed according to the parametric surface equation of the curved screen, and the surface deformation correction term is introduced into the nonlinear mapping function to obtain a nonlinear correction matrix. Performing matrix multiplication operation on the basic transformation matrix and the nonlinear correction matrix to obtain an initial mapping matrix; The mapping parameter correction amount is calculated according to the scene frequency domain feature data to obtain the parameter adjustment matrix, and the initial mapping matrix and the parameter adjustment matrix are combined to obtain the surface mapping matrix.

[0023] In this example, the parametric surface equation of the curved screen is constructed based on the curvature parameter and radian parameter of the curved screen. The curvature parameter of the curved screen indicates the degree of curvature of the curved screen surface, while the radian parameter defines the curvature angle of the curved screen. The function is expressed as a two-dimensional coordinate system in a parametric way, and the polar coordinate system is used to represent the surface characteristics. Assuming that the center of the curved screen is at the origin, the parametric surface equation is written as: ; ; ; in, is the radius of the curved screen, and are the angle parameters, Indicates the angle change in the horizontal direction. Indicates the angle change in the vertical direction. In this way, the spatial position of each point on the curved screen surface is described. The parameterized surface equation of the curved screen is transformed and analyzed to obtain a 4-dimensional system including translation component, rotation component and scaling component. The installation position and angle of the curved screen will be offset, so the surface equation is transformed spatially. The transformation process includes translation, rotation and scaling operations. These transformation operations are uniformly represented as a 4x4 transformation matrix. Assume that the translation component is , the rotation component is (rotation angle around an axis), the scaling component is , then the transformation matrix is ​​expressed as: ; in, are the scaling factors, is the translation amount. The matrix is ​​used to transform the surface equation to obtain the transformed surface coordinates. A nonlinear mapping function is constructed based on the parameterized surface equation of the curved screen, and the surface deformation correction term is introduced into the nonlinear mapping function to obtain a nonlinear correction matrix. When displayed on a curved screen, due to the curved nature of the display device, the rendered scene will be geometrically distorted, especially at the edge of the screen. In order to correct these deformations, a nonlinear mapping function is introduced, which makes necessary deformation adjustments to the geometric shape of the scene. The nonlinear correction matrix is ​​achieved by introducing a surface deformation correction term, which is dynamically adjusted according to the curvature and curvature. The nonlinear mapping matrix is ​​expressed as: ; in, is the curvature correction function, Represents the curvature value. This function adjusts the deformation according to the local curvature of the scene, usually increasing the deformation as the curvature increases, thereby correcting the distortion in the edge area. Perform matrix multiplication on the basic transformation matrix and the nonlinear correction matrix to obtain the initial mapping matrix. By multiplying the basic transformation matrix and the nonlinear correction matrix, an initial mapping matrix containing all transformations (translation, rotation, scaling, correction) is obtained: ; The initial mapping matrix contains the geometric transformation and surface correction information of the curved screen, and is the basic matrix used to map the scene data during the entire rendering process. The mapping parameter correction is calculated based on the scene frequency domain feature data to obtain the parameter adjustment matrix. Since the frequency domain characteristics of the scene (such as texture, lighting, etc.) behave differently in different curved areas, additional corrections are made based on the specific frequency domain characteristics of the scene. This process obtains the mapping parameter correction by analyzing the frequency domain feature data of the scene (such as features obtained through Laplace pyramid decomposition or wavelet transform). Assume that the mapping parameter correction is , then the parameter adjustment matrix is ​​expressed as: ; in, is the adjusted scaling factor, is the adjusted translation amount. These parameters are dynamically calculated according to the specific frequency domain characteristics of the scene. The initial mapping matrix and the parameter adjustment matrix are combined to obtain the final surface mapping matrix: ; The mapping matrix is ​​used to map all geometric objects in the 3D scene to the display area of ​​the curved screen, ensuring that the rendering result can accurately reflect the curvature characteristics of the curved screen and the spatial transformation of the scene. Through the mapping process, the 3D scene displayed by the curved screen obtains the correct geometric mapping, thus avoiding distortion caused by the curvature of the screen.

[0024] In this embodiment, after obtaining the surface mapping matrix and before performing spatial feature mapping on the 3D scene, it also includes: constructing a nonlinear affine model for the surface mapping matrix, decomposing the surface mapping matrix into a linear mapping part and a nonlinear disturbance part, and obtaining an initial affine model; designing a nonlinear disturbance observer based on the initial affine model, performing state estimation on the nonlinear disturbance part, and obtaining a disturbance state estimation value; constructing an integral sliding mode dynamic switching surface according to the disturbance state estimation value, establishing a disturbance compensation function through the dynamic change of the switching surface, and obtaining compensation parameters; performing homogeneous second-order quasi-continuous optimization on the surface mapping matrix based on the compensation parameters to eliminate the influence of nonlinear disturbances, and obtaining to the optimized surface mapping matrix; perform Lyapunov stability analysis on the optimized surface mapping matrix, construct stability constraints, and obtain stability parameters; adaptively adjust the curvature parameters of the optimized surface mapping matrix according to the stability parameters to obtain adjusted curvature parameters; use the adjusted curvature parameters to update the nonlinear mapping function of the optimized surface mapping matrix to obtain the compensated surface mapping matrix; perform curvature adaptive verification on the compensated surface mapping matrix to ensure that the mapping accuracy meets the preset threshold requirements, obtain the final surface mapping matrix, and use the final surface mapping matrix as the input of subsequent spatial feature mapping.

[0025] In one example, 3D scene spatial feature mapping is performed based on scene frequency domain feature data and a surface mapping matrix to obtain a spatial mapping feature map, including: The scene frequency domain feature data is input into the attention mechanism unit, and the spatial attention feature and channel attention feature are extracted through the 3×3 deformable convolution and 1×1 convolution in the attention mechanism unit respectively to obtain a dual attention feature map; Perform skip connection processing on the dual attention feature map, establish associations between features at different levels, obtain multi-level association features, and calculate the depth importance of the multi-level association features based on the scene depth information to obtain the depth weight coefficient; Perform weighted calculation on multi-level correlation features according to the depth weight coefficient to obtain a weighted feature map; Decompose the surface mapping matrix into a geometric transformation part and a nonlinear correction part, perform step-by-step mapping on the weighted feature map, and obtain mapping feature data; An adaptive bilinear interpolation operation based on curvature is performed on the mapping feature data to obtain a spatial mapping feature map.

[0026] In this example, the scene frequency domain feature data is input into the attention mechanism unit. The scene frequency domain feature data is obtained by frequency domain decomposition of the 3D scene, which contains different frequency components of the scene and reflects different spatial information. The attention mechanism automatically and dynamically adjusts its weights according to the characteristics of the input data, thereby strengthening the expression of important features and suppressing irrelevant features. Two convolution operations are used in the attention mechanism unit: 3×3 deformable convolution and 1×1 convolution. 3×3 deformable convolution is used to extract spatial attention features. Through the convolution operation, the network adaptively adjusts the shape of the convolution kernel at different positions and spatial scales to better capture the spatial information of the scene. The formula is as follows: ; in, represents the extracted spatial attention features, represents a 3×3 deformable convolution operation, is the input frequency domain feature data. 1×1 convolution is used to extract channel attention features. Different from spatial attention features, channel attention features focus on weighting between different feature channels and dynamically adjust according to the importance of different channels to the rendering results. The formula is as follows: ; in, represents the extracted channel attention features, yes Convolution operation. Through these two convolution operations, the dual attention feature maps of space and channel are obtained. The dual attention feature maps are processed by skip connection. Skip connection is a technique in deep learning that can effectively connect feature maps of different levels so that the network can use the correlation information of low-level and high-level features. The spatial attention feature map and the channel attention feature map are skipped to obtain multi-level correlation features. The skip connection processing is implemented by simple addition or splicing operations, and the formula is as follows: ; Through this operation, low-level and high-level features are effectively integrated, enhancing the ability of feature expression. Multi-level associated features are weighted based on the depth information of the scene. The depth information of the scene provides weights for different depth positions, which helps to determine the importance of features at different levels. The depth information is obtained by normalizing the depth value of each pixel to obtain the depth weight coefficient. Let the depth value be , and its normalized depth weight coefficient is expressed as: ; in, represents the depth weight coefficient, It is The depth value of the pixel, is the maximum depth value in the scene. Through this step, different weights are assigned to each pixel according to its depth to achieve depth importance calculation. Based on the depth weight coefficient, multi-level associated features are weighted to obtain a weighted feature map. The weighted calculation is achieved by applying the depth weight coefficient to the feature value of each pixel. The formula is as follows: ; in, is the weighted feature map, It is a multi-level feature map after skip connection. is the depth weight coefficient. Based on the geometric transformation and nonlinear correction part of the surface mapping matrix, the weighted feature map is mapped step by step. The geometric transformation part of the surface mapping matrix includes scaling, rotation and translation operations, while the nonlinear correction part fine-tunes the features by introducing curvature correction terms. The mapping operation is implemented by matrix multiplication or other transformation techniques to map the weighted feature map from 3D space to the display space of the curved screen. The mapping formula is expressed as: ; in, is the feature data after mapping, is the geometric transformation matrix. For the mapped feature data, an adaptive bilinear interpolation operation based on curvature is performed to optimize the mapping result. The curvature value is obtained by calculating the curvature of each point on the surface, which reflects the degree of curvature of the surface at that point. In order to cope with the curvature change of the curved screen, an adaptive bilinear interpolation method is used to dynamically adjust the interpolation strategy according to the curvature values ​​at different positions. The bilinear interpolation formula is expressed as: ; in, is the interpolated eigenvalue, and are the coordinates of the two pixels used for interpolation, is the interpolation weight, which depends on the curvature value. Through the above steps, the final spatial mapping feature map can accurately reflect the frequency domain characteristics of the scene, and optimize the geometric and nonlinear correction of the curved screen to ensure the realism and visual effect of the scene.

[0027] Among them, before the spatial mapping feature map is input into the multi-curvature convolution kernel, it also includes feature error correction of the spatial mapping feature map, specifically including: uniformly gridding the spatial mapping feature map, setting feature sampling points in each grid, constructing a feature sampling network, and obtaining a feature sampling point distribution map; extracting features from the spatial mapping feature map from different perspectives to obtain multiple groups of perspective feature data; calculating the feature mapping error for each sampling point in the feature sampling point distribution map to obtain a feature error distribution map; weighting the multiple groups of perspective feature data according to the feature error distribution map to obtain weighted perspective feature data; constructing a feature error correction model based on the weighted perspective feature data to obtain feature correction parameters; using the feature correction parameters to iteratively optimize the feature error distribution map to obtain optimized feature error data; performing error compensation on the spatial mapping feature map according to the optimized feature error data to obtain an error-corrected feature map; performing feature consistency evaluation on the error-corrected feature map to generate a corrected spatial mapping feature map.

[0028] In one example, the spatial mapping feature map is input into a multi-curvature convolution kernel for surface feature processing to obtain a surface enhancement feature map, including: According to the geometric characteristics of the curved screen, multi-curvature convolution kernels with curvature values ​​of k1, k2 and k3 are constructed. Each group of curvature convolution kernels in the multi-curvature convolution kernels contains two convolution layers and one batch normalization layer; The spatial mapping feature maps are input into the multi-curvature convolution kernel for feature calculation, and three sets of feature data with different curvature perspectives are obtained; The arc screen is divided into n equally spaced arc segments, and an independent feature enhancement parameter is assigned to each arc segment to obtain an arc segment parameter set; Construct n arcs into a graph structure, where each arc is a graph node and the relationship between adjacent arcs is a graph edge, to obtain an arc relationship graph; Perform graph convolution operations on the feature data and arc segment relationship graphs of three sets of different curvature perspectives to establish spatial dependency relationships between features and obtain spatial dependency features. Feature enhancement operation is performed according to the spatial dependency features and arc parameter set to obtain a surface enhancement feature map.

[0029] In this example, the curvature values ​​are constructed according to the geometric characteristics of the curved screen. , and The curvature of the curved screen is a key factor affecting the display effect, which determines the degree of curvature of each area on the screen. In order to adapt to areas with different curvatures, three convolution kernels with different curvature values ​​are designed, corresponding to different areas on the screen. For example, Indicates the curvature of the central area of ​​the screen, represents the curvature of the area close to the edge, Represents extreme curvature areas (such as the two ends of the screen). Each convolution kernel consists of two convolution layers and a batch normalization layer to better extract local features and improve the training stability of the model. Each convolution layer of the convolution kernel uses a different kernel size and stride to capture spatial information of different scales through local feature extraction. The batch normalization layer is used to normalize the output to reduce overfitting during training.

[0030] Conv2D with BatchNorm; in, It is the feature calculated by multi-curvature convolution kernel. Conv2D represents the convolution operation. is the input feature map. The convolution kernel corresponding to each curvature value performs convolution operations on the spatial mapping feature map respectively to obtain three sets of feature data with different curvature perspectives, namely , corresponding to the characteristics of different curvature areas. The screen is divided into multiple areas by equally dividing arc segments. Each area corresponds to an independent feature enhancement parameter, ensuring that the display effects of different areas can be independently optimized according to their curvature characteristics. The parameter set of each arc segment contains all the parameters required for feature enhancement, such as brightness, contrast, color enhancement, etc. of a specific area. Through this division, the rendering effect can be adjusted more flexibly, so that the details of different areas can be better presented.

[0031] ; in, Indicates arc segments, is the total number of arc segments. Each arc segment corresponds to a set of feature enhancement parameters ,Right now , these parameters are optimized independently for each arc segment. The arcs are constructed into a graph structure, where each arc is a graph node, and graph edges are established between adjacent arcs, thus obtaining an arc relationship graph, where each node represents an arc and the edge represents the relationship between adjacent arcs. This graph structure enables the system to capture the spatial dependencies between different arcs through a graph convolutional network (GCN) and process the features of each arc from different curvature perspectives.

[0032] ; in, is the arc segment relationship diagram, is the set of nodes in the graph, representing each arc segment, is the edge set in the graph, which represents the relationship between adjacent arcs. Through the graph convolution network, the spatial dependency features of each arc are calculated and the relationship between each arc is established. Relationship diagram of arc segment Perform graph convolution operations. Graph convolution is a neural network method for processing non-Euclidean data (such as graph data). Through graph convolution, the features of each arc segment are calculated under the structure of the arc segment relationship graph, and then the spatial dependency relationship between different arc segments is learned. The calculation formula of graph convolution is: ; in, is the output feature after graph convolution, It is The set of neighbor nodes of a node, is the adjacency matrix of the graph, representing the nodes and nodes The connection strength between is the convolution kernel weight, is the activation function. Through the graph convolution operation, the spatial relationship between different arcs is captured, and these relationships are integrated into the features of each arc to obtain spatial dependency features. , which contains the interactive information between different arcs in space. and segment parameter sets Perform feature enhancement operations to obtain a surface enhancement feature map. Through feature enhancement, the visual effect of each arc segment is adjusted to achieve more refined rendering. Feature enhancement is performed through weighted calculation, nonlinear transformation, etc., and the final surface enhancement feature map is obtained. Contains the final features after spatial dependency and feature enhancement: ; in, is the surface enhancement feature map, is a set of arc segment parameters. Through this step, a feature map optimized according to the curvature characteristics and spatial dependency of the curved screen is obtained.

[0033] In one example, the initial rendering parameter set is divided into a surface-sensitive parameter set and a surface-insensitive parameter set, and rendering parameter optimization is performed to obtain a target rendering parameter set, including: The initial rendering parameter set is divided into a surface-sensitive parameter set and a surface-insensitive parameter set. The surface-sensitive parameter set includes geometric transformation parameters and perspective projection parameters, and the surface-insensitive parameter set includes lighting parameters and material parameters. A surface-sensitive optimization function including a surface distortion measurement term and a feature preservation term is constructed for the surface-sensitive parameter set. Meanwhile, an optimization objective function based on mean square error is constructed for the surface-insensitive parameter set to obtain a surface-insensitive optimization function. Performing a gradient descent operation on the surface sensitive parameter set according to the surface sensitive optimization function, and dynamically adjusting the update step size through the curvature value to obtain an optimized surface sensitive parameter set; Performing Adam optimizer operation on the surface-insensitive parameter set according to the surface-insensitive optimization function to obtain an optimized surface-insensitive parameter set; The optimized surface-sensitive parameter set and the optimized surface-insensitive parameter set are combined to obtain a target rendering parameter set.

[0034] In this example, the initial rendering parameter set is divided into parameter types to divide the rendering parameters into a surface-sensitive parameter set and a surface-insensitive parameter set. The surface-sensitive parameter set contains geometric transformation parameters and perspective projection parameters related to the screen curvature, which have a strong influence on the rendering effect of the surface. Geometric transformation parameters include rotation matrix, translation matrix and scaling matrix, which are used to describe the transformation of scenes or objects on the surface. Perspective projection parameters include viewing distance, viewing angle, etc., which directly affect the spatial perspective effect of the scene. The surface-insensitive parameter set contains lighting parameters and material parameters. The lighting parameters are used to control the position, brightness and type of the light source, while the material parameters affect the reflection, refraction, texture and other characteristics of the object surface. Through this division, it is identified which parameters are sensitive to surface changes and which parameters have little effect on surface changes. After completing the parameter type division, optimization functions are constructed for the surface-sensitive parameter set and the surface-insensitive parameter set. For the surface-sensitive parameter set, an optimization function containing a surface distortion measurement item and a feature preservation item is constructed. During the optimization process, the surface distortion metric is used to measure the geometric distortion caused by surface transformation, while the feature preservation term ensures that the features of the original scene or object are preserved during the rendering process. For example, the surface distortion metric is expressed using the following formula: ; in, is the surface distortion measure, It is The transformation matrix of a point, is an ideal transformation matrix. By minimizing this metric, the geometric distortion in the surface transformation process is reduced. The feature preservation term is used to ensure that the geometric features of each object in the scene (such as edges, textures, etc.) are well preserved during the transformation process. The feature preservation term is implemented through a certain similarity measure (such as the structural similarity index). For the surface-insensitive parameter set, an optimization objective function based on the mean square error is constructed. The mean square error is one of the most commonly used optimization objective functions, which can effectively measure the difference between the rendered image and the target image. In the optimization process, the goal is to minimize the error between the rendering result and the real effect to achieve better lighting and material effects. The formula for the mean square error is: ; in, is the mean square error, It is The rendered image value of pixels, It is The target image value of pixels, is the total number of pixels in the image. Minimizing this loss function can optimize lighting and material effects and reduce errors in the rendering process. Optimize the surface-sensitive parameter set. Use the gradient descent algorithm to optimize these parameters. The gradient descent algorithm calculates the gradient of the loss function relative to the parameter and updates the parameters along the gradient direction to minimize the loss function. During the optimization process, the curvature value is introduced to dynamically adjust the update step size to avoid excessive parameter updates in areas with large curvature, resulting in unstable rendering effects. For example, suppose there is a surface-sensitive parameter set , the corresponding optimization function is , then the update rule is: ; in, is the learning rate (step size), is the gradient of the loss function. On this basis, the learning rate is adjusted according to the curvature value, and a smaller learning rate is used in areas with larger curvature to ensure the stability of parameter updates. For surface-insensitive parameter sets, the Adam optimizer is used for optimization. The Adam optimizer is an optimization algorithm that combines the momentum method and adaptive learning rate, which can converge to a better solution in a shorter time. By using the Adam optimizer, the lighting and material parameters can be adjusted efficiently to ensure that the details of the rendering effect are more realistic. The update rule of the Adam optimizer is: ; ; ; in, and are the first-order moment estimate and the second-order moment estimate of the gradient, and is the momentum parameter, is the learning rate, It is a constant to prevent division by zero errors. After gradient descent and iterative optimization by the Adam optimizer, the optimized surface-sensitive parameter set and surface-insensitive parameter set are obtained respectively. The two optimized parameter sets are merged to obtain the final target rendering parameter set.

[0035] In one example, 3D scene rendering is performed using a target rendering parameter set and a curved surface enhancement feature map to obtain a curved screen display image, including: Performing vertex transformation operations on vertex data of the 3D scene according to the geometric transformation parameters in the target rendering parameter set to obtain transformed vertex coordinates; Performing a projection matrix transformation on the transformed vertex coordinates according to the viewing angle projection parameters in the target rendering parameter set to obtain projection vertex data; Divide the projected vertex data into triangular patches, and perform curvature adaptive sampling on the triangular patches, where the sampling density is proportional to the curvature value, to obtain sampling point data; According to the lighting parameters and material parameters in the target rendering parameter set, the sampling point data is subjected to lighting shading calculation to obtain the initial pixel data; The curved surface enhancement feature map is subjected to feature injection operation with the initial pixel data to obtain enhanced pixel data, and curvature-based anti-aliasing processing is performed on the enhanced pixel data to obtain a curved screen display image.

[0036] In this example, the geometric transformation parameters in the target rendering parameter set include rotation, translation, and scaling operations, and the spatial position adjustment of the 3D scene vertices is achieved through the transformation matrix. Suppose there is a vertex data set in a 3D scene , where each vertex These are the spatial coordinates of the objects in the scene. According to the geometric transformation parameters in the target rendering parameter set, a 4×4 transformation matrix is ​​used To perform vertex transformation, get the new vertex coordinates , where the transformed coordinates are obtained through matrix operations: ; in, is the transformation matrix, which includes operations such as rotation, translation, and scaling. are the original vertex coordinates, is the transformed vertex coordinate. Through this step, the objects of the entire 3D scene are transformed to the new spatial position according to the geometric transformation requirements in the target rendering parameters. Use the perspective projection parameters in the target rendering parameter set to perform a projection matrix transformation on the transformed vertex coordinates. The projection matrix is ​​used to transform points in 3D space into points on the 2D image plane, including perspective projection and orthogonal projection. Assume that the transformed vertex coordinates are , the projection matrix Used to map these coordinates to the viewing projection plane to obtain the projected vertex data : ; in, It is the perspective projection matrix, which is adjusted according to the camera position, field of view angle and other parameters. are the transformed vertex coordinates, are the vertex coordinates after projection. The projection process is implemented through the perspective projection matrix. The viewing angle distortion is calculated according to parameters such as viewing distance and focal length to obtain the final 2D image coordinates. Triangle facet division is performed based on the projected vertex data. In 3D graphics processing, triangle facets are the most basic geometric units, defined by three vertices. For a complex 3D scene, multiple vertices are divided into multiple triangle facets according to certain rules, and each facet may have different sampling densities. In order to cope with the impact of the curvature of the curved screen, curvature adaptive sampling is adopted. Curvature adaptive sampling refers to dynamically adjusting the sampling density according to the curvature value of each triangle facet. Assume that the curvature value of a triangle facet is , the density of sampling points Proportional to the curvature value: ; in, is a constant coefficient, is the curvature of the patch, is the sampling density. The greater the curvature, the higher the sampling density, and vice versa. In this way, those areas with large curvature changes can be controlled more accurately to improve rendering accuracy, especially in complex curved areas. The lighting parameters and material parameters are applied to these sampling point data to calculate the lighting shading value of each sampling point. The basic goal of lighting shading calculation is to determine the color value of each sampling point based on factors such as light source, material properties, and viewing angle. Assume that the normal vector of the sampling point is , the intensity of the light source is , the observation angle is , the material reflection coefficient is , then according to the Phong reflection model, the lighting shading value Calculated by the following formula: ; in, is the light source direction vector, is the reflection direction vector, is the specular index, is the ambient light reflection coefficient, is the diffuse reflectance, is the specular reflection coefficient, and are the normal vector and the viewing direction vector of the sampling point, respectively. Through the above calculations, the initial pixel data is generated for each sampling point. The surface enhancement feature map is injected into the initial pixel data. The surface feature map previously extracted by deep learning or other methods is combined with the initial pixel data calculated during the rendering process to enhance the rendering effect. Assume that the surface enhancement feature map is , the initial pixel data is , combining the two through a weighted operation: ; in, is the weighting coefficient, which controls the degree of fusion between the initial pixel data and the enhanced feature map. is the enhanced pixel data. By injecting features, the quality of rendering is effectively improved, making the surface details and lighting effects of objects more realistic. Curvature-based anti-aliasing is performed on the enhanced pixel data to reduce the jagged phenomenon caused by insufficient pixel resolution in the image. In the curvature-based anti-aliasing process, the strength of anti-aliasing is dynamically adjusted according to the curvature value of each pixel. Assume that the curvature value of each pixel is , then the strength of anti-aliasing Proportional to the curvature value: ; in, is a constant coefficient, is the anti-aliasing strength, is the curvature value. In areas with greater curvature, the anti-aliasing processing intensity is greater, thereby effectively reducing the aliasing phenomenon generated in these areas. Through the above steps, a smooth and clear curved screen display image is finally obtained.

[0037] Reference Figure 2 This embodiment provides a 3D scene rendering device based on a curved screen, including: The frequency domain decomposition module 1 is used to perform frequency domain decomposition on the 3D scene data to obtain scene frequency domain feature data; Construction module 2, used to construct a surface mapping matrix according to the curvature parameters and radian parameters of the curved screen; Feature mapping module 3, used for performing 3D scene spatial feature mapping based on scene frequency domain feature data and surface mapping matrix to obtain a spatial mapping feature map; Feature processing module 4, used for inputting the spatial mapping feature map into the multi-curvature convolution kernel to perform surface feature processing to obtain a surface enhancement feature map; A rendering parameter optimization module 5 is used to divide the initial rendering parameter set into a surface-sensitive parameter set and a surface-insensitive parameter set, and perform rendering parameter optimization to obtain a target rendering parameter set; The 3D scene rendering module 6 is used to perform 3D scene rendering through a target rendering parameter set and a surface enhancement feature map to obtain a curved screen display image.

[0038] In this embodiment, for the specific implementation of each unit in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.

[0039] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0040] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0041] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0042] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A 3D scene rendering method based on a curved screen, characterized in that: The following steps are involved: Perform frequency domain decomposition on 3D scene data to obtain scene frequency domain feature data; Constructing a surface mapping matrix according to the curvature parameters and radian parameters of the curved screen; Performing 3D scene spatial feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map; Inputting the spatial mapping feature map into a multi-curvature convolution kernel to perform surface feature processing to obtain a surface enhancement feature map; The initial rendering parameter set is divided into a surface-sensitive parameter set and a surface-insensitive parameter set, and rendering parameter optimization is performed to obtain a target rendering parameter set; 3D scene rendering is performed using the target rendering parameter set and the curved surface enhancement feature map to obtain a curved screen display image.

2. The 3D scene rendering method based on a curved screen according to claim 1, characterized in that: The step of performing frequency domain decomposition on the 3D scene data to obtain scene frequency domain feature data includes: Performing a 5-layer Laplacian pyramid decomposition on the 3D scene data to obtain multi-level scene data; Performing wavelet transform processing on the multi-level scene data to obtain horizontal component data, vertical component data and diagonal component data of each level; Input the horizontal component data, the vertical component data, and the diagonal component data into a first feature extraction branch, and perform three consecutive feature extraction operations through a 3×3 convolution kernel and a ReLU activation function of the first feature extraction branch to obtain high-frequency feature data; Inputting the horizontal component data, the vertical component data and the diagonal component data into a second feature extraction branch, performing a feature extraction operation through a 7×7 convolution kernel and a linear activation function of the second feature extraction branch, and obtaining low-frequency feature data; The high-frequency feature data and the low-frequency feature data are used as scene frequency domain feature data.

3. The 3D scene rendering method based on a curved screen according to claim 1, characterized in that: The step of constructing a curved surface mapping matrix according to the curvature parameters and radian parameters of the curved screen includes: Constructing the parameterized surface equation of the curved screen according to the curvature parameter and the radian parameter of the curved screen; Performing transformation analysis on the curved screen parameterized surface equation to obtain a 4×4 basic transformation matrix including a translation component, a rotation component, and a scaling component; Constructing a nonlinear mapping function according to the curved screen parameterized surface equation, introducing the surface deformation correction term into the nonlinear mapping function, and obtaining a nonlinear correction matrix; Performing a matrix multiplication operation on the basic transformation matrix and the nonlinear correction matrix to obtain an initial mapping matrix; The mapping parameter correction amount is calculated according to the scene frequency domain feature data to obtain a parameter adjustment matrix, and the initial mapping matrix and the parameter adjustment matrix are combined to obtain a surface mapping matrix.

4. The 3D scene rendering method based on a curved screen according to claim 1, characterized in that: The performing 3D scene spatial feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map includes: The scene frequency domain feature data is input into the attention mechanism unit, and the spatial attention feature and the channel attention feature are extracted respectively through the 3×3 deformable convolution and the 1×1 convolution in the attention mechanism unit to obtain a dual attention feature map; Performing skip connection processing on the dual attention feature map, establishing associations between features at different levels, obtaining multi-level associated features, and performing depth importance calculation on the multi-level associated features based on scene depth information to obtain a depth weight coefficient; Performing weighted calculation on the multi-level associated features according to the depth weight coefficient to obtain a weighted feature map; Decomposing the surface mapping matrix into a geometric transformation part and a nonlinear correction part, mapping the weighted feature map step by step to obtain mapping feature data; An adaptive bilinear interpolation operation based on curvature is performed on the mapping feature data to obtain a spatial mapping feature map.

5. The 3D scene rendering method based on a curved screen according to claim 1, characterized in that: The step of inputting the spatial mapping feature map into a multi-curvature convolution kernel to perform surface feature processing to obtain a surface enhancement feature map comprises: Constructing multi-curvature convolution kernels with curvature values ​​of k1, k2 and k3 respectively according to the geometric characteristics of the curved screen, wherein each group of curvature convolution kernels in the multi-curvature convolution kernels includes two convolution layers and one batch normalization layer; Inputting the spatial mapping feature graphs into the multi-curvature convolution kernels for feature calculation respectively to obtain three sets of feature data at different curvature viewing angles; Dividing the arc screen into n equally spaced arc segments, and assigning independent feature enhancement parameters to each arc segment to obtain an arc segment parameter set; Construct n arcs into a graph structure, where each arc is a graph node and the relationship between adjacent arcs is a graph edge, to obtain an arc relationship graph; Performing graph convolution operation on the three groups of feature data of different curvature perspectives and the arc segment relationship graph to establish spatial dependency relationships between features and obtain spatial dependency features; A feature enhancement operation is performed according to the spatial dependency feature and the arc segment parameter set to obtain the surface enhancement feature map.

6. The 3D scene rendering method based on a curved screen according to claim 1, characterized in that: The initial rendering parameter set is divided into a surface-sensitive parameter set and a surface-insensitive parameter set, and rendering parameter optimization is performed to obtain a target rendering parameter set, including: Dividing the initial rendering parameter set into parameter types to obtain a surface-sensitive parameter set and a surface-insensitive parameter set, wherein the surface-sensitive parameter set includes geometric transformation parameters and perspective projection parameters, and the surface-insensitive parameter set includes lighting parameters and material parameters; Constructing a surface-sensitive optimization function including a surface distortion measurement term and a feature preservation term for the surface-sensitive parameter set, and at the same time, constructing an optimization objective function based on a mean square error for the surface-insensitive parameter set to obtain a surface-insensitive optimization function; Performing a gradient descent operation on the surface sensitive parameter set according to the surface sensitive optimization function, and dynamically adjusting the update step size through the curvature value to obtain an optimized surface sensitive parameter set; Performing an Adam optimizer operation on the surface-insensitive parameter set according to the surface-insensitive optimization function to obtain an optimized surface-insensitive parameter set; The optimized surface-sensitive parameter set and the optimized surface-insensitive parameter set are combined to obtain a target rendering parameter set.

7. The 3D scene rendering method based on a curved screen according to claim 1, characterized in that: The step of performing 3D scene rendering using the target rendering parameter set and the curved surface enhancement feature map to obtain a curved screen display image includes: Performing vertex transformation operations on vertex data of the 3D scene according to the geometric transformation parameters in the target rendering parameter set to obtain transformed vertex coordinates; Performing a projection matrix transformation on the transformed vertex coordinates according to the viewing angle projection parameters in the target rendering parameter set to obtain projection vertex data; Divide the projection vertex data into triangular patches, and perform curvature adaptive sampling on the triangular patches, wherein the sampling density is proportional to the curvature value, to obtain sampling point data; According to the illumination parameters and material parameters in the target rendering parameter set, performing illumination shading calculation on the sampling point data to obtain initial pixel data; The curved surface enhancement feature map and the initial pixel data are subjected to feature injection operation to obtain enhanced pixel data, and curvature-based anti-aliasing processing is performed on the enhanced pixel data to obtain a curved screen display image.

8. A 3D scene rendering device based on a curved screen, characterized in that: For implementing the steps of the method according to any one of claims 1 to 7, the device comprises: The frequency domain decomposition module is used to perform frequency domain decomposition on 3D scene data to obtain scene frequency domain feature data; A construction module, used for constructing a surface mapping matrix according to the curvature parameters and radian parameters of the curved screen; A feature mapping module, used to perform 3D scene spatial feature mapping based on the scene frequency domain feature data and the surface mapping matrix to obtain a spatial mapping feature map; A feature processing module, used for inputting the spatial mapping feature map into a multi-curvature convolution kernel to perform surface feature processing to obtain a surface enhancement feature map; A rendering parameter optimization module, used to divide the initial rendering parameter set into a surface-sensitive parameter set and a surface-insensitive parameter set, and perform rendering parameter optimization to obtain a target rendering parameter set; The 3D scene rendering module is used to perform 3D scene rendering through the target rendering parameter set and the curved surface enhancement feature map to obtain a curved screen display image.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images