A graphics rendering acceleration method based on deep learning

Abstract

The present invention relates to the field of three-dimensional graphics rendering technology, and specifically to a graphics rendering acceleration method based on deep learning. The method comprises the following steps: first, obtaining geometric information, material properties, lighting environment data and camera parameters of different scene types, and processing and generating a data set. A standard three-dimensional scene data set is constructed through steps such as data cleaning, normalization, and outlier detection. Then, a three-dimensional scene feature association learning model is used to extract similar and heterogeneous three-dimensional scene features, and a feature association matrix is established. Finally, the features are input into a three-dimensional graphics rendering acceleration model, and the rendering parameters are optimized through an adaptive adjustment mechanism to improve rendering efficiency and output rendered graphics. This method can significantly improve the rendering speed and quality of complex three-dimensional scenes.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional graphics rendering technology, and specifically to a graphics rendering acceleration method based on deep learning. Background Art

[0002] Graphics rendering is a key task in computer graphics, referring to the process of converting digital models or scene data into visual images. It transforms three-dimensional graphics information into two-dimensional images and involves calculating multiple visual effects, such as lighting, shadows, and textures, to enhance the image's realism and detail. Graphics rendering technology is widely used in fields such as gaming, film special effects, architectural design, and virtual reality, and is a core technology for generating realistic images and animations.

[0003] Traditional graphics rendering technologies are typically based on classic computer graphics theories and methods, relying on fixed rendering pipelines and algorithms to generate images, such as rasterization, lighting models, and texture mapping. However, these methods have limitations when dealing with complex scenes: The rendering effects struggle to achieve ideal realism, or the rendering process is computationally intensive and time-consuming, making it impossible to meet the requirements of real-time rendering.

[0004] With the continuous improvement of computer hardware, especially GPU performance, 3D graphics rendering technology is experiencing significant advancements, breaking through the limitations of traditional rendering methods. The emergence of new rendering acceleration technologies has achieved remarkable results in improving image realism and rendering efficiency, thus expanding the possibilities for producing high-quality 3D images. The application of rendering acceleration technology has shown tremendous potential and prospects, especially in fields such as gaming, film special effects, animation, and virtual reality.

[0005] Deep learning-based 3D graphics rendering acceleration technologies leverage neural networks to achieve efficient image generation and rendering by simulating real-world lighting, material properties, and interactions between objects. These technologies not only automatically optimize rendering details, improving rendering quality and real-time performance, but also effectively reduce the burden of complex calculations in traditional methods, providing users with a more immersive and smooth virtual experience. In this way, the improvement in image rendering quality not only manifests itself in improved visual effects, but also better adapts to the needs of different devices and environments, laying the foundation for the widespread application of virtual reality.

[0006] However, training deep learning models typically requires significant computing resources, especially for 3D graphics rendering, where large datasets and high-dimensional features make training extremely time-consuming. Even with powerful GPU acceleration, training complex rendering networks still takes a long time. While reducing the amount of training or the feature dimension can improve training speed to a certain extent, this can also affect rendering results, resulting in 3D graphics rendering quality failing to meet expected standards. Therefore, balancing training speed and rendering quality remains a key challenge in rendering acceleration technology.

[0007] To this end, the present invention proposes a graphics rendering acceleration method based on deep learning. Summary of the Invention

[0008] The purpose of the present invention is to provide a graphics rendering acceleration method based on deep learning, which constructs a standard three-dimensional scene dataset by collecting and processing geometric information, material properties, lighting environment data and camera parameters of different scene types. The dataset undergoes steps such as cleaning, normalization, and outlier detection, and generates similar and heterogeneous scene feature association matrices through a feature association learning model. Then, these data are input into a three-dimensional graphics rendering acceleration model, and the rendering parameters are optimized through an adaptive adjustment mechanism to achieve efficient rendering. During the rendering process, the node connectivity is calculated using temporal and spatial relationships, and a graph convolutional neural network is used to extract features and perform rendering optimization, ultimately generating expected rendering graphics, and adjusting the rendering strategy through real-time feedback to improve rendering effect and efficiency.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] Obtain geometric information, material properties, lighting environment data and camera parameters of three-dimensional scenes of different scene types; wherein the geometric information includes: vertex coordinates, normal vectors and texture coordinates of three-dimensional objects; the material properties include: surface reflectivity, glossiness and refractive index; the lighting environment data includes: light source type, light source position, lighting environment and shadow information.

[0011] Furthermore, the scene type, the geometric information, the material properties, the lighting environment data and the camera parameters are processed to construct a standard 3D scene dataset. The construction process of the standard 3D scene dataset includes:

[0012] Converting the scene type into a text feature representation to obtain a scene text feature vector;

[0013] Furthermore, data cleaning is performed on the geometric information, the material properties, the lighting environment data, and the camera parameters to obtain a first three-dimensional scene data set;

[0014] Further, normalizing the data in the first three-dimensional scene dataset to obtain a second three-dimensional scene dataset;

[0015] Further, performing outlier detection on the data in the second three-dimensional scene dataset to obtain a third three-dimensional scene dataset;

[0016] Further, merging the scene text feature vector into the third three-dimensional scene dataset to obtain a fourth three-dimensional scene dataset;

[0017] Further, compressing the data in the fourth three-dimensional scene data set to obtain a fifth three-dimensional scene data set;

[0018] Further, performing data conversion on the data in the fifth three-dimensional scene dataset to obtain the standard three-dimensional scene dataset;

[0019] Furthermore, the standard three-dimensional scene data set is input into a three-dimensional scene feature association learning model to obtain a similar scene feature association matrix and a heterogeneous scene feature association matrix;

[0020] The scene feature correlation matrix includes:

[0021] An input layer, configured to receive data from the standard three-dimensional scene dataset;

[0022] a classification layer, configured to classify the data received by the input layer according to the scene type to obtain a plurality of standard three-dimensional scene data subsets;

[0023] A feature extraction layer, configured to extract features from the data in the standard three-dimensional scene data subset;

[0024] The feature association learning layer is used to perform association learning on the extracted features, including: a similar 3D scene feature association learning module and a heterogeneous 3D scene feature association learning module;

[0025] The similar three-dimensional scene feature association learning module includes:

[0026] Acquire feature data of the same type of 3D scenes, which are recorded as a feature set of the same type of 3D scenes;

[0027] Establishing a multi-level graph structure based on the feature attributes in the feature set of the same type of three-dimensional scene;

[0028] Embedding the feature data into the nodes of the multi-level graph structure;

[0029] The node connectivity between the nodes is calculated based on the temporal relationship and the spatial geometric relationship; wherein the calculation formula of the node connectivity is:

[0030] R(i, j) = λ1·D Features (i,j)+λ2·TS(i,j)+λ3·Geo(,j);

[0031] Where R(i, j) is the node connectivity between node i and node j; D Features () is the function for calculating the characteristic distance between nodes; TS() is the function for calculating the temporal relationship between nodes; Geo() is the function for calculating the spatial relationship between nodes; λ1, λ2 and λ3 are the weight coefficients of characteristic distance, temporal relationship and spatial relationship respectively;

[0032] Assigning the node connectivity to the connection edge value of the multi-level graph structure, and connecting the nodes;

[0033] Inputting the connection relationship of the multi-level graph structure and the connection edge value into a graph convolutional neural network to obtain the feature association matrix of the same type of scene;

[0034] The heterogeneous three-dimensional scene feature association learning module includes:

[0035] Acquire feature data of different types of three-dimensional scenes, recorded as a heterogeneous three-dimensional scene feature set;

[0036] reducing the features in the heterogeneous three-dimensional scene feature set to the same dimension to obtain a heterogeneous three-dimensional scene reduced-dimensional feature set;

[0037] Clustering the features in the heterogeneous three-dimensional scene dimensionality reduction feature set to obtain multiple three-dimensional scene feature subsets;

[0038] locally enhancing the features in the three-dimensional scene feature subset according to the spatial relationship to obtain a three-dimensional scene enhanced feature subset;

[0039] Calculating the correlation degree of features in the 3D scene enhancement feature subset to obtain a plurality of heterogeneous 3D scene feature correlation vectors;

[0040] A plurality of the heterogeneous three-dimensional scene feature association vectors are integrated to obtain the heterogeneous scene feature association matrix.

[0041] An output layer is used to output the feature association matrix of the same scene and the feature association matrix of the different scene obtained by the feature association learning layer;

[0042] Furthermore, description information of the three-dimensional scene to be rendered is obtained;

[0043] Further, converting the description information into a text description vector;

[0044] Furthermore, the text description vector, the homogeneous scene feature association matrix, and the heterogeneous scene feature association matrix are input into a 3D graphics rendering acceleration model to obtain a rendered graphic; wherein the 3D graphics rendering acceleration model includes:

[0045] A receiving layer, configured to receive the text description vector, the homogeneous scene feature association matrix, and the heterogeneous scene feature association matrix;

[0046] An embedding layer, configured to convert the text description vector into a low-dimensional description feature vector;

[0047] A scene matching layer, configured to match a rendering scene according to the description feature vector;

[0048] A self-attention layer for focusing on features in the descriptive feature vector based on the rendered scene;

[0049] A rendering acceleration layer, configured to optimize the description features of interest based on the homogeneous scene feature association matrix and the heterogeneous scene feature association matrix;

[0050] A multi-level rendering layer, used for performing layered rendering according to the output of the rendering acceleration layer;

[0051] The graphics output layer is used to output the rendered graphics.

[0052] In addition, when the 3D graphics rendering acceleration model renders the 3D scene to be rendered, it also includes adjusting rendering parameters through an adaptive adjustment mechanism;

[0053] The adaptive adjustment mechanism includes:

[0054] Obtaining an expected rendering target of the three-dimensional scene to be rendered;

[0055] Furthermore, the rendering parameters are initialized;

[0056] Furthermore, the system performance of multiple rendering layers is monitored during rendering and real-time load indicators are calculated;

[0057] Furthermore, a rendering adjustment strategy is obtained according to the homogeneous scene feature correlation matrix and the heterogeneous scene feature correlation matrix;

[0058] Further, adjusting the initialization rendering parameters according to the rendering adjustment strategy;

[0059] Further, calculating a rendering error between the rendered graphics after the rendering parameters are adjusted and an expected rendering target;

[0060] Furthermore, rendering feedback is performed according to the real-time load indicator and the rendering error.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. By inputting a standard 3D scene dataset into a 3D scene feature association learning model, feature association matrices for similar and heterogeneous scenes are generated. The system first classifies the scene data and extracts features, which are then processed separately through the feature association learning modules for similar and heterogeneous scenes. The similar module establishes node connectivity based on a multi-level graph structure and utilizes a graph convolutional neural network to generate a similar association matrix. The heterogeneous module integrates feature association vectors through dimensionality reduction, clustering, and local enhancement to generate a heterogeneous association matrix. Finally, the output layer provides these two association matrices to the subsequent graphics rendering acceleration process. Through in-depth association learning of similar and heterogeneous scene features, the rendering process can more accurately process details and changes in the scene, thereby reducing the computational burden, rapidly generating high-quality 3D graphics, and improving rendering efficiency and image quality.

[0063] 2. The present invention proposes a three-dimensional graphics accelerated rendering model, which first obtains and converts the text description information of the three-dimensional scene into a vector form, combines the feature association matrix of the same and heterogeneous scenes, and inputs this information into a three-dimensional graphics rendering acceleration model. The model undergoes multiple levels of processing, including an embedding layer, a scene matching layer, a self-attention layer, a rendering acceleration layer, a multi-level rendering layer, and a rendering feedback layer, and finally outputs rendered graphics. The design of the rendering acceleration model can effectively improve the efficiency of the rendering process by optimizing the attention mechanism and scene matching of the feature vector. Specifically, the self-attention layer enhances the focus on important features, the rendering acceleration layer optimizes the rendering effect with the help of the feature association matrix, and the layered rendering and feedback mechanism further optimizes the rendering quality. The adaptive adjustment of the multi-level rendering and rendering feedback layers ensures higher quality three-dimensional graphics output, while accelerating the rendering process, thereby improving the rendering rate and quality.

[0064] 3. The present invention proposes an adaptive adjustment mechanism for adjusting the three-dimensional scene during rendering, by obtaining the expected rendering target of the three-dimensional scene to be rendered and initializing the rendering parameters. Next, the system performance of the multi-level rendering layer during the rendering process is monitored and the real-time load index is calculated. On this basis, the rendering adjustment strategy is generated through the feature association matrix of similar and heterogeneous scenes, and the initialized rendering parameters are optimized. After the rendering parameters are adjusted, the rendering error is calculated, and further adjustments are made based on the real-time load and error feedback to optimize the rendering effect. This adaptive adjustment mechanism brings benefits to the rendering rate and quality of three-dimensional graphics. By dynamically adjusting the rendering parameters, the system performance can be optimized and the rendering speed can be improved without sacrificing the rendering quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1A flowchart of a deep learning-based graphics rendering acceleration method provided by an embodiment of the present invention;

[0066] Figure 2 A diagram illustrating the graphics rendering process provided by an embodiment of the present invention;

[0067] Figure 3 This is a time comparison chart of graphics rendering before and after the experiment provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0069] With the increasing complexity of rendered scenes and higher requirements for rendering quality, traditional graphics rendering technologies face significant performance bottlenecks and computational resource consumption, especially when dealing with large-scale three-dimensional scenes and real-time rendering. These traditional methods often require a large amount of computing and memory resources, resulting in inefficient rendering processes. In addition, with the rapid advancement of hardware technology, users have higher expectations for the quality and speed of rendered images. In this context, researchers have combined deep learning with graphics rendering technology, using neural networks to achieve efficient image generation and rendering, thereby automatically optimizing rendering details and improving rendering quality and real-time performance.

[0070] However, training deep learning models typically requires enormous computing resources, especially in 3D graphics rendering, where large datasets and high-dimensional features make training extremely time-consuming. While powerful GPU acceleration can reduce training time to some extent, training complex rendering networks still requires a long time. Furthermore, while reducing the amount of training or the number of feature dimensions can increase training speed, this can compromise rendering quality, resulting in the final 3D graphics rendering quality falling short of expectations.

[0071] To address this issue, the present invention proposes a deep learning-based graphics rendering acceleration method, which aims to optimize the rendering efficiency of three-dimensional scenes. Next, the technical details of the present invention will be elaborated in detail through the following two embodiments.

[0072] Example 1:

[0073] In the embodiment of the present application, the process of rendering graphics by scene A is described. It is known that scene A is a complex urban landscape scene, including multiple objects such as multiple buildings, roads, and green belts. Scene A can be based on Figure 1Rendering is achieved by Figure 1 The implementation steps of the present invention include: S10. obtaining relevant information of three-dimensional scenes of different scene types; S20. processing the obtained information and constructing a standard three-dimensional scene data set; S30. learning the feature association matrix of similar scenes and the feature association matrix of heterogeneous scenes; S40. obtaining description information of the three-dimensional scene to be rendered; S50. converting the description information into a text description vector; S60. using a three-dimensional graphics rendering acceleration model to perform graphics rendering to obtain rendered graphics.

[0074] Furthermore, according to the above description:

[0075] According to the content of step S10, relevant information of three-dimensional scenes of different scene types is obtained, including: geometric information, material properties, lighting environment data and camera parameters; wherein, in the embodiment of the present application, relevant information can be obtained from different scenes such as cities, villages and natural landscapes; the corresponding information specifically includes: geometric information including: vertex coordinates, normal vectors and texture coordinates of three-dimensional objects; material properties including: surface reflectivity, glossiness and refractive index; lighting environment data including: light source type, light source position, lighting environment and shadow information.

[0076] In the embodiments of the present application, by acquiring various information from different scenarios and learning the correlation between features of similar scenarios and the feature differences between different scenarios, it is possible to optimize the rendering algorithm and improve the efficiency and quality of graphics rendering. Feature correlation learning can help the model identify the similarities and differences between different scenarios and accurately select the appropriate rendering strategy during rendering, thereby reducing the amount of calculation, accelerating the rendering speed, and avoiding excessive calculation of the rendering results.

[0077] Further, the acquired information is processed according to the content of S20, and the processed information is constructed into a standard three-dimensional scene dataset; wherein the specific processing process includes: converting the scene type into a text feature representation to obtain a scene text feature vector; performing data cleaning on the geometric information, material properties, lighting environment data and camera parameters to obtain a first three-dimensional scene dataset; normalizing the data in the first three-dimensional scene dataset to obtain a second three-dimensional scene dataset; performing outlier detection on the data in the second three-dimensional scene dataset to obtain a third three-dimensional scene dataset; merging the scene text feature vector into the third three-dimensional scene dataset to obtain a fourth three-dimensional scene dataset; compressing the data in the fourth three-dimensional scene dataset to obtain a fifth three-dimensional scene dataset; and performing data conversion on the data in the fifth three-dimensional scene dataset to obtain a standard three-dimensional scene dataset.

[0078] In the embodiments of this application, by performing feature correlation learning on the processed data, the relationships between different scene elements can be effectively identified and captured, optimizing the rendering model's processing strategy. Feature correlation learning helps the model adapt to the common features and differences of different scenes during the rendering process, thereby improving rendering efficiency and effectiveness. It can reduce redundant calculations, increase rendering speed, and ensure the accuracy and quality of the rendering results.

[0079] Further, according to the content of step S30, the standard three-dimensional scene data set is used to learn the feature correlation matrix of the same scene and the feature correlation matrix of the different scene; wherein, the feature correlation matrix of the same scene and the feature correlation matrix of the different scene are learned using the three-dimensional scene feature correlation learning model,

[0080] The 3D scene feature association learning model includes: input layer, classification layer, feature extraction layer, feature association learning layer and output layer;

[0081] The specific implementation process of the 3D scene feature association learning model includes:

[0082] Input the data from the standard 3D scene dataset through the input layer;

[0083] Furthermore, the input data is classified through a classification layer; wherein the classification layer divides the data into the same set according to the scene, thereby obtaining multiple standard three-dimensional scene data subsets;

[0084] Furthermore, the classified data is input into the feature extraction layer for feature extraction; the feature extraction layer uses an autoencoder to obtain features. The autoencoder can effectively reduce the dimensionality of high-dimensional data and learn feature representations, and is suitable for complex scene data.

[0085] Furthermore, the features extracted by the feature extraction layer are input into the feature association learning layer; wherein the feature association learning layer includes: a similar 3D scene feature association learning module and a heterogeneous 3D scene feature association learning module;

[0086] Among them, similar 3D scene feature association learning modules include:

[0087] Acquire feature data of the same type of 3D scenes, which are recorded as a feature set of the same type of 3D scenes;

[0088] Furthermore, a multi-level graph structure is established based on the feature attributes in the feature set of similar three-dimensional scenes;

[0089] Furthermore, the feature data is embedded into the nodes of the multi-level graph structure;

[0090] Furthermore, the node connectivity between the nodes is calculated based on the temporal relationship and the spatial geometric relationship; wherein the calculation formula of the node connectivity is:

[0091] R(i, j) = λ1·D Features (i,j)+λ2·TS(i,j)+λ3·Geo(i,j);

[0092] Where R(i, j) is the node connectivity between node i and node j; D Features () is the function for calculating the characteristic distance between nodes; TS() is the function for calculating the temporal relationship between nodes; Geo() is the function for calculating the spatial relationship between nodes; λ1, λ2 and λ3 are the weight coefficients of characteristic distance, temporal relationship and spatial relationship respectively;

[0093] Furthermore, assigning the node connectivity to the connection edge value of the multi-level graph structure, and connecting the nodes;

[0094] Furthermore, the connection relationship of the multi-level graph structure and the connection edge value are input into a graph convolutional neural network to obtain a feature association matrix of similar scenes;

[0095] In this application, the correlation between features of scenes of the same type is learned using a similar 3D scene feature association learning module, which can establish the intrinsic connection between different scene elements through a multi-level graph structure. By calculating the connectivity between nodes and combining temporal relationships and spatial geometric relationships, a scene feature association matrix with deep features is formed. By inputting these association matrices into the graph convolutional neural network, the potential patterns between the elements in the scene can be further refined, thereby providing accurate feature support for the rendering model.

[0096] The heterogeneous 3D scene feature association learning module includes:

[0097] Acquire feature data of different types of three-dimensional scenes, recorded as a heterogeneous three-dimensional scene feature set;

[0098] Furthermore, the features in the heterogeneous three-dimensional scene feature set are reduced in dimension to the same dimension to obtain a heterogeneous three-dimensional scene reduced-dimensional feature set;

[0099] Furthermore, the features in the heterogeneous 3D scene dimensionality reduction feature set are clustered to obtain multiple 3D scene feature subsets;

[0100] Furthermore, the features in the 3D scene feature subset are locally enhanced according to the spatial relationship to obtain a 3D scene enhanced feature subset; wherein, in the embodiment of the present application, the feature enhancement formula in the 3D scene feature subset is:

[0101]

[0102] in, Enhance feature i in the 3D scene feature subset; f i is the original feature i in the 3D scene feature subset; β is the parameter for adjusting the enhancement strength; d ij is the geometric distance between the original feature i and the original feature j in the 3D scene feature subset; d max is the maximum eigenvalue in the 3D scene feature subset; θ ij is the relative angle between the original feature i and the original feature j in the 3D scene feature subset;

[0103] Furthermore, the correlation degree of the features in the 3D scene enhancement feature subset is calculated to obtain a plurality of heterogeneous 3D scene feature correlation vectors;

[0104] Furthermore, multiple heterogeneous three-dimensional scene feature correlation vectors are integrated to obtain a heterogeneous scene feature correlation matrix;

[0105] In this application, a heterogeneous 3D scene feature association learning module is used to learn the correlations between features in different scene types. This feature association learning method can significantly improve rendering accuracy and detail. By learning and enhancing the spatial relationships between different objects in a 3D scene, the rendering system can achieve a higher sense of realism in details, reduce visual distortion, enhance the depth and layering of the image, and reduce the amount of parameter calculations required during graphics rendering.

[0106] Furthermore, the output layer is used to output the feature correlation matrix of the same scene and the feature correlation matrix of the different scene;

[0107] Refer to Table 1, which shows the changes in rendering efficiency and computational complexity after the introduction of the 3D scene feature association learning model;

[0108] Table 1 Performance comparison of the 3D scene feature association learning model

[0109]

[0110] Furthermore, according to the content of step S40, the description information of the 3D scene to be rendered (scene A) is obtained; wherein the description information includes: description, position, size, color and other information of all objects in the scene;

[0111] Furthermore, according to the content of step S50, the description information is converted into a text description vector; wherein, the text is converted into a vector representation in digital form through a natural language processing (NLP) model. The text description vector is a numerical representation used to represent the text content;

[0112] Further, the graphics rendering is performed using a 3D graphics rendering acceleration model according to the S60 content; see Figure 2 , to accelerate the rendering process of the model for 3D image rendering:

[0113] Input the text description vector, the feature correlation matrix of the same scene, and the feature correlation matrix of the different scene through the receiving layer;

[0114] Furthermore, the text description vector is converted into a low-dimensional description feature vector using an embedding layer;

[0115] In the specific 3D image rendering acceleration model, the text description vector is converted into a low-dimensional description feature vector through the embedding layer. Suppose we have a text description, such as: "blue road sign, position (0, 0, 0), size 50×30×75".

[0116] Step 1: Text input to the model

[0117] Text description: The text is preprocessed, usually by tokenization, to break it into words or subwords. For example, "blue road sign, position (0, 0, 0)" might be broken down into ["blue", "road sign", "position", "at", "(0", "0", "0)"].

[0118] Step 2: Word vector initialization

[0119] Word embedding initialization: Each word or subword is mapped to a dense vector of fixed dimension through the word embedding layer. For each word, the embedding layer assigns it a vector that is learned and can be continuously optimized during training.

[0120] For example, suppose “blue” is mapped to [0.2, 0.4, 0.1] and “street sign” is mapped to [0.3, 0.6, 0.2]. The dimensions of these vectors can be set via hyperparameters, and are typically chosen to be between tens and hundreds of dimensions.

[0121] Step 3: Combination of word embedding vectors

[0122] Merge word embeddings: Combine the embedding vectors of each word into a matrix and send it as input to the next layer of the neural network. The size of this matrix is vocabulary × embedding dimension, and each row represents the embedding vector of a word.

[0123] Step 4: Low-dimensional description feature vector

[0124] Converting embedding vectors into low-dimensional feature vectors: Typically, a text description vector contains multiple words or subwords. After obtaining the vector for each word through the embedding layer, the next step is to aggregate these word vectors into an overall description feature vector through pooling, averaging, and other methods.

[0125] For example, using the average pooling method, the vectors of all words are averaged to obtain a unified vector [0.25, 0.5, 0.15]. This is the low-dimensional feature vector of the text description.

[0126] Furthermore, the rendering scene is matched through the scene matching layer according to the description feature vector;

[0127] Furthermore, a self-attention layer is used to focus on the features in the description feature vector according to the rendering scene;

[0128] The self-attention layer performs weighted attention on the descriptive feature vector, emphasizing important features while ignoring less relevant information. It allows the model to adaptively assign different attention weights to different parts of the scene.

[0129] Self-attention layer implements the following functions:

[0130] (1) Weighted Attention: The self-attention mechanism allows the model to establish associations between each part of the description feature vector. For example, when describing a 3D scene, the model can focus on certain key features in the description (such as the color and shape of the object) and adjust the rendering process based on these features.

[0131] (2) Dynamic feature selection: The self-attention mechanism does not require pre-definition of which features are more important. It can dynamically adjust the degree of attention to features based on the specific needs of the scenario. This enables the model to flexibly process information based on the context when facing diverse scenarios.

[0132] Calculation process: In the self-attention mechanism, each input feature vector will calculate an attention score to determine the importance of the feature in the rendering process. The calculation process includes:

[0133] (1) Query, Key, Value: Each descriptive feature vector generates a Query vector, a Key vector, and a Value vector through different linear transformations.

[0134] (2) Attention score: Calculate the similarity between the query and the key (for example, by dot product calculation) to obtain the attention score.

[0135] (3) Weighted summation: Perform weighted summation on the Value vectors according to the scores to obtain the weighted output vector.

[0136] (4) Output: The output of the self-attention layer is a weighted version of each feature in the feature vector. These weighted features will be passed to the next layer (such as the embedding layer, scene rendering layer, etc.) for further processing and rendering.

[0137] In this embodiment, the scene matching layer and the self-attention layer are combined to enhance the connection between the descriptive feature vector and the rendered scene. Through these processing steps, the model can intelligently select an appropriate 3D scene based on the input scene description information and accurately render an image or 3D scene that is highly consistent with the description information through the self-attention mechanism.

[0138] Furthermore, the description features of interest are optimized in the rendering acceleration layer through the feature correlation matrix of the same scene and the feature correlation matrix of the different scene. The specific rendering acceleration formula of the rendering acceleration layer is:

[0139] V 优化 =ω·V+α·(M 同类 ·V)+β·(M 异类 V);

[0140] V 加速 =σ(V 优化 +b 加速 );

[0141] Among them, V 优化 is the optimized descriptive feature; ω is the weight of the descriptive feature vector; V is the descriptive feature of interest; M 同类 is the feature correlation matrix of similar scenes; M 异娄 is the correlation matrix of heterogeneous scene features; α and β are the influence of adjusting the homogeneous and heterogeneous features; V 加速 is the description feature of the optimized acceleration; σ() is the nonlinear activation function; b 加速 is the acceleration constant;

[0142] Furthermore, a multi-level rendering layer is used to perform layered rendering according to the output of the rendering acceleration layer; wherein, the multi-level rendering layer also includes dynamically adjusting the rendering parameters using an adaptive adjustment mechanism;

[0143] The adaptive adjustment mechanism includes:

[0144] Get the expected rendering target of the 3D scene to be rendered (such as target resolution, frame rate, lighting effects, etc.);

[0145] Furthermore, the rendering parameters P are initialized 初始 ;

[0146] Among them, the initial rendering parameters include: resolution R 初始 , light intensity L 初始 , detail level D 初始 and texture complexity C 初始 ;

[0147] Furthermore, the system performance of the multi-level rendering layer is monitored during rendering, and real-time load indicators are calculated, including: GPU usage U GPU 、CPU usage U CPU , memory usage U 内存 and rendering frame time T 帧 ;

[0148] Furthermore, a rendering adjustment strategy is obtained based on the feature correlation matrix of the same scene and the feature correlation matrix of the different scene;

[0149] S 调整 =G(M 同类 , M 异类 ,MLR(V 加速 ));

[0150] Among them, S 调整 is the rendering adjustment strategy; G() is the rendering adjustment strategy generation function; MLR() is the multi-level rendering layer function;

[0151] Further, adjusting the initialization rendering parameters according to the rendering adjustment strategy;

[0152] P 更新 =P 初始 +ΔP;

[0153] Among them, P 更新 is to update the rendering parameters; ΔP is the adjustment amount; specifically:

[0154] Resolution adjustment: R 更新 =R 初始 -κ R ·(U GPU -U 目标GPU );

[0155] Light intensity adjustment: L 更新 =L 初始 ·(1-κ L ΔT 帧 );

[0156] Level of Detail Adjustment: D 更新 =D 初始 -κ D ΔU 内存 ;

[0157] Texture complexity adjustment: C 更新 =C 初始 ·(1-κ T ΔU GPU );

[0158] Among them, R 更新 is the resolution adjustment value; L 更新D is the light intensity adjustment value; 更新 Adjust the value for the level of detail; C 更新 is the texture complexity adjustment value; κ R is the resolution adjustment coefficient; κ L is the light intensity adjustment coefficient; κ D is the detail level adjustment coefficient; κ T is the texture complexity adjustment coefficient; U 目标GPU is the target GPU utilization; ΔT 帧 is the change in rendering frame time; ΔU 内存 is the change in memory usage; ΔU GPU is the change in GPU usage;

[0159] Further, the rendering error between the rendered graphics after the rendering parameters are adjusted and the expected rendering target is calculated;

[0160] E 渲染 =||O 当前 -O 目标 ||2;

[0161] Among them, E 渲染 is the rendering error; O 当前 Output for current rendering; O 目标 For the intended rendering target;

[0162] Furthermore, rendering feedback is provided based on real-time load indicators and rendering errors;

[0163] F 反馈 =g(U GPU , U CPU , U 内存 , T 帧 , E 渲染 );

[0164] Among them, F 反馈 is the feedback mechanism; g() is the feedback function;

[0165] See Table 2, which shows the performance after optimization by the adaptive adjustment mechanism.

[0166] Table 2 Performance comparison after optimization of adaptive adjustment mechanism

[0167]

[0168] In an embodiment of the present application, an adaptive adjustment mechanism is used to adaptively adjust the rendering parameters during the rendering process. The mechanism dynamically optimizes the rendering parameters based on the real-time monitoring of system performance and rendering errors. This process automatically adjusts the rendering parameters by calculating the feature association matrix of similar and heterogeneous scenes to ensure that the rendering results are close to the expected target and maintain a reasonable level of system load. Through continuous feedback and adjustment, the mechanism can flexibly allocate resources and optimize performance under different rendering requirements. The advantage of this adaptive adjustment mechanism is that it can improve the efficiency and accuracy of rendering and ensure the balance of the rendering process in different scenarios. Through real-time monitoring and adjustment, the system can avoid excessive calculation and waste of resources while ensuring image quality, thereby improving rendering speed, reducing latency, and reducing hardware load, ultimately providing a high-quality, low-latency graphics rendering experience. This mechanism makes three-dimensional graphics rendering more adaptable and intelligent in practical applications, especially in complex dynamic scenes, and can quickly respond to changes and ensure stable output quality.

[0169] Furthermore, the graphics output layer renders the graphics for output; see Figure 3 ,exist Figure 3 A comparison of graphics rendering time when the present invention is adopted and when the present invention is not adopted is given in FIG.

[0170] The present application embodiment uses a three-dimensional graphics rendering acceleration model for graphics rendering. By receiving text description vectors and combining them with the optimization of feature matrices for similar and heterogeneous scenes, a multi-level feature extraction and attention mechanism is used to achieve efficient acceleration and refined processing of the rendering process. This model greatly improves the efficiency and quality of graphics rendering through the optimization of the self-attention layer and the rendering acceleration layer. In particular, it can significantly reduce the consumption of computing resources in the rendering of complex scenes while maintaining high-precision graphics output, ultimately achieving a smoother user experience and faster rendering time.

[0171] In the embodiment of the present application, the method of the present invention is used to realize the graphics rendering of scene A; specifically, by converting the geometric information, material properties, lighting environment data and camera parameters of scene A into a standard three-dimensional scene data set, and combining the feature association matrix of similar and heterogeneous scenes, a three-dimensional graphics rendering acceleration model is used for rendering. At the same time, the rendering parameters are dynamically optimized using an adaptive adjustment mechanism to ensure that the rendering quality and performance are optimal. This method not only greatly improves the rendering speed and shortens the rendering time, but also ensures high-quality output of graphics through precise parameter adjustment and multi-level rendering level optimization. In particular, it can effectively reduce the system burden, improve rendering efficiency, and meet the needs of high-performance computing in complex scenes.

[0172] Example 2:

[0173] In Example 1, efficient and high-quality graphics rendering is achieved for scene A. To illustrate again that the present invention is applicable in different scenarios, the following examples will be used to illustrate. In this example, graphics rendering is performed for scene B (forest scene), as follows:

[0174] Obtain geometric information, material properties, lighting environment data and camera parameters of three-dimensional scenes of different scene types;

[0175] Furthermore, the scene type, geometric information, material properties, lighting environment data and camera parameters are processed and a standard three-dimensional scene dataset is constructed.

[0176] Furthermore, the standard 3D scene dataset is input into the 3D scene feature association learning model to obtain the feature association matrix of the same type of scenes and the feature association matrix of the different types of scenes;

[0177] Furthermore, description information of the 3D scene to be rendered (scene B) is obtained;

[0178] Furthermore, the description information is converted into a text description vector;

[0179] Furthermore, the text description vector, the homogeneous scene feature association matrix, and the heterogeneous scene feature association matrix are input into a 3D graphics rendering acceleration model to obtain a rendered graphic. The 3D graphics rendering acceleration model includes:

[0180] The receiving layer is used to receive the text description vector, the feature correlation matrix of the same scene, and the feature correlation matrix of the different scene;

[0181] Embedding layer, used to convert the text description vector into a low-dimensional description feature vector;

[0182] The scene matching layer is used to match the rendering scene according to the description vector;

[0183] A self-attention layer that focuses on features in the description feature vector based on the rendered scene;

[0184] The rendering acceleration layer is used to optimize the description features of interest based on the feature correlation matrix of the same scene and the feature correlation matrix of the different scene;

[0185] Multi-level rendering layer, used for layered rendering based on the output of the rendering acceleration layer; rendering parameters are adjusted in the multi-level rendering layer through an adaptive adjustment mechanism;

[0186] The adaptive adjustment mechanism includes:

[0187] Obtain the expected rendering target of the 3D scene to be rendered;

[0188] Furthermore, the rendering parameters are initialized;

[0189] Furthermore, the system performance of multiple rendering layers is monitored during rendering and real-time load indicators are calculated;

[0190] Furthermore, a rendering adjustment strategy is obtained based on the feature correlation matrix of the same scene and the feature correlation matrix of the different scene;

[0191] Further, adjusting the initialization rendering parameters according to the rendering adjustment strategy;

[0192] Further, the rendering error between the rendered graphics after the rendering parameters are adjusted and the expected rendering target is calculated;

[0193] Furthermore, rendering feedback is provided based on real-time load indicators and rendering errors;

[0194] Graphics output layer, used to output rendered graphics.

[0195] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A graphics rendering acceleration method based on deep learning, characterized in that: include: Obtain geometric information, material properties, lighting environment data and camera parameters of three-dimensional scenes of different scene types; Processing the scene type, the geometric information, the material properties, the lighting environment data, and the camera parameters, and constructing a standard three-dimensional scene dataset; Inputting the standard three-dimensional scene data set into a three-dimensional scene feature association learning model to obtain a similar scene feature association matrix and a heterogeneous scene feature association matrix; The three-dimensional scene feature association learning model includes: An input layer, configured to receive data from the standard three-dimensional scene dataset; a classification layer, configured to classify the data received by the input layer according to the scene type to obtain a plurality of standard three-dimensional scene data subsets; A feature extraction layer, configured to extract features from the data in the standard three-dimensional scene data subset; The feature association learning layer is used to perform association learning on the extracted features, including: a similar 3D scene feature association learning module and a heterogeneous 3D scene feature association learning module; the similar 3D scene feature association learning module includes: Acquire feature data of the same type of 3D scenes, which are recorded as a feature set of the same type of 3D scenes; Establishing a multi-level graph structure based on the feature attributes in the feature set of the same type of three-dimensional scene; Embedding the feature data into the nodes of the multi-level graph structure; The node connectivity between the nodes is calculated based on the temporal relationship and the spatial geometric relationship; wherein the calculation formula of the node connectivity is: ； in, For nodes With node Node connectivity; Calculate the characteristic distance function between nodes; Compute functions for temporal relationships between nodes; Compute functions for spatial relationships between nodes; 、 and are the weight coefficients of feature distance, temporal relationship and spatial relationship respectively; Assigning the node connectivity to the connection edge value of the multi-level graph structure, and connecting the nodes; Inputting the connection relationship of the multi-level graph structure and the connection edge value into a graph convolutional neural network to obtain the feature association matrix of the same type of scene; An output layer is used to output the feature association matrix of the same scene and the feature association matrix of the different scene obtained by the feature association learning layer; Get the description information of the 3D scene to be rendered; Converting the description information into a text description vector; The text description vector, the homogeneous scene feature association matrix, and the heterogeneous scene feature association matrix are input into a three-dimensional graphics rendering acceleration model to obtain a rendered graphic.

2. The method for accelerating graphics rendering based on deep learning according to claim 1, wherein: The geometric information includes: vertex coordinates, normal vectors and texture coordinates of the three-dimensional object; the material properties include: surface reflectivity, glossiness and refractive index; the lighting environment data includes: light source type, light source position, lighting environment and shadow information.

3. The method for accelerating graphics rendering based on deep learning according to claim 1, wherein: The process of constructing the standard 3D scene dataset includes: Converting the scene type into a text feature representation to obtain a scene text feature vector; Performing data cleaning on the geometric information, the material properties, the lighting environment data, and the camera parameters to obtain a first three-dimensional scene data set; Normalizing the data in the first three-dimensional scene dataset to obtain a second three-dimensional scene dataset; performing outlier detection on the data in the second three-dimensional scene dataset to obtain a third three-dimensional scene dataset; Merging the scene text feature vector into the third three-dimensional scene dataset to obtain a fourth three-dimensional scene dataset; compressing the data in the fourth three-dimensional scene dataset to obtain a fifth three-dimensional scene dataset; Data conversion is performed on the data in the fifth three-dimensional scene dataset to obtain the standard three-dimensional scene dataset.

4. The method for accelerating graphics rendering based on deep learning according to claim 1, wherein: The heterogeneous three-dimensional scene feature association learning module includes: Acquire feature data of different types of three-dimensional scenes, recorded as a heterogeneous three-dimensional scene feature set; reducing the features in the heterogeneous three-dimensional scene feature set to the same dimension to obtain a heterogeneous three-dimensional scene reduced-dimensional feature set; Clustering the features in the heterogeneous three-dimensional scene dimensionality reduction feature set to obtain multiple three-dimensional scene feature subsets; locally enhancing the features in the three-dimensional scene feature subset according to the spatial relationship to obtain a three-dimensional scene enhanced feature subset; Calculating the correlation degree of features in the 3D scene enhancement feature subset to obtain a plurality of heterogeneous 3D scene feature correlation vectors; A plurality of the heterogeneous three-dimensional scene feature association vectors are integrated to obtain the heterogeneous scene feature association matrix.

5. The method for accelerating graphics rendering based on deep learning according to claim 1, wherein: The three-dimensional graphics rendering acceleration model includes: A receiving layer, configured to receive the text description vector, the homogeneous scene feature association matrix, and the heterogeneous scene feature association matrix; An embedding layer, configured to convert the text description vector into a low-dimensional description feature vector; A scene matching layer, configured to match a rendering scene according to the description feature vector; A self-attention layer for focusing on features in the descriptive feature vector based on the rendered scene; A rendering acceleration layer, configured to optimize the description features of interest based on the homogeneous scene feature association matrix and the heterogeneous scene feature association matrix; A multi-level rendering layer, used for performing layered rendering according to the output of the rendering acceleration layer; The graphics output layer is used to output the rendered graphics.

6. The method for accelerating graphics rendering based on deep learning according to claim 1, wherein: When the 3D graphics rendering acceleration model renders the 3D scene to be rendered, the method further includes adjusting rendering parameters through an adaptive adjustment mechanism; Among them, the adaptive adjustment mechanism includes: obtaining the expected rendering target of the three-dimensional scene to be rendered; initializing rendering parameters; monitoring the system performance of multiple rendering layers during rendering and calculating real-time load indicators; obtaining a rendering adjustment strategy based on the feature association matrix of the same scene and the feature association matrix of the heterogeneous scene; adjusting the rendering parameters of the initialized rendering parameters according to the rendering adjustment strategy; calculating the rendering error between the rendered graphics and the expected rendering target after the rendering parameters are adjusted; and providing rendering feedback based on the real-time load indicator and the rendering error.

Images