Graph rendering acceleration method based on deep learning

Abstract

The invention relates to the technical field of three-dimensional graphic rendering, in particular to a graphic rendering acceleration method based on deep learning. Comprising the following steps: firstly, acquiring geometric information, material characteristics, illumination environment data and camera parameters of different scene types, and processing to generate a data set; a standard three-dimensional scene data set is constructed through the steps of data cleaning, normalization, abnormal value detection and the like. Then, utilizing a three-dimensional scene feature association learning model to extract similar and heterogeneous three-dimensional scene features, and establishing a feature association matrix; and finally, inputting to a three-dimensional graph rendering acceleration model, optimizing rendering parameters through a self-adaptive adjustment mechanism, improving the rendering efficiency, and outputting a rendered graph. According to the method, the rendering speed and quality of the complex three-dimensional scene can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional graphics rendering technology, and specifically to a graphics rendering acceleration method based on deep learning. Background Art

[0002] Graphics rendering is a key task in computer graphics, which refers to the process of converting digital models or scene data into visual images. It converts three-dimensional graphic information into two-dimensional images, and involves the calculation of multiple visual effects such as lighting, shadows, textures, etc., to improve the realism and detail of the image. Graphics rendering technology is widely used in many fields such as games, movie special effects, architectural design, virtual reality, etc., and is the core technology for generating realistic images and animations.

[0003] Traditional graphics rendering technology is usually based on classic computer graphics theories and methods, relying on fixed rendering pipelines and algorithms to generate images, such as rasterization, lighting models, and texture mapping. However, these methods have certain limitations when dealing with complex scenes: the rendering effect is difficult to achieve the ideal sense of reality, or the rendering process is computationally intensive and time-consuming, which cannot meet the needs of real-time rendering.

[0004] With the continuous improvement of computer hardware, especially GPU performance, 3D graphics rendering technology is experiencing significant progress, breaking through the limitations of traditional rendering methods. The emergence of new rendering acceleration technology has achieved remarkable results in improving image realism and rendering efficiency, which provides greater possibilities for the generation of high-quality 3D images. Especially in the fields of games, movie special effects, animation and virtual reality, the application of rendering acceleration technology has shown great potential and prospects.

[0005] Deep learning-based 3D graphics rendering acceleration technologies use neural networks to achieve efficient image generation and rendering by simulating real-world lighting, material properties, and interactions between objects. These technologies can not only automatically optimize rendering details, improve rendering quality and real-time performance, but also effectively reduce the burden of complex calculations in traditional methods, thereby providing users with a more immersive and smooth virtual experience. In this way, the improvement in image rendering quality is not only reflected in improved visual effects, but also better adapts to the needs of different devices and environments, thus laying the foundation for the widespread application of virtual reality.

[0006] However, the training process of deep learning models usually requires a lot of computing resources. Especially in 3D graphics rendering, large-scale data sets and high-dimensional features make the training process very time-consuming. Even with the help of powerful GPU acceleration, training complex rendering networks still takes a long time. Although the training speed can be increased to a certain extent by reducing the amount of training or reducing the feature dimension, this may also affect the rendering effect, resulting in the 3D graphics rendering quality failing to meet the expected standards. Therefore, how to balance the training speed and rendering quality is still an important challenge in rendering acceleration technology.

[0007] To this end, the present invention proposes a graphics rendering acceleration method based on deep learning. Summary of the invention

[0008] The object of the present invention is to provide a graphics rendering acceleration method based on deep learning, which constructs a standard three-dimensional scene data set by collecting and processing geometric information, material properties, lighting environment data and camera parameters of different scene types. The data set undergoes steps such as cleaning, normalization, and outlier detection, and generates similar and heterogeneous scene feature association matrices through a feature association learning model. Then, these data are input into a three-dimensional graphics rendering acceleration model, and the rendering parameters are optimized through an adaptive adjustment mechanism to achieve efficient rendering. During the rendering process, the node connectivity is calculated using temporal and spatial relationships, and a graph convolutional neural network is used to extract features and perform rendering optimization, ultimately generating expected rendering graphics, and adjusting the rendering strategy through real-time feedback to improve rendering effect and efficiency.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] Obtain geometric information, material properties, lighting environment data and camera parameters of three-dimensional scenes of different scene types; wherein the geometric information includes: vertex coordinates, normal vectors and texture coordinates of three-dimensional objects; the material properties include: surface reflectivity, glossiness and refractive index; the lighting environment data includes: light source type, light source position, lighting environment and shadow information.

[0011] Furthermore, the scene type, the geometric information, the material characteristics, the lighting environment data and the camera parameters are processed to construct a standard three-dimensional scene dataset. The construction process of the standard three-dimensional scene dataset includes:

[0012] Convert the scene type into a text feature representation to obtain a scene text feature vector;

[0013] Further, the geometric information, the material characteristics, the lighting environment data and the camera parameters are cleaned to obtain a first three-dimensional scene data set;

[0014] Further, normalizing the data in the first three-dimensional scene data set to obtain a second three-dimensional scene data set;

[0015] Further, performing outlier detection on the data in the second three-dimensional scene data set to obtain a third three-dimensional scene data set;

[0016] Further, the scene text feature vector is merged into the third three-dimensional scene data set to obtain a fourth three-dimensional scene data set;

[0017] Further, compressing the data in the fourth three-dimensional scene data set to obtain a fifth three-dimensional scene data set;

[0018] Further, data conversion is performed on the data in the fifth three-dimensional scene data set to obtain the standard three-dimensional scene data set;

[0019] Further, the standard three-dimensional scene data set is input into a three-dimensional scene feature association learning model to obtain a similar scene feature association matrix and a heterogeneous scene feature association matrix;

[0020] Wherein, the scene feature association matrix includes:

[0021] An input layer, used for receiving data in the standard three-dimensional scene dataset;

[0022] A classification layer, used for classifying the data received by the input layer according to the scene type to obtain a plurality of standard three-dimensional scene data subsets;

[0023] A feature extraction layer, used for extracting features from the data in the standard three-dimensional scene data subset;

[0024] The feature association learning layer is used to perform association learning on the extracted features, including: a homogeneous 3D scene feature association learning module and a heterogeneous 3D scene feature association learning module;

[0025] The similar three-dimensional scene feature association learning module includes:

[0026] Acquire feature data of the same type of three-dimensional scenes, recorded as a feature set of the same type of three-dimensional scenes;

[0027] Establishing a multi-level graph structure according to the feature attributes in the feature set of the same type of three-dimensional scene;

[0028] embedding the feature data into the nodes of the multi-level graph structure;

[0029] The node connectivity between the nodes is calculated according to the temporal relationship and the spatial geometric relationship; wherein the calculation formula of the node connectivity is:

[0030] R(i, j) = λ1·D Features (i,j)+λ2·TS(i,j)+λ3·Geo(,j);

[0031] Where R(i, j) is the node connectivity between node i and node j; D Features () is the characteristic distance calculation function between nodes; TS() is the temporal relationship calculation function between nodes; Geo() is the spatial relationship calculation function between nodes; λ1, λ2 and λ3 are the weight coefficients of characteristic distance, temporal relationship and spatial relationship respectively;

[0032] Assigning the node connectivity to the connection edge value of the multi-level graph structure, and connecting the nodes;

[0033] Inputting the connection relationship of the multi-level graph structure and the connection edge value into a graph convolutional neural network to obtain the feature association matrix of the same type of scene;

[0034] The heterogeneous three-dimensional scene feature association learning module includes:

[0035] Acquire feature data of different types of three-dimensional scenes, recorded as a heterogeneous three-dimensional scene feature set;

[0036] The features in the heterogeneous three-dimensional scene feature set are reduced in dimension to the same dimension to obtain a heterogeneous three-dimensional scene reduced-dimensional feature set;

[0037] Clustering the features in the heterogeneous three-dimensional scene dimensionality reduction feature set to obtain multiple three-dimensional scene feature subsets;

[0038] Locally enhancing the features in the three-dimensional scene feature subset according to the spatial relationship to obtain a three-dimensional scene enhanced feature subset;

[0039] Calculating the correlation degree of features in the three-dimensional scene enhancement feature subset to obtain a plurality of heterogeneous three-dimensional scene feature correlation vectors;

[0040] A plurality of the heterogeneous three-dimensional scene feature association vectors are integrated to obtain the heterogeneous scene feature association matrix.

[0041] An output layer, used to output the feature association matrix of the same scene and the feature association matrix of the different scene obtained by the feature association learning layer;

[0042] Further, obtaining description information of the three-dimensional scene to be rendered;

[0043] Further, converting the description information into a text description vector;

[0044] Furthermore, the text description vector, the homogeneous scene feature association matrix and the heterogeneous scene feature association matrix are input into a three-dimensional graphics rendering acceleration model to obtain a rendered graphic; wherein the three-dimensional graphics rendering acceleration model includes:

[0045] A receiving layer, used for receiving the text description vector, the homogeneous scene feature association matrix and the heterogeneous scene feature association matrix;

[0046] An embedding layer, used to convert the text description vector into a low-dimensional description feature vector;

[0047] A scene matching layer, used for matching the rendering scene according to the description feature vector;

[0048] A self-attention layer for focusing on features in the descriptive feature vector according to the rendering scene;

[0049] A rendering acceleration layer, used for optimizing the description features of interest according to the homogeneous scene feature association matrix and the heterogeneous scene feature association matrix;

[0050] A multi-level rendering layer, used for performing layered rendering according to the output of the rendering acceleration layer;

[0051] The graphics output layer is used to output the rendered graphics.

[0052] In addition, when the 3D graphics rendering acceleration model renders the 3D scene to be rendered, it also includes adjusting rendering parameters through an adaptive adjustment mechanism;

[0053] The adaptive adjustment mechanism includes:

[0054] Obtaining an expected rendering target of the three-dimensional scene to be rendered;

[0055] Further, initializing rendering parameters;

[0056] Furthermore, the system performance of multiple rendering layers is monitored during rendering, and real-time load indicators are calculated;

[0057] Further, a rendering adjustment strategy is obtained according to the homogeneous scene feature association matrix and the heterogeneous scene feature association matrix;

[0058] Further, adjusting the initialization rendering parameters according to the rendering adjustment strategy;

[0059] Further, calculating a rendering error between the rendered graphics and an expected rendering target after the rendering parameters are adjusted;

[0060] Furthermore, rendering feedback is performed according to the real-time load indicator and the rendering error.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. Generate similar and heterogeneous scene feature association matrices by inputting the standard 3D scene data set into the 3D scene feature association learning model. The system first classifies the scene data and extracts features, and processes them separately through the feature association learning modules of similar and heterogeneous scenes. The similar module establishes node connectivity based on the multi-level graph structure and uses the graph convolutional neural network to generate a similar association matrix; the heterogeneous module integrates feature association vectors to generate a heterogeneous association matrix through dimensionality reduction, clustering and local enhancement. Finally, the output layer provides these two association matrices to the subsequent graphics rendering acceleration process. Through in-depth association learning of similar and heterogeneous scene features, the rendering process can more accurately process the details and changes in the scene, thereby reducing the computational burden, quickly generating high-quality 3D graphics, and improving rendering efficiency and image quality.

[0063] 2. The present invention proposes a three-dimensional graphics accelerated rendering model, which first obtains and converts the text description information of the three-dimensional scene into a vector form, combines the feature association matrix of the same and heterogeneous scenes, and inputs this information into a three-dimensional graphics rendering acceleration model. The model undergoes multiple levels of processing, including an embedding layer, a scene matching layer, a self-attention layer, a rendering acceleration layer, a multi-level rendering layer, and a rendering feedback layer, and finally outputs a rendered graphic. The design of the rendering acceleration model can effectively improve the efficiency of the rendering process by optimizing the attention mechanism and scene matching of the feature vector. Specifically, the self-attention layer enhances the focus on important features, the rendering acceleration layer optimizes the rendering effect with the help of the feature association matrix, and the layered rendering and feedback mechanism further optimizes the rendering quality. The adaptive adjustment of the multi-level rendering and rendering feedback layers ensures higher quality three-dimensional graphics output, while accelerating the rendering process, thereby improving the rendering rate and quality.

[0064] 3. The present invention proposes an adaptive adjustment mechanism for adjusting the three-dimensional scene during rendering, by obtaining the expected rendering target of the three-dimensional scene to be rendered and initializing the rendering parameters. Then, the system performance of the multi-level rendering layer during the rendering process is monitored and the real-time load index is calculated. On this basis, the rendering adjustment strategy is generated through the feature association matrix of the same and heterogeneous scenes, and the initialization rendering parameters are optimized. After the rendering parameters are adjusted, the rendering error is calculated, and further adjustments are made based on the real-time load and error feedback to optimize the rendering effect. This adaptive adjustment mechanism brings benefits in terms of rendering rate and quality of three-dimensional graphics. By dynamically adjusting the rendering parameters, the system performance can be optimized and the rendering speed can be improved without sacrificing the rendering quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1A flowchart of a deep learning-based graphics rendering acceleration method provided by an embodiment of the present invention;

[0066] Figure 2 A diagram illustrating the process of graphics rendering provided by an embodiment of the present invention;

[0067] Figure 3 A time comparison chart of graphics rendering before and after the experiment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] With the increasing complexity of rendering scenes and higher requirements for rendering quality, traditional graphics rendering technology faces significant performance bottlenecks and computing resource consumption, especially when dealing with large-scale three-dimensional scenes and real-time rendering. These traditional methods often require a lot of computing and memory resources, resulting in inefficiency in the rendering process. In addition, with the rapid advancement of hardware technology, users have higher expectations for the quality and speed of rendered images. In this context, researchers combine deep learning with graphics rendering technology and use neural networks to achieve efficient image generation and rendering, which can automatically optimize rendering details and improve rendering quality and real-time performance.

[0070] However, the training of deep learning models usually requires huge computing resources, especially in 3D graphics rendering, where huge data sets and high-dimensional features make the training process very time-consuming. Although the use of powerful GPU acceleration can shorten the training time to a certain extent, it still takes a long time to train complex rendering networks. In addition, although reducing the amount of training or reducing the feature dimension can increase the training speed, it may affect the rendering effect, resulting in the final 3D graphics rendering quality failing to meet the expected standards.

[0071] To solve this problem, the present invention proposes a graphics rendering acceleration method based on deep learning, aiming to optimize the rendering efficiency of three-dimensional scenes. Next, the technical details of the present invention will be described in detail through the following two embodiments.

[0072] Embodiment 1:

[0073] In the embodiment of the present application, the process of rendering graphics by scene A is described. It is known that scene A is a complex urban landscape scene, including multiple objects such as multiple buildings, roads, and green belts. Scene A can be based on Figure 1Rendering of the content, Figure 1 The implementation steps of the present invention include: S10. Obtaining relevant information of three-dimensional scenes of different scene types; S20. Processing the acquired information and constructing a standard three-dimensional scene data set; S30. Learning the feature association matrix of the same scene and the feature association matrix of the different scene; S40. Obtaining description information of the three-dimensional scene to be rendered; S50. Converting the description information into a text description vector; S60. Using a three-dimensional graphics rendering acceleration model to render graphics to obtain rendered graphics.

[0074] Further, according to the above description:

[0075] According to the content of step S10, relevant information of three-dimensional scenes of different scene types is obtained, including: geometric information, material properties, lighting environment data and camera parameters; wherein, in the embodiment of the present application, relevant information can be obtained from different scenes such as cities, villages and natural landscapes; the corresponding information specifically includes: geometric information including: vertex coordinates, normal vectors and texture coordinates of three-dimensional objects; material properties including: surface reflectivity, glossiness and refractive index; lighting environment data including: light source type, light source position, lighting environment and shadow information.

[0076] In the embodiments of the present application, by acquiring various information in different scenes and learning the correlation of features of the same scene and the feature differences between different scenes, the rendering algorithm can be optimized to improve the efficiency and quality of graphics rendering. Feature correlation learning can help the model identify the commonalities and differences between different scenes and accurately select the appropriate rendering strategy when rendering, thereby reducing the amount of calculation, speeding up the rendering speed, and avoiding excessive calculation of the rendering results.

[0077] Further, the acquired information is processed according to the content of S20, and the processed information is constructed into a standard three-dimensional scene dataset; wherein the specific processing process includes: converting the scene type into a text feature representation to obtain a scene text feature vector; performing data cleaning on geometric information, material characteristics, lighting environment data and camera parameters to obtain a first three-dimensional scene dataset; normalizing the data in the first three-dimensional scene dataset to obtain a second three-dimensional scene dataset; performing outlier detection on the data in the second three-dimensional scene dataset to obtain a third three-dimensional scene dataset; merging the scene text feature vector into the third three-dimensional scene dataset to obtain a fourth three-dimensional scene dataset; compressing the data in the fourth three-dimensional scene dataset to obtain a fifth three-dimensional scene dataset; and performing data conversion on the data in the fifth three-dimensional scene dataset to obtain a standard three-dimensional scene dataset.

[0078] In the embodiment of the present application, by learning the feature correlation of the processed data, the relationship between different scene elements can be effectively identified and captured, and the processing strategy of the rendering model can be optimized. Feature correlation learning helps the model to make adaptive adjustments according to the common features and differences of different scenes during the rendering process, thereby improving the efficiency and effect of rendering. It can reduce redundant calculations, increase rendering speed, and ensure the accuracy and quality of the rendering results.

[0079] Further, according to the content of step S30, the feature association matrix of the same scene and the feature association matrix of the different scene are learned using the standard three-dimensional scene data set; wherein the feature association matrix of the same scene and the feature association matrix of the different scene are learned using the three-dimensional scene feature association learning model,

[0080] The three-dimensional scene feature association learning model includes: an input layer, a classification layer, a feature extraction layer, a feature association learning layer and an output layer;

[0081] Among them, the specific implementation process of the 3D scene feature association learning model includes:

[0082] Input the data from the standard 3D scene dataset through the input layer;

[0083] Further, the input data is classified through a classification layer; wherein the classification layer divides the data into the same set according to the scene, and obtains a plurality of standard three-dimensional scene data subsets;

[0084] Furthermore, the classified data is input into the feature extraction layer for feature extraction; the feature extraction layer uses an autoencoder to obtain features, and the autoencoder can effectively reduce the dimension of high-dimensional data and learn feature representation, which is suitable for complex scene data;

[0085] Furthermore, the features extracted by the feature extraction layer are input into the feature association learning layer; wherein the feature association learning layer includes: a homogeneous three-dimensional scene feature association learning module and a heterogeneous three-dimensional scene feature association learning module;

[0086] Among them, similar 3D scene feature association learning modules include:

[0087] Acquire feature data of the same type of three-dimensional scenes, recorded as a feature set of the same type of three-dimensional scenes;

[0088] Furthermore, a multi-level graph structure is established based on the feature attributes in the feature set of the same type of three-dimensional scene;

[0089] Furthermore, the feature data is embedded into the nodes of the multi-level graph structure;

[0090] Furthermore, the node connectivity between the nodes is calculated based on the temporal relationship and the spatial geometric relationship; wherein the calculation formula of the node connectivity is:

[0091] R(i, j) = λ1·D Features (i,j)+λ2·TS(i,j)+λ3·Geo(i,j);

[0092] Where R(i, j) is the node connectivity between node i and node j; D Features () is the characteristic distance calculation function between nodes; TS() is the temporal relationship calculation function between nodes; Geo() is the spatial relationship calculation function between nodes; λ1, λ2 and λ3 are the weight coefficients of characteristic distance, temporal relationship and spatial relationship respectively;

[0093] Further, assigning the node connectivity to the connection edge value of the multi-level graph structure, and connecting the nodes;

[0094] Furthermore, the connection relationship of the multi-level graph structure and the connection edge value are input into a graph convolutional neural network to obtain a feature association matrix of the same type of scenes;

[0095] In this application, the correlation between the features of scenes of the same type is learned by using the same type of three-dimensional scene feature association learning module, and the intrinsic connection between different scene elements can be established through a multi-level graph structure. By calculating the connectivity between nodes, combining the temporal relationship and the spatial geometric relationship, a scene feature association matrix with deep features is formed. By inputting these association matrices into the graph convolutional neural network, the potential rules between the elements in the scene can be further refined, thereby providing accurate feature support for the rendering model.

[0096] Among them, the heterogeneous 3D scene feature association learning module includes:

[0097] Acquire feature data of different types of three-dimensional scenes, recorded as a heterogeneous three-dimensional scene feature set;

[0098] Further, the features in the heterogeneous three-dimensional scene feature set are reduced in dimension to the same dimension to obtain a heterogeneous three-dimensional scene reduced dimension feature set;

[0099] Furthermore, the features in the dimensionality reduction feature set of the heterogeneous three-dimensional scene are clustered to obtain multiple three-dimensional scene feature subsets;

[0100] Furthermore, the features in the three-dimensional scene feature subset are locally enhanced according to the spatial relationship to obtain a three-dimensional scene enhanced feature subset; wherein, in the embodiment of the present application, the feature enhancement formula in the three-dimensional scene feature subset is:

[0101]

[0102] in, Enhance feature i in the 3D scene feature subset; f i is the original feature i in the 3D scene feature subset; β is the parameter for adjusting the enhancement strength; d ij is the geometric distance between the original feature i and the original feature j in the 3D scene feature subset; d max is the maximum eigenvalue in the 3D scene feature subset; θ ij is the relative angle between the original feature i and the original feature j in the 3D scene feature subset;

[0103] Further, the correlation degree of the features in the 3D scene enhancement feature subset is calculated to obtain a plurality of heterogeneous 3D scene feature correlation vectors;

[0104] Furthermore, multiple heterogeneous three-dimensional scene feature association vectors are integrated to obtain a heterogeneous scene feature association matrix;

[0105] In this application, a heterogeneous 3D scene feature association learning module is used to learn the correlation between features of scenes of different types. This feature association learning method can significantly improve rendering accuracy and detail performance. By learning and enhancing the spatial relationship between different objects in a 3D scene, the rendering system can show a higher sense of reality in details, reduce visual distortion, enhance the depth and layering of the image, and reduce the amount of parameter calculation when rendering graphics.

[0106] Furthermore, the output layer is used to output the feature correlation matrix of the same scene and the feature correlation matrix of the different scene;

[0107] Refer to Table 1, which shows the changes in rendering efficiency and computational complexity after the introduction of the 3D scene feature association learning model;

[0108] Table 1 Performance comparison of the 3D scene feature association learning model

[0109]

[0110] Further, according to the content of step S40, the description information of the three-dimensional scene to be rendered (scene A) is obtained; wherein the description information includes: description, position, size, color and other information of all objects in the scene;

[0111] Further, according to the content of step S50, the description information is converted into a text description vector; wherein. The text is converted into a vector representation in digital form through a natural language processing (NLP) model. The text description vector is a numerical representation used to represent the text content;

[0112] Further, the three-dimensional image rendering acceleration model is used to perform graphics rendering according to the S60 content; see Figure 2 , speed up the rendering process of the model for 3D image rendering:

[0113] The text description vector, the feature correlation matrix of the same scene, and the feature correlation matrix of the different scene are input through the receiving layer;

[0114] Furthermore, the text description vector is converted into a low-dimensional description feature vector using an embedding layer;

[0115] In the specific 3D image rendering acceleration model, the text description vector is converted into a low-dimensional description feature vector through the embedding layer. Suppose we have a text description, for example: "blue road sign, position (0, 0, 0), size 50×30×75".

[0116] Step 1: Text input to the model

[0117] Text description: The text is preprocessed, usually by tokenization, to split it into words or subwords. For example, “blue road sign, position (0, 0, 0)” may be decomposed into [“blue”, “road sign”, “position”, “at”, “(0”, “0”, “0)”].

[0118] Step 2: Word vector initialization

[0119] Word embedding initialization: Each word or subword is mapped to a dense vector of fixed dimension through the word embedding layer. For each word, the embedding layer assigns it a vector, which is learned and can be continuously optimized during training.

[0120] For example, suppose “blue” is mapped to [0.2, 0.4, 0.1] and “street sign” is mapped to [0.3, 0.6, 0.2]. The dimensions of these vectors can be set via hyperparameters, and are typically chosen to be between tens and hundreds of dimensions.

[0121] Step 3: Combination of word embedding vectors

[0122] Merge word embeddings: Combine the embedding vectors of each word into a matrix and send it as input to the next layer of the neural network. The size of this matrix is ​​vocabulary × embedding dimension, and each row represents the embedding vector of a word.

[0123] Step 4: Low-dimensional description feature vector

[0124] Convert the embedding vector into a low-dimensional feature vector: Usually, the text description vector contains multiple words or subwords. After obtaining the vector of each word through the embedding layer, the next step is to aggregate these word vectors into an overall description feature vector through pooling, averaging, etc.

[0125] For example, using the average pooling method, the vectors of all words are averaged to obtain a unified vector [0.25, 0.5, 0.15]. This is the low-dimensional feature vector of the text description.

[0126] Further, matching the rendering scene through a scene matching layer according to the description feature vector;

[0127] Furthermore, the self-attention layer is used to focus on the features in the description feature vector according to the rendering scene;

[0128] The self-attention layer performs weighted attention on the descriptive feature vector, emphasizing important features while ignoring less relevant information. It allows the model to adaptively assign different attention weights to different parts of the scene.

[0129] Self-attention layer realizes the following functions:

[0130] (1) Weighted attention: The self-attention mechanism allows the model to establish associations between each part of the description feature vector. For example, when describing a three-dimensional scene, the model can focus on certain key features in the description (such as the color and shape of the object) and adjust the rendering process based on these features.

[0131] (2) Dynamic feature selection: The self-attention mechanism does not need to predefine which features are more important. It can dynamically adjust the degree of attention to features according to the specific needs of the scene. This enables the model to flexibly process information according to the context when facing diverse scenes.

[0132] Calculation process: In the self-attention mechanism, each input feature vector will calculate an attention score to determine the importance of the feature in the rendering process. The calculation process includes:

[0133] (1) Query, Key, Value: Each descriptive feature vector generates a Query vector, a Key vector, and a Value vector through different linear transformations.

[0134] (2) Attention score: Calculate the similarity between the query and the key (for example, by dot product calculation) to obtain the attention score.

[0135] (3) Weighted summation: Perform weighted summation on the Value vectors according to the scores to obtain a weighted output vector.

[0136] (4) Output: The output of the self-attention layer is a weighted version of each feature in the feature vector. These weighted features will be passed to the next layer (such as the embedding layer, scene rendering layer, etc.) for further processing and rendering.

[0137] In the embodiment of the present application, by combining the scene matching layer and the self-attention layer, the two layers can enhance the connection between the description feature vector and the rendered scene by working together. Through these processing steps, the model can intelligently select the appropriate three-dimensional scene based on the input scene description information, and accurately render an image or three-dimensional scene that is highly consistent with the description information through the self-attention mechanism.

[0138] Furthermore, in the rendering acceleration layer, the description features of interest are optimized through the feature association matrix of the same scene and the feature association matrix of the different scene; wherein the specific rendering acceleration formula of the rendering acceleration layer is:

[0139] V 优化 =ω·V+α·(M 同类 ·V)+β·(M 异类 V);

[0140] V 加速 =σ(V 优化 +b 加速 );

[0141] Among them, V 优化 is the optimized descriptive feature; ω is the weight of the descriptive feature vector; V is the descriptive feature of interest; M 同类 is the feature correlation matrix of the same scene; M 异娄 is the correlation matrix of heterogeneous scene features; α and β are the influences of adjusting homogeneous and heterogeneous features; V 加速 is the acceleration description feature after optimization; σ() is the nonlinear activation function; b 加速 is the acceleration constant;

[0142] Furthermore, a multi-level rendering layer is used to perform layered rendering according to the output of the rendering acceleration layer; wherein the multi-level rendering layer also includes dynamically adjusting rendering parameters using an adaptive adjustment mechanism;

[0143] Among them, the adaptive adjustment mechanism includes:

[0144] Get the expected rendering target (such as target resolution, frame rate, lighting effect, etc.) of the 3D scene to be rendered;

[0145] Further, initialize the rendering parameters P 初始 ;

[0146] The initial rendering parameters include: resolution R 初始 , light intensity L 初始 , Level of detail D 初始 and texture complexity C 初始 ;

[0147] Furthermore, the system performance of the multi-level rendering layer is monitored during rendering, and real-time load indicators are calculated, including: GPU usage U GPU 、CPU usage U CPU 、Memory usage U 内存 and rendering frame time T 帧 ;

[0148] Furthermore, a rendering adjustment strategy is obtained according to the feature correlation matrix of the same scene and the feature correlation matrix of the different scene;

[0149] S 调整 =G(M 同类 , M 异类 ,MLR(V 加速 ));

[0150] Among them, S 调整 is the rendering adjustment strategy; G() is the rendering adjustment strategy generation function; MLR() is the multi-level rendering layer function;

[0151] Further, adjusting the initialization rendering parameters according to the rendering adjustment strategy;

[0152] P 更新 =P 初始 +ΔP;

[0153] Among them, P 更新 is to update the rendering parameters; ΔP is the adjustment amount; specifically:

[0154] Resolution adjustment: R 更新 =R 初始 -κ R ·(U GPU -U 目标GPU );

[0155] Light intensity adjustment: L 更新 =L 初始 ·(1-κ L ΔT 帧 );

[0156] Level of detail adjustment: D 更新 =D 初始 -κ D ΔU 内存 ;

[0157] Texture complexity adjustment: C 更新 =C 初始 ·(1-κ T ΔU GPU );

[0158] Among them, R 更新 is the resolution adjustment value; L 更新D is the light intensity adjustment value; 更新 Adjust the value for the level of detail; C 更新 is the texture complexity adjustment value; κ R is the resolution adjustment factor; κ L is the light intensity adjustment coefficient; κ D is the detail level adjustment coefficient; κ T is the texture complexity adjustment factor; U 目标GPU is the target GPU utilization; ΔT 帧 is the change in rendering frame time; ΔU 内存 is the change in memory usage; ΔU GPU is the change in GPU usage;

[0159] Further, a rendering error between the rendered graphics after the rendering parameters are adjusted and the expected rendering target is calculated;

[0160] E 渲染 =||O 当前 -O 目标 ||2;

[0161] Among them, E 渲染 is the rendering error; O 当前 Output for current rendering; O 目标 is the intended rendering target;

[0162] Furthermore, rendering feedback is provided based on real-time load indicators and rendering errors;

[0163] F 反馈 =g(U GPU , U CPU , U 内存 , T 帧 , E 渲染 );

[0164] Among them, F 反馈 is the feedback mechanism; g() is the feedback function;

[0165] See Table 2, which shows various performances optimized by the adaptive adjustment mechanism;

[0166] Table 2 Performance comparison after optimization of adaptive adjustment mechanism

[0167]

[0168] In an embodiment of the present application, an adaptive adjustment mechanism is used to adaptively adjust the rendering parameters in the rendering process, and the mechanism dynamically optimizes the rendering parameters according to the system performance and rendering error monitored in real time. This process automatically adjusts the rendering parameters by calculating the feature association matrix of the same and heterogeneous scenes to ensure that the rendering results are close to the expected goals and maintain a reasonable level of system load. Through continuous feedback and adjustment, the mechanism can flexibly allocate resources and optimize performance under different rendering requirements. The advantage of this adaptive adjustment mechanism is that it can improve the efficiency and accuracy of rendering and ensure the balance of the rendering process in different scenes. Through real-time monitoring and adjustment, the system can avoid excessive calculation and waste of resources while ensuring image quality, thereby improving rendering speed, reducing delays, reducing hardware loads, and ultimately providing a high-quality, low-latency graphics rendering experience. This mechanism makes three-dimensional graphics rendering more adaptable and intelligent in practical applications, especially in complex dynamic scenes, and can quickly respond to changes and ensure stable output quality.

[0169] Further, the graphics output layer renders the graphics for output; see Figure 3 ,exist Figure 3 A comparison of graphics rendering time when the present invention is adopted and when the present invention is not adopted is given in FIG.

[0170] The embodiment of the present application adopts a three-dimensional graphics rendering acceleration model for graphics rendering. By receiving the text description vector and combining the optimization of the feature matrix of the same and different scenes, the multi-level feature extraction and attention mechanism is used to achieve efficient acceleration and refined processing of the rendering process. The model greatly improves the efficiency and quality of graphics rendering through the optimization of the self-attention layer and the rendering acceleration layer. Especially in the rendering of complex scenes, it can significantly reduce the consumption of computing resources while maintaining high-precision graphics output, ultimately achieving a smoother user experience and faster rendering time.

[0171] In the embodiment of the present application, the method of the present invention is used to realize the graphics rendering of scene A; specifically, by converting the geometric information, material characteristics, lighting environment data and camera parameters of scene A into a standard three-dimensional scene data set, and combining the feature association matrix of the same and heterogeneous scenes, a three-dimensional graphics rendering acceleration model is used for rendering. At the same time, the rendering parameters are dynamically optimized using an adaptive adjustment mechanism to ensure that the rendering quality and performance are optimal. This method not only greatly improves the rendering speed and shortens the rendering time, but also ensures high-quality output of graphics through precise parameter adjustment and multi-level rendering hierarchy optimization, especially in complex scenes, it can effectively reduce the system burden, improve rendering efficiency, and meet the needs of high-performance computing.

[0172] Embodiment 2:

[0173] In Example 1, efficient and high-quality graphics rendering of scene A is achieved; in order to illustrate again that the present invention is applicable in different scenarios, the following will be explained through the examples of the present application. In the example of the present application, graphics rendering is performed on scene B (forest scene), as follows:

[0174] Obtain geometric information, material properties, lighting environment data and camera parameters of 3D scenes of different scene types;

[0175] Furthermore, the scene type, geometric information, material properties, lighting environment data and camera parameters are processed, and a standard three-dimensional scene dataset is constructed.

[0176] Furthermore, the standard three-dimensional scene data set is input into the three-dimensional scene feature association learning model to obtain a similar scene feature association matrix and a heterogeneous scene feature association matrix;

[0177] Further, obtaining description information of the three-dimensional scene to be rendered (scene B);

[0178] Further, the description information is converted into a text description vector;

[0179] Furthermore, the text description vector, the homogeneous scene feature association matrix and the heterogeneous scene feature association matrix are input into a 3D graphics rendering acceleration model to obtain a rendered graphic; wherein the 3D graphics rendering acceleration model includes:

[0180] A receiving layer, used to receive a text description vector, a homogeneous scene feature association matrix, and a heterogeneous scene feature association matrix;

[0181] Embedding layer, used to convert the text description vector into a low-dimensional description feature vector;

[0182] The scene matching layer is used to match the rendering scene according to the description vector;

[0183] A self-attention layer that focuses on features in the description feature vector based on the rendered scene;

[0184] The rendering acceleration layer is used to optimize the description features of interest based on the feature association matrix of the same scene and the feature association matrix of the different scene;

[0185] Multi-level rendering layers are used to perform layered rendering based on the output of the rendering acceleration layer; rendering parameters are adjusted in the multi-level rendering layers through an adaptive adjustment mechanism;

[0186] Among them, the adaptive adjustment mechanism includes:

[0187] Obtaining the expected rendering target of the 3D scene to be rendered;

[0188] Further, initializing rendering parameters;

[0189] Furthermore, the system performance of multiple rendering layers is monitored during rendering, and real-time load indicators are calculated;

[0190] Furthermore, a rendering adjustment strategy is obtained according to the feature correlation matrix of the same scene and the feature correlation matrix of the different scene;

[0191] Further, adjusting the initialization rendering parameters according to the rendering adjustment strategy;

[0192] Further, a rendering error between the rendered graphics after the rendering parameters are adjusted and the expected rendering target is calculated;

[0193] Furthermore, rendering feedback is provided based on real-time load indicators and rendering errors;

[0194] Graphics output layer, used to output rendered graphics.

[0195] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A graphics rendering acceleration method based on deep learning, characterized in that: include: Obtain geometric information, material properties, lighting environment data and camera parameters of 3D scenes of different scene types; Processing the scene type, the geometric information, the material characteristics, the lighting environment data and the camera parameters, and constructing a standard three-dimensional scene data set; Inputting the standard three-dimensional scene data set into a three-dimensional scene feature association learning model to obtain a similar scene feature association matrix and a heterogeneous scene feature association matrix; Wherein, the three-dimensional scene feature association learning model includes: An input layer, used for receiving data in the standard three-dimensional scene dataset; A classification layer, used for classifying the data received by the input layer according to the scene type to obtain a plurality of standard three-dimensional scene data subsets; A feature extraction layer, used for extracting features from the data in the standard three-dimensional scene data subset; The feature association learning layer is used to perform association learning on the extracted features, including: a homogeneous 3D scene feature association learning module and a heterogeneous 3D scene feature association learning module; An output layer, used to output the feature association matrix of the same scene and the feature association matrix of the different scene obtained by the feature association learning layer; Obtain description information of the 3D scene to be rendered; Converting the description information into a text description vector; The text description vector, the homogeneous scene feature association matrix and the heterogeneous scene feature association matrix are input into a three-dimensional graphics rendering acceleration model to obtain a rendered graphic.

2. The method for accelerating graphics rendering based on deep learning according to claim 1, characterized in that: The geometric information includes: vertex coordinates, normal vectors and texture coordinates of three-dimensional objects; the material properties include: surface reflectivity, glossiness and refractive index; the lighting environment data includes: light source type, light source position, lighting environment and shadow information.

3. The method for accelerating graphics rendering based on deep learning according to claim 1, characterized in that: The construction process of the standard three-dimensional scene dataset includes: Convert the scene type into a text feature representation to obtain a scene text feature vector; Performing data cleaning on the geometric information, the material properties, the lighting environment data, and the camera parameters to obtain a first three-dimensional scene data set; Normalizing the data in the first three-dimensional scene data set to obtain a second three-dimensional scene data set; Performing outlier detection on the data in the second three-dimensional scene data set to obtain a third three-dimensional scene data set; Merging the scene text feature vector into the third three-dimensional scene data set to obtain a fourth three-dimensional scene data set; compressing the data in the fourth three-dimensional scene data set to obtain a fifth three-dimensional scene data set; Data conversion is performed on the data in the fifth three-dimensional scene data set to obtain the standard three-dimensional scene data set.

4. The method for accelerating graphics rendering based on deep learning according to claim 1, characterized in that: The similar three-dimensional scene feature association learning module includes: Acquire feature data of the same type of three-dimensional scenes, recorded as a feature set of the same type of three-dimensional scenes; Establishing a multi-level graph structure according to the feature attributes in the feature set of the same type of three-dimensional scene; embedding the feature data into the nodes of the multi-level graph structure; The node connectivity between the nodes is calculated according to the temporal relationship and the spatial geometric relationship; wherein the calculation formula of the node connectivity is: R(i,j)=λ1·D Features (i,j)+λ2·TS(i,j)+λ3·Geo(,j); Where R(i, j) is the node connectivity between node i and node j; D Features ( ) is the function for calculating the characteristic distance between nodes; TS( ) is the function for calculating the temporal relationship between nodes; Geo( ) is the function for calculating the spatial relationship between nodes; λ1, λ2 and λ3 are the weight coefficients of characteristic distance, temporal relationship and spatial relationship respectively; Assigning the node connectivity to the connection edge value of the multi-level graph structure, and connecting the nodes; The connection relationship of the multi-level graph structure and the connection edge value are input into the graph convolutional neural network to obtain the feature association matrix of the same type of scenes.

5. The method for accelerating graphics rendering based on deep learning according to claim 1, characterized in that: The heterogeneous three-dimensional scene feature association learning module includes: Acquire feature data of different types of three-dimensional scenes, recorded as a heterogeneous three-dimensional scene feature set; The features in the heterogeneous three-dimensional scene feature set are reduced in dimension to the same dimension to obtain a heterogeneous three-dimensional scene reduced-dimension feature set; Clustering the features in the heterogeneous three-dimensional scene dimensionality reduction feature set to obtain multiple three-dimensional scene feature subsets; Locally enhancing the features in the three-dimensional scene feature subset according to the spatial relationship to obtain a three-dimensional scene enhanced feature subset; Calculating the correlation degree of features in the three-dimensional scene enhancement feature subset to obtain a plurality of heterogeneous three-dimensional scene feature correlation vectors; A plurality of the heterogeneous three-dimensional scene feature association vectors are integrated to obtain the heterogeneous scene feature association matrix.

6. The method for accelerating graphics rendering based on deep learning according to claim 1, characterized in that: The three-dimensional graphics rendering acceleration model includes: A receiving layer, used for receiving the text description vector, the homogeneous scene feature association matrix and the heterogeneous scene feature association matrix; An embedding layer, used to convert the text description vector into a low-dimensional description feature vector; A scene matching layer, used for matching the rendering scene according to the description feature vector; A self-attention layer for focusing on features in the descriptive feature vector according to the rendering scene; A rendering acceleration layer, used for optimizing the description features of interest according to the homogeneous scene feature association matrix and the heterogeneous scene feature association matrix; A multi-level rendering layer, used for performing layered rendering according to the output of the rendering acceleration layer; The graphics output layer is used to output the rendered graphics.

7. The method for accelerating graphics rendering based on deep learning according to claim 1, characterized in that: When the 3D graphics rendering acceleration model renders the 3D scene to be rendered, it also includes adjusting rendering parameters through an adaptive adjustment mechanism; Among them, the adaptive adjustment mechanism includes: obtaining the expected rendering target of the three-dimensional scene to be rendered; initializing rendering parameters; monitoring the system performance of multiple rendering layers during rendering, and calculating real-time load indicators; obtaining a rendering adjustment strategy based on the feature association matrix of the same scene and the feature association matrix of the heterogeneous scene; adjusting the rendering parameters of the initialized rendering parameters according to the rendering adjustment strategy; calculating the rendering error between the rendered graphics and the expected rendering target after the rendering parameters are adjusted; and performing rendering feedback according to the real-time load indicator and the rendering error.

Images