Dynamic adaptive sampling rate optimization method and device based on artificial intelligence, and medium

Abstract

The invention relates to the technical field related to rendering, in particular to a dynamic adaptive sampling rate optimization method and device based on artificial intelligence and a medium. The method is applied to a real-time rendering system, and comprises the steps that a neural network model is constructed and trained, and the neural network model is used for automatically predicting the optimal sampling rate according to an input scene and a rendering quality target; constructing a dynamic adaptive sampling rate algorithm for automatically adjusting the sampling rate based on the output of the neural network model; in the real-time rendering process, the optimal sampling rate is predicted in real time through the neural network model, and the sampling rate of rendering is adjusted to the optimal sampling rate based on the dynamic adaptive sampling rate algorithm in the rendering process.

Description

Technical Field

[0001] The present application relates to the field of rendering-related technologies, and in particular to a method, device and medium for dynamically adaptive sampling rate optimization based on artificial intelligence. Background Art

[0002] In the field of real-time rendering, in order to strike a balance between high image quality and performance, it is usually necessary to implement different levels of sampling operations on the scene. However, traditional sampling strategies often rely on preset rules or user input. This approach has obvious limitations, as follows:

[0003] First, when faced with a complex and dynamically changing rendering environment, the preset rules cannot flexibly adjust according to the real-time changes of the scene. For example, when the complexity of objects in the scene suddenly increases, the lighting conditions change dramatically, or a large number of reflection / refraction effects appear, the preset sampling rules may not be able to accurately adapt to these changes, resulting in an unreasonable sampling rate. This may cause the rendered image to lose details in certain areas, affecting the overall image quality.

[0004] Secondly, the sampling method that relies on user input requires a high level of professional knowledge and experience from the user. The user needs to accurately judge the characteristics of each scene and manually set the appropriate sampling rate. In actual operation, this not only increases the user's workload, but also, due to the subjectivity and errors of human judgment, it is difficult to ensure that the sampling rate set each time is optimal. This inaccuracy may also lead to a waste of resources or suboptimal image quality. Summary of the invention

[0005] In view of this, the embodiments of the present application are directed to providing an artificial intelligence-based dynamic adaptive sampling rate optimization method, device and medium for better rendering.

[0006] The present application provides a dynamic adaptive sampling rate optimization method based on artificial intelligence, comprising:

[0007] Building and training a neural network model, the neural network model is used to automatically predict the optimal sampling rate according to the input scene and rendering quality target;

[0008] Constructing a dynamic adaptive sampling rate algorithm for automatically adjusting the sampling rate based on the output of the neural network model;

[0009] During the real-time rendering process, the neural network model is used to predict the optimal sampling rate in real time, and the rendering sampling rate is adjusted to the optimal sampling rate based on the dynamic adaptive sampling rate algorithm during rendering.

[0010] In some embodiments, the training of the neural network model includes:

[0011] Collect a preset number of high-quality rendering samples under different scenarios, device states, and rendering requirements as training data;

[0012] The neural network model is trained based on the training data.

[0013] In some embodiments, collecting a preset number of high-quality rendering samples under different scenarios, device states, and rendering requirements as training data includes:

[0014] Collect rendering samples under different scenarios, device states, and rendering requirements;

[0015] Perform quality assessment on rendering samples and remove rendering samples that fail the quality assessment;

[0016] Marks the rendering samples corresponding to the optimal sampling rate.

[0017] In some embodiments, the neural network model is a convolutional neural network.

[0018] In some embodiments, in the real-time rendering process, using the neural network model to predict the optimal sampling rate in real time, and adjusting the rendering sampling rate to the optimal sampling rate based on the dynamic adaptive sampling rate algorithm during rendering, includes:

[0019] In the real-time rendering process, the input scene information, rendering quality and device performance parameters are provided to the trained neural network model to obtain the predicted optimal sampling rate;

[0020] Based on the dynamic adaptive sampling rate algorithm, the sampling rate in the current rendering process is updated in time to be the optimal sampling rate.

[0021] In some embodiments, the principles of disease prediction by the neural network model include:

[0022] In complex scenarios, increase the sampling rate;

[0023] In simple scenarios, reduce the sampling rate;

[0024] Further adjust the sampling rate based on the computing power and frame rate limitations of the device.

[0025] In some embodiments, it also includes:

[0026] Use hardware acceleration and distributed computing to improve computing speed.

[0027] In some embodiments, it also includes:

[0028] Get feedback;

[0029] The neural network model is retrained based on the feedback information.

[0030] The present application also provides an electronic device, including:

[0031] A processor, and a memory for storing a program executable by the processor;

[0032] The processor is used to implement the above-mentioned dynamic adaptive sampling rate optimization method based on artificial intelligence by running the program in the memory.

[0033] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the above-mentioned dynamic adaptive sampling rate optimization method based on artificial intelligence.

[0034] The present application provides a dynamic adaptive sampling rate optimization method based on artificial intelligence. First, a neural network model is constructed and trained. The neural network model is used to automatically predict the optimal sampling rate according to the input scene and rendering quality target; a dynamic adaptive sampling rate algorithm is constructed to automatically adjust the sampling rate based on the output of the neural network model; in the real-time rendering process, the neural network model is used to predict the optimal sampling rate in real time, and the rendering sampling rate is adjusted to the optimal sampling rate based on the dynamic adaptive sampling rate algorithm in the rendering. In this way, by constructing and training the neural network model, the optimal sampling rate can be automatically predicted according to the input scene and rendering quality target. Different scenes (such as complex 3D game scenes containing many detailed objects, architectural scenes with different lighting conditions, etc.) have different rendering requirements. The neural network can learn various features in the scene, such as the relationship between the complexity of the texture of the object, the distribution and intensity change of the light, and the sampling rate. For example, in an ancient building scene containing fine textures, the neural network can identify that a higher sampling rate is required to accurately render the texture details, thereby improving the final rendering quality and avoiding the occurrence of texture blur or distortion. The dynamic adaptive sampling rate algorithm can automatically adjust the sampling rate based on the output of the neural network model. During real-time rendering, the scene may change constantly, such as the movement of characters in the game scene, the addition or deletion of objects, and the change of lighting angle. This dynamic adjustment ensures that rendering can be performed at the most appropriate sampling rate under any scene change. For example, when a character in the game walks from a dark indoor room to a sunny outdoor room, the algorithm can adjust the sampling rate in time to better render the details of the bright outdoor scene, so that the picture quality can be maintained at a high level during various scene transitions. Avoid oversampling. The traditional fixed sampling rate method may use too high a sampling rate in some simple scenes or areas with low rendering quality requirements, resulting in a waste of computing resources. This method can adjust the sampling rate in real time according to the scene and quality goals, avoiding oversampling in places where high sampling rates are not required. For example, in a scene with the sky as the main background, the sky part is relatively simple and does not require as high a sampling rate as the character model in the scene. Through dynamic adjustment, the sampling rate of the sky part can be reduced, thereby saving computing resources and improving rendering efficiency. From the perspective of rendering quality, users can enjoy clearer and more realistic visual effects. Whether it is the scene details in the game or the texture and lighting effects of exhibits in applications such as virtual exhibition halls, they can all be rendered at close to the optimal sampling rate, enhancing the visual appeal and realism. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0036] Figure 1 It is a flowchart of a dynamic adaptive sampling rate optimization method based on artificial intelligence provided by an embodiment of the present application.

[0037] Figure 2 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Application Overview

[0040] With the rapid development of computer graphics and virtual reality technology, real-time rendering technology is increasingly used in games, movies, simulation training and other fields. The core goal of real-time rendering is to provide high-quality image output as much as possible under limited computing resources. In order to achieve this goal, sampling strategy has become one of the key technologies.

[0041] Sampling is a fundamental step in the rendering process, which involves deciding how to obtain discrete data points from continuous signals (such as the scene's lighting, color, depth, etc.). The sampling rate directly affects the quality and performance of the rendered image. A high sampling rate can provide richer details, but it will increase the computational burden and cause the rendering speed to decrease; while a low sampling rate can increase the rendering speed, but it may cause image noise and loss of details.

[0042] Traditional sampling strategies usually rely on preset rules or user input, such as fixed time interval sampling or simple adaptive methods based on scene complexity. These methods often fail to make optimal decisions when faced with complex and dynamically changing rendering environments, resulting in resource waste or image quality degradation.

[0043] Fixed sampling rate: The same sampling rate is used in all scenarios and cannot be dynamically adjusted according to actual needs, resulting in uneven resource utilization.

[0044] Simple adaptive sampling: Although the sampling rate can be adjusted according to the local complexity of the scene, it lacks comprehensive consideration of the overall rendering requirements and device performance.

[0045] User input: Relying on user experience to adjust the sampling rate lacks flexibility and automation and is difficult to adapt to rapidly changing scenarios.

[0046] To solve the above problems, a smarter and more automated sampling strategy is needed that can dynamically adjust the sampling rate according to the specific needs of real-time rendering and device performance. This strategy should be able to:

[0047] Quickly respond to scene changes and adjust the sampling rate in real time to adapt to different rendering requirements.

[0048] Comprehensively consider image quality, rendering speed and device performance to achieve optimal allocation of resources.

[0049] Reduce manual intervention and provide a more automated and intelligent rendering experience.

[0050] In recent years, artificial intelligence (AI) technology, especially deep learning, has made remarkable progress in the fields of image processing and pattern recognition. AI technology has shown great potential in processing complex data and making intelligent decisions. Applying AI technology to sampling strategies in real-time rendering can provide new ideas for solving the limitations of existing technologies.

[0051] By training a deep learning model, we can build a system that can understand and predict the optimal sampling rate for different scenarios, device states, and rendering requirements. Such a system can:

[0052] Analyze scene characteristics and predict the optimal sampling rate.

[0053] Adjust sampling rates based on real-time feedback to optimize rendering results.

[0054] Learn and adapt to new rendering scenarios and requirements, and continuously improve performance.

[0055] The present invention aims to provide a dynamic adaptive sampling rate optimization method and system based on artificial intelligence, which uses a machine learning model to predict the optimal sampling rate by analyzing the characteristics of the rendering scene in real time, so as to achieve efficient resource utilization and high-quality image output. This method is expected to significantly improve the performance and image quality of real-time rendering and meet the growing demand for high-quality rendering.

[0056] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0057] Exemplary Methods

[0058] Figure 1 FIG. 1 is a flow chart of a dynamic adaptive sampling rate optimization method based on artificial intelligence provided by an embodiment of the present application. Figure 1 As shown, the method includes the following contents.

[0059] Step S110, constructing and training a neural network model, wherein the neural network model is used to automatically predict an optimal sampling rate according to an input scene and a rendering quality target;

[0060] Specifically, this step aims to create a neural network model that can automatically predict the optimal sampling rate based on the input scene and rendering quality target. The neural network model has strong learning and prediction capabilities, and can find complex relationships between input and output by learning from large amounts of data.

[0061] Step S120, constructing a dynamic adaptive sampling rate algorithm for automatically adjusting the sampling rate based on the output of the neural network model;

[0062] This step is to build a dynamic adaptive sampling rate algorithm based on the output of the trained neural network model. The purpose of this algorithm is to automatically adjust the sampling rate according to the sampling rate predicted by the neural network model and other related factors to adapt to different rendering scenes and device performance. First, it is necessary to obtain various relevant parameters in the rendering process, including scene information (such as lighting conditions, object complexity, motion blur, etc.), rendering quality goals (such as image detail requirements), and the computing power and frame rate limit of the device. These parameters will serve as the basis for the algorithm to adjust the sampling rate. Different parameter combinations may lead to different sampling rate adjustment strategies. The obtained input parameters are input into the trained neural network model to obtain the optimal sampling rate of its output. Then, according to the scene information and rendering quality requirements, determine whether the sampling rate needs to be increased or decreased. For example, if the scene is complex, contains a lot of details and reflection / refraction effects, or the rendering quality requirements are high, the sampling rate needs to be increased; if the scene is simple, the lighting is relatively uniform and the objects are relatively simple, or the device performance is limited, the sampling rate needs to be reduced. At the same time, factors such as the computing power and frame rate limit of the device need to be considered to further optimize the sampling rate adjustment strategy to ensure that good rendering effects can be achieved on different devices.

[0063] Step S130, during the real-time rendering process, the neural network model is used to predict the optimal sampling rate in real time, and the rendering sampling rate is adjusted to the optimal sampling rate based on the dynamic adaptive sampling rate algorithm during rendering.

[0064] During the real-time rendering process, the input scene information, rendering quality, device performance and other parameters are provided to the trained neural network model. Based on these inputs, the neural network model predicts the optimal sampling rate in real time. Since the rendering scene changes dynamically, the real-time prediction can capture the changes in the scene in a timely manner and provide an accurate basis for the subsequent sampling rate adjustment. According to the prediction results of the neural network model, the dynamic adaptive sampling rate algorithm is used to interact with the rendering system interface to pass the new sampling rate value to the rendering engine, and adjust the rendering sampling rate of the current frame in real time. This ensures that during the rendering process, the sampling rate can be adjusted in time according to the changes in the scene and device, achieving efficient resource utilization and high-quality image output.

[0065] Specifically, the neural network model is a convolutional neural network.

[0066] The structural characteristics and advantages of convolutional neural networks are as follows:

[0067] Local connection

[0068] In a convolutional neural network, the connections between neurons are local. For example, for a two-dimensional image data (rendered samples can be regarded as image data), the neurons in the convolution layer are only connected to a local area of ​​the input image. This local connection method reduces the number of parameters of the model and reduces the computational complexity. Compared with a fully connected network, CNN can use computing resources more efficiently and reduce the risk of overfitting when processing large-scale image data.

[0069] Weight sharing

[0070] Another important feature of CNN is weight sharing. In the convolution layer, the same convolution kernel slides over the entire image to perform the convolution operation, using the same weight parameters. This means that the model uses the same feature extraction method for local areas at different locations. Weight sharing further reduces the number of parameters of the model, making it easier to train the model and able to learn common feature patterns in the image. For example, when identifying the edges of objects in rendered samples, no matter where the object edges appear in the image, the convolution kernel can extract the edge features through the same weight parameters.

[0071] Hierarchical feature extraction

[0072] CNN usually consists of multiple convolutional layers, pooling layers, and fully connected layers. Through a layer-by-layer approach, more and more abstract and advanced features can be extracted from the original image data. In the lower-level convolutional layers, the main features of the image are extracted, such as edges, lines, textures, etc.; as the number of layers increases, more complex object shapes, structures and other features can be extracted in the higher-level convolutional layers. This hierarchical feature extraction method is consistent with the human visual system's cognitive process of images, and can better understand and process complex scene information in rendered samples, thereby providing more valuable feature representation for accurately predicting the optimal sampling rate.

[0073] The application details of the convolutional neural network in this patent technology are as follows:

[0074] Convolutional layer design

[0075] Convolution kernel size selection: Select the appropriate convolution kernel size based on the characteristics of the rendered sample and the features to be extracted. For capturing local details in the image, such as object edges and textures, a smaller convolution kernel, such as 3×3 or 5×5, may be selected. For extracting features in a larger range, such as the overall shape of an object, a larger convolution kernel may be used, but an overly large convolution kernel will increase the amount of calculation and the number of parameters, so you need to weigh the pros and cons.

[0076] Number of convolution kernels: Determine the number of convolution kernels in each convolution layer. More convolution kernels can extract richer features, but will also increase computational complexity and the number of model parameters. In this patented technology, the appropriate number of convolution kernels is determined through experiments and optimization based on the requirements for feature extraction under different scenes, device states, and rendering requirements. For example, when processing rendering samples under complex lighting conditions, more convolution kernels may be required to extract lighting features of different directions and intensities.

[0077] Step size setting: The step size of the convolution kernel sliding on the image will also affect the effect of feature extraction. A smaller step size can make the convolution kernel cover the image more densely and extract more detailed features, but it will increase the amount of calculation; a larger step size can reduce the amount of calculation, but may miss some detailed features. According to the resolution of the rendered sample and the requirements for feature extraction, the step size should be set reasonably. For example, for high-resolution rendered samples, a smaller step size may be selected in order to fully extract detailed features.

[0078] Pooling layer application

[0079] Pooling method selection: Commonly used pooling methods include maximum pooling and average pooling. Maximum pooling selects the maximum value in the local area as the output, which can retain the significant features in the image; average pooling takes the average value of the local area as the output to smooth the image. In this patented technology, a suitable pooling method is selected according to the characteristics of the rendered sample and the features that need to be retained. For example, when processing a rendered sample with obvious object edges and highlight areas, the maximum pooling method may be selected to highlight these significant features.

[0080] Pooling window size and step size: Similar to the convolution layer, the pooling layer also needs to determine the pooling window size and step size. The pooling window size determines the range of the local area, and the step size determines the frequency of the pooling window sliding on the image. Reasonable setting of these parameters can reduce the amount of data while retaining important features. For example, for some small objects in the image, if the pooling window is too large, the features of these small objects may be lost, so it is necessary to select the appropriate pooling window size and step size according to the size and distribution of the objects.

[0081] Function of the fully connected layer

[0082] After extracting features through multiple convolutional layers and pooling layers, the size of the feature map will gradually decrease, and the feature representation will become more and more abstract. The role of the fully connected layer is to map these abstract features to the final output space, that is, to predict the optimal sampling rate. The fully connected layer comprehensively processes all the features extracted previously, and outputs the predicted sampling rate value by learning the complex relationship between the features. In this patented technology, the number of parameters and connection methods of the fully connected layer need to be optimized according to the amount of training data and the feature dimension to ensure that the model can accurately predict the optimal sampling rate.

[0083] The training of the neural network model includes: collecting a preset number of high-quality rendering samples under different scenes, device states and rendering requirements as training data; and training the neural network model based on the training data. The collecting of a preset number of high-quality rendering samples under different scenes, device states and rendering requirements as training data includes: collecting rendering samples under different scenes, device states and rendering requirements; performing quality assessment on the rendering samples, and removing rendering samples that fail the quality assessment; and marking the optimal sampling rate corresponding to the rendering samples. The neural network model is a convolutional neural network.

[0084] 1. Collect a preset number of high-quality rendering samples under different scenes, device states, and rendering requirements as training data

[0085] (I) Collect rendering samples under different scenarios, device states, and rendering requirements

[0086] Scene Diversity

[0087] Indoor scenes: include rooms with various functions, such as bedrooms (with different furniture arrangements, bedding materials and colors), living rooms (different sofa styles, carpet materials, TV placements, etc.), kitchens (different kitchenware, cabinet materials and colors, lighting methods, etc.). These indoor scenes may have natural light shining through windows at different angles, as well as different intensities and color temperatures of artificial lighting.

[0088] Outdoor scenes: covering urban streets (different architectural styles, road materials, vehicle and pedestrian traffic, etc.), parks (different vegetation types, topography, water characteristics, etc.), mountainous areas (different mountain trends, rock materials, vegetation coverage, etc.), beaches (different beach textures, sea water colors and clarity, surrounding reef conditions, etc.). The lighting conditions of outdoor scenes are complex and diverse, including direct sunlight, shadow areas, and changes in light angles at different times.

[0089] Special scenes: Some special scenes are also considered, such as factory workshops (with various large-scale machinery and equipment, different production process areas, specific lighting and ventilation conditions, etc.), stadiums (different venue materials, audience seating layouts, lighting and sound equipment, etc.), hospital wards (different medical equipment, bed layouts, disinfection and cleaning requirements, etc.). These special scenes have unique environmental characteristics and rendering requirements.

[0090] Device status differences

[0091] Hardware performance differences: Collect rendering samples under different graphics processing unit (GPU) performance. For example, high-end GPUs such as the NVIDIA RTX series have a large number of CUDA cores and high memory bandwidth, which can handle complex graphics calculations and high-resolution rendering. Low-end GPUs such as some integrated graphics cards have limited computing power and memory bandwidth, and may experience freezes or image quality degradation when processing complex scenes. At the same time, consider the impact of different CPU performance on rendering, because the CPU is also involved in the processing of some rendering tasks, such as data preprocessing and logic control.

[0092] Device configuration differences: Consider other device configuration factors, such as memory size (from a small 4GB to a large 32GB or more), storage type (such as traditional hard disk vs. solid-state drive), and display resolution (from common 1920×1080 to high-resolution 3840×2160 or even 8K resolution). Different memory sizes may affect the cache and reading speed of rendering data, storage types will affect data transfer speed, and display resolution is directly related to the quality of the rendering output and the required sampling rate.

[0093] Diversity of rendering requirements

[0094] Image quality requirements: Samples are collected for different image quality requirements. For example, for film and television production and high-end advertising design, extremely high image quality is required, including realistic color reproduction, delicate detail expression, accurate light and shadow effects, etc. For some mobile applications and web games, more attention may be paid to rendering speed and resource optimization, and rendering time and device resource consumption should be reduced as much as possible while ensuring a certain image quality.

[0095] Real-time requirements: Consider the difference in requirements between real-time rendering and non-real-time rendering. Real-time rendering requires high-quality images to be generated in a short period of time (usually dozens of frames per second), such as in video games, virtual reality (VR), and augmented reality (AR) applications. Non-real-time rendering focuses more on the quality of the final image and may take a long time to render, such as in film and television special effects production and architectural design rendering.

[0096] (II) Perform quality assessment on the rendering samples and remove the rendering samples that fail the quality assessment

[0097] Quality Assessment Metrics

[0098] Structural Similarity (SSIM): SSIM measures sample quality by comparing the rendering result with the reference image in terms of brightness, contrast, and structure. Its calculation formula is relatively complex and involves statistics such as the mean, variance, and covariance of the image. The value of SSIM is between [-1, 1]. The closer it is to 1, the higher the similarity between the rendered sample and the reference image, and the better the quality.

[0099] Peak Signal-to-Noise Ratio (PSNR): PSNR is based on the principle of signal processing and evaluates sample quality by calculating the ratio of the peak power of the signal to the noise power. Its calculation formula is: PSNR = 10*log10((MAX^2) / MSE), where MAX is the maximum possible value of the pixel value in the image (for 8-bit images, MAX = 255), and MSE is the mean square error. The higher the PSNR value, the higher the quality of the sample.

[0100] Evaluation Process

[0101] For each collected rendering sample, compare it with the corresponding reference image (if any) or a high-quality benchmark image determined according to industry standards. Calculate its SSIM value and PSNR value respectively. If the SSIM value is lower than a certain threshold (such as 0.8) and the PSNR value is lower than another threshold (such as 30dB), the sample is considered to have failed the quality assessment and is removed. In actual operation, the choice of threshold may be adjusted according to specific application scenarios and requirements.

[0102] Purpose

[0103] By removing low-quality samples, we ensure that the training data used to train the neural network model has high quality. High-quality training data helps improve the learning effect and prediction accuracy of the model, and prevents the model from being misled by low-quality data, so that it can better learn the correct relationship between different scenes, device states, rendering requirements and the optimal sampling rate.

[0104] (III) Mark the optimal sampling rate corresponding to the rendered sample

[0105] How to determine the best sampling rate

[0106] Method based on manual experience: When collecting samples, professional rendering engineers directly determine the optimal sampling rate for each rendering sample based on their own experience and judgment of the scene, device status, and rendering requirements. This method relies on the knowledge and experience of professionals, but may be somewhat subjective.

[0107] Algorithm search-based method: some algorithms are used to search for the best sampling rate. For example, a method based on exhaustive search can be used to try different sampling rates within a certain range (for example, from the minimum sampling rate to the maximum sampling rate, with a certain step size), and then determine the best sampling rate based on the final rendering quality evaluation results (such as SSIM and PSNR values). This method is relatively objective, but may require a long computing time and more computing resources.

[0108] importance

[0109] Labeling the optimal sampling rate is a key step in training the neural network model. The model learns the relationship between the rendering samples and their corresponding optimal sampling rates, so that it can accurately predict the optimal sampling rate based on the input scene and rendering quality target in the subsequent prediction process. Accurate labeling ensures that the model learns the correct mapping relationship, improving the prediction ability and practicality of the model.

[0110] 2. Training the neural network model based on the training data

[0111] (I) Convolutional Neural Network Structure

[0112] Local connections and weight sharing

[0113] Convolutional neural networks have the characteristics of local connection and weight sharing. In the convolution layer, neurons are only connected to local areas of the input image, and the same convolution kernel uses the same weight parameters when sliding over the entire image. This structural design greatly reduces the number of model parameters and computational complexity, while being able to extract common feature patterns of the image. For example, for a two-dimensional image, the convolution kernel can extract basic features such as edges, lines, and textures in the image, and the way these features are extracted is consistent across the entire image.

[0114] Hierarchical feature extraction

[0115] Convolutional neural networks are usually composed of multiple convolutional layers, pooling layers, and fully connected layers. Through a layer-by-layer approach, more and more abstract and advanced academic papers are extracted from the original image data.

[0116] In the lower convolution layers, the basic features of the image are mainly extracted, such as edges, lines, textures, etc. As the number of layers increases, more complex concepts can be extracted in the higher convolution layers, such as the shape, structure, and combination of objects. This hierarchical feature extraction method is consistent with the human visual system's cognitive process of images, and can better understand and process complex scene information in rendering samples, thereby providing more valuable color differences, different materials, different lighting conditions and other complex information for accurately predicting the optimal sampling rate, thereby providing more valuable feature representation for accurately predicting the optimal sampling rate.

[0117] (II) Training process

[0118] Model initialization

[0119] Choose the appropriate convolutional neural network structure: Choose the appropriate convolutional neural network structure according to the characteristics of the rendered sample and the features to be extracted. For example, for rendered samples with a lot of details in the image, you can choose a structure with more convolutional layers and smaller convolutional kernels to better extract detail features. For rendered samples with more large objects and simple structures in the image, you can choose a structure with fewer convolutional layers and larger convolutional kernels to extract the main features faster.

[0120] Initialize model parameters: Initialize the selected convolutional neural network. You can use random initialization, that is, randomly generate weight values ​​and bias values ​​within a certain range. You can also use pre-trained model weights for initialization, and then fine-tune them according to specific application scenarios and needs. For example, if there is a pre-trained model related to the current training task, you can use its weights as the initial value, and then fine-tune it through training data, which can speed up training and improve prediction accuracy.

[0121] Input training data

[0122] The labeled rendering samples and their corresponding optimal sampling rates are input into the convolutional neural network as training data. Specifically, the scene and rendering quality target are used as input, and the optimal sampling rate is used as output. In this way, the model can predict the optimal sampling rate under different scenes and rendering quality targets by learning the relationship between input and output.

[0123] Parameter Adjustment

[0124] The model parameters are adjusted by back-propagation algorithm and gradient descent optimization method. First, the gradient of the loss function to the model parameters is calculated. The loss function is a function used to measure the gap between the model's predicted results and the actual results. Commonly used loss functions include mean square error (MSE). Then, the error is back-propagated from the output layer to the input layer, and the model parameters are adjusted according to the direction of gradient descent. In this process, the learning rate is a very important parameter. If the learning rate is too large, the model may skip the optimal solution, resulting in unstable training; if the learning rate is too small, the training process will be very slow. Therefore, it is necessary to select an appropriate learning rate according to the specific application scenario and requirements.

[0125] Convergence criteria

[0126] Continue training until the model reaches a certain convergence criterion. The convergence criterion can be that the value of the loss function is lower than a certain threshold, or that the loss remains basically unchanged in several consecutive training iterations.

[0127] For example, when the value of the loss function is less than 0.01, or the change in the loss function is less than 0.001 in 10 consecutive training iterations, the model can be considered to have reached the convergence criterion. When the model reaches the convergence criterion, it can be considered that the model has learned enough knowledge to be used for prediction tasks.

[0128] 3. Training effectiveness evaluation and supervision mechanism

[0129] Evaluation Methodology

[0130] Use a portion of reserved validation data (split out from the original training data) to evaluate the prediction accuracy of the model. Input the validation data into the trained convolutional neural network and calculate the prediction error of the model on the validation data. Common prediction error metrics are mean square error (MSE) and mean absolute error (MAE). By calculating these error metrics, you can understand the prediction accuracy of the model and make adjustments as needed.

[0131] Supervision Mechanism

[0132] If the evaluation results show that the prediction accuracy of the model is not ideal, the training process needs to be adjusted. This may include adjusting the learning rate, increasing the amount or diversity of training data, adjusting the model structure such as adding or reducing the number of network layers, etc. By continuously adjusting and supervising the training process, the prediction accuracy and generalization ability of the model can be improved, so that it can better adapt to the optimal sampling rate prediction task under different scenarios and rendering requirements.

[0133] In some embodiments, the principles of disease prediction by the neural network model include:

[0134] In complex scenarios, increase the sampling rate;

[0135] In simple scenarios, reduce the sampling rate;

[0136] Further adjust the sampling rate based on the computing power and frame rate limitations of the device.

[0137] The following is a detailed explanation of the sampling rate prediction principles for neural network models:

[0138] 1. Increase the sampling rate in complex scenes

[0139] Feature recognition in complex scenes

[0140] The neural network model analyzes the input scene information and identifies the features of complex scenes. Complex scenes may contain a large number of details, such as object surfaces with rich textures, complex geometric shapes, and the overlap and interaction of multiple objects. For example, in a busy city street scene, there are numerous buildings, vehicles, pedestrians, and various billboards and logos, which constitute a complex visual environment.

[0141] There may also be complex lighting conditions, such as mixed illumination from multiple light sources, reflection and refraction effects, etc. For example, in an indoor showroom scene, there may be multiple spotlights, chandeliers, and natural light from windows, which produce complex reflections and refractions on the surface of objects, increasing the complexity of the scene.

[0142] Reasons and Effects of Increasing Sampling Rate

[0143] When complex scenes are identified, the sampling rate is increased to better capture these details and complex lighting effects. A higher sampling rate can make the rendering results more realistic and present more detailed information, such as tiny textures on the surface of objects, gradients of shadows, etc. This helps improve image quality and meet the demand for high-quality rendering. For example, in film and television special effects production or high-end advertising design, complex scenes require high-precision rendering to show their visual effects.

[0144] 2. Reduce the sampling rate in simple scenarios

[0145] Feature recognition in simple scenes

[0146] The neural network model is also able to recognize the features of simple scenes. Simple scenes usually have fewer objects, and the shapes and structures of the objects are relatively simple. For example, an empty white room with only a few simple pieces of furniture has a low scene structure and low object complexity.

[0147] Lighting conditions may also be relatively simple, such as with only one uniform light source. For example, a small conference room illuminated by only one chandelier has a relatively uniform light distribution without complex reflection and refraction effects.

[0148] Reasons and Effects of Reducing Sampling Rate

[0149] Lowering the sampling rate in simple scenes can reduce unnecessary calculations and increase rendering speed. Since simple scenes are inherently less complex, they do not require high sampling rates to render details. Lower sampling rates are sufficient to meet image quality requirements, while saving computing resources and improving rendering efficiency. This is very important for some applications with high real-time requirements, such as video games or real-time interactive systems, to ensure that the system can quickly generate images and provide a smooth user experience.

[0150] 3. Further adjust the sampling rate based on the computing power of the device and frame rate limitations

[0151] Impact of device computing power

[0152] The computing power of different devices varies. For example, high-end graphics processors (GPUs) have strong computing power and can handle higher sampling rates, while low-end integrated graphics cards have weaker computing power and may not be able to withstand excessively high sampling rates. The neural network model adjusts the sampling rate based on the computing power of the device. For devices with strong computing power, the sampling rate can be appropriately increased to obtain better image quality; for devices with weak computing power, the sampling rate needs to be reduced to avoid lag or failure to render properly.

[0153] Effects of frame rate limiting

[0154] Frame rate refers to the frequency at which the image is updated on the screen. For some real-time rendering applications, certain frame rate requirements need to be met. For example, video games usually require 30 or 60 frames per second or more. If the sampling rate is too high, the frame rate may drop, affecting the user experience. The neural network model will take into account the frame rate limitation factor and adjust the sampling rate based on meeting the frame rate requirements. For example, if the current frame rate is lower than the required frame rate, the sampling rate may need to be reduced to increase the frame rate to ensure that the image can be updated in time and provide a smooth visual experience.

[0155] Comprehensive adjustment methods and effects

[0156] The neural network model comprehensively considers the computing power and frame rate limitation factors of the device, and achieves a balance between image quality and rendering efficiency by dynamically adjusting the sampling rate. When the computing power of the device allows and the frame rate meets the requirements, the sampling rate can be appropriately increased to improve image quality; when the computing power of the device is limited or the frame rate is insufficient, the sampling rate needs to be reduced to increase the rendering speed and ensure the stability of the frame rate. This comprehensive adjustment method enables the rendering system to achieve better performance on different devices and meet the needs of different users.

[0157] In some embodiments, hardware acceleration and distributed computing may be used to increase computing speed.

[0158] 1. Hardware Acceleration

[0159] Graphics Processing Unit (GPU) Acceleration Principle

[0160] GPU has powerful parallel computing capabilities and is suitable for accelerating deep learning calculations, such as the calculation process of neural network models. Compared with traditional central processing units (CPUs), GPUs have a large number of computing cores and can process multiple data at the same time. In neural network models, for example, in the convolution operation of the convolution layer and the matrix multiplication operation of the fully connected layer, GPUs can process multiple data in parallel, greatly improving the calculation speed.

[0161] Take the convolutional neural network (CNN) as an example. In the convolution layer of CNN, convolution operations need to be performed on each local area of ​​the input image. GPU can perform convolution on multiple local areas at the same time, instead of processing sequentially like CPU. This parallel processing method greatly improves computing efficiency.

[0162] Application of GPU programming framework

[0163] Use GPU programming frameworks (such as CUDA or OpenCL) to write parallel computing code to effectively utilize the performance of the GPU. These programming frameworks provide a series of functions and tools that allow developers to easily program on the GPU. For example, CUDA provides a programming environment similar to C language. Developers can use CUDA functions and instructions to write code to control and utilize the GPU. Through these programming frameworks, developers can distribute the computing tasks of the neural network model to the various computing cores of the GPU to achieve efficient parallel computing.

[0164] The effect of hardware acceleration

[0165] By using GPU for hardware acceleration, the computational speed of neural network models can be significantly improved. For large-scale neural network models and large amounts of training data, hardware acceleration can significantly shorten training time and prediction time. For example, when training a complex convolutional neural network model for image recognition, using GPU acceleration can reduce training time from several days to a few hours or even shorter. This enables researchers and developers to train and optimize models faster and improve work efficiency.

[0166] 2. Distributed Computing

[0167] Principles of Distributed Computing

[0168] Distributed computing is to decompose a computing task into multiple subtasks and assign these subtasks to multiple computing nodes (such as different computers or servers) for parallel computing. In a neural network model, for example, during the training process, the training data can be divided into multiple parts, and each part is assigned to a computing node for training. Then, each computing node summarizes and integrates its own training results to obtain the final training results.

[0169] Take TensorFlow's distributed training function as an example. It can distribute the model's inference process to multiple devices or computing nodes for calculation. In this process, data transmission and communication are required between different computing nodes to ensure that each node can work together. By reasonably arranging data transmission and communication methods, the efficiency of distributed computing can be improved.

[0170] Application of distributed computing framework

[0171] Distributed computing is achieved by using distributed computing frameworks (such as TensorFlow's distributed training function). These frameworks provide a series of functions and tools for managing and coordinating the work between various computing nodes. For example, TensorFlow's distributed training function can automatically distribute training data to various computing nodes, and can monitor and manage the training progress of each node. Through these frameworks, developers can easily implement distributed computing and improve computing efficiency.

[0172] The Effect of Distributed Computing

[0173] Distributed computing can further improve the computing speed of neural network models. By distributing computing tasks to multiple computing nodes for parallel computing, the computing time can be greatly shortened. In particular, for large-scale neural network models and massive training data, the advantages of distributed computing are more obvious. For example, when training a very large neural network model for natural language processing, using distributed computing can reduce the training time from several weeks to a few days, or even shorter. At the same time, distributed computing can also improve the generalization ability of the model, because each computing node can process different parts of the data, allowing the model to learn a wider range of knowledge.

[0174] 3. Software and Hardware Collaborative Optimization

[0175] Model design adapted to hardware architecture

[0176] In the process of hardware acceleration and distributed computing, software and hardware coordinated optimization is required. Among them, model design that adapts to the hardware architecture is an important part. For example, when using GPU for hardware acceleration, the structure of the neural network model needs to be designed according to the characteristics of the GPU. For GPU, its memory access speed is relatively slow, so when designing the model, the number of memory accesses of the model can be reduced. For example, some memory-efficient network structures, such as some variants of ResNet, can be used to improve computing efficiency by reducing the number of memory accesses.

[0177] In distributed computing, the communication cost between each computing node needs to be considered. Therefore, when designing the model, some distributed-friendly network structures can be adopted, such as Distributed Batch Normalization. These network structures can reduce the communication cost between each computing node and improve the efficiency of distributed computing.

[0178] Algorithm selection and optimization

[0179] It is also important to choose algorithms that are suitable for hardware acceleration and distributed computing. For example, in the GPU acceleration process, mixed precision computing can improve computing efficiency. Mixed precision computing means using both single-precision and double-precision data types during the calculation process, switching between them as needed. This method can improve the calculation speed while ensuring the calculation accuracy.

[0180] In distributed computing, the use of some efficient data transmission and communication algorithms can improve the efficiency of distributed computing. For example, the use of asynchronous transmission can reduce waiting time and improve data transmission efficiency. At the same time, the use of some efficient communication protocols, such as MPI (Message Passing Interface) or RPC (Remote Procedure Call), can improve the communication efficiency between various computing nodes.

[0181] Data transmission and communication optimization

[0182] In the process of hardware acceleration and distributed computing, data transmission and communication are key links. Optimizing data transmission and communication can improve computing efficiency. For example, in the process of GPU acceleration, the data transmission time can be reduced by optimizing the data transmission path inside the GPU. In distributed computing, the communication cost and time can be reduced by optimizing the connection method and communication path between each computing node. For example, a high-speed network such as InfiniBand can be used to connect each computing node to improve data transmission and communication efficiency.

[0183] The effect of hardware and software collaborative optimization

[0184] Through software-hardware co-optimization, the computational speed and efficiency of neural network models can be further improved. Model design and algorithm selection that adapt to the hardware architecture can improve computational efficiency, while optimizing data transmission and communication can reduce costs and time. For example, through software-hardware co-optimization, the computational speed of a neural network model for image recognition can be increased by 2-3 times, while the cost can also be reduced. This co-optimization method can enable neural network models to function better in hardware acceleration and distributed computing environments, improving computing performance.

[0185] In some embodiments, the method further includes obtaining feedback information;

[0186] The neural network model is retrained based on the feedback information.

[0187] 1. Get feedback

[0188] Feedback source

[0189] In the real-time rendering process, feedback information is obtained by evaluating the quality of the rendering results. Specifically, the actual rendering results are compared with the expected rendering quality to obtain the differences and errors, which are the feedback information.

[0190] For example, when rendering a complex indoor scene, the expected rendering quality may be to clearly present the details of various objects in the room and accurate lighting effects. If the details of some objects are lost or the lighting effects are inaccurate in the actual rendering results, the information corresponding to these situations that do not meet expectations constitutes feedback information.

[0191] Content of feedback information

[0192] The feedback includes differences in image quality, such as the difference between the structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) and the expected standard. If the SSIM value is lower than expected or the PSNR value is lower than expected, it indicates that there is a problem with the image quality. The specific values ​​of these indicators and the difference from the expected values ​​constitute part of the feedback information.

[0193] It also includes information related to scene features. For example, if an object that should have been highlighted in a scene is not rendered correctly, then relevant information such as the feature information of the object and its position in the scene will also be included in the feedback information.

[0194] 2. Retraining the neural network model based on feedback information

[0195] Data collection and updating

[0196] Use the feedback to collect new high-quality rendering samples. For example, if the feedback indicates that there are problems with the rendering of a certain indoor scene, you can collect more targeted samples for this scene. You can adjust rendering parameters such as sampling rate, lighting settings, etc., and re-render the scene to obtain new samples.

[0197] These new samples are added to the original training data set to update the training data set. This makes the training data set richer and more comprehensive, covering more scene conditions and rendering problems, and helps improve the generalization ability of the neural network model.

[0198] Model updates and iterations

[0199] Retrain the neural network model using the updated training data set. During the retraining process, the model adjusts its weights and parameters based on the new data. For example, for a convolutional neural network (CNN), the weights of the convolution kernel and the parameters of the fully connected layer are adjusted based on the relationship between the new sample and the corresponding rendering quality target through the back-propagation algorithm and the gradient descent optimization method.

[0200] Through continuous model updates and iterations, the neural network model can better adapt to changes in different scenarios and rendering requirements. Each retraining is an optimization of the model, making its predicted sampling rate more accurate, thereby improving the quality and efficiency of real-time rendering.

[0201] The following is a comprehensive description of the solutions provided by this application in combination with the above preferred embodiments:

[0202] The present invention provides an AI-based solution for improving dynamic adaptive sampling rate, comprising:

[0203] AI model training: Using deep learning technology, we collect a large number of high-quality rendering samples under different scenes and rendering requirements as training data. By training the neural network model, it can automatically predict the optimal sampling rate based on the input scene and rendering quality target.

[0204] 1) Data collection: Collect rendering samples under different scenes and rendering requirements. This can be achieved by rendering under various environmental conditions and recording the sampling rate and the corresponding rendering results. The samples should cover as many scenes as possible, including indoor and outdoor, different lighting conditions, different materials and textures, etc.

[0205] 2) Quality assessment: In order to obtain high-quality rendering samples, it is necessary to evaluate the quality of the samples. Professional rendering evaluation indicators: structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) are used to evaluate the similarity between the rendering results and the reference images.

[0206] 3) Data preprocessing: Preprocess the collected rendering samples. This may include image enhancement, data alignment and standardization. Preprocessing can make the quality of training data more consistent and improve the training effect of the model.

[0207] 4) Network model design: Design a neural network model suitable for dynamic adaptive sampling rate prediction. A convolutional neural network (CNN) is used to predict the optimal sampling rate by learning the relationship between the input scene and the rendering quality target.

[0208] 5) Model training: Using the collected and preprocessed training data, the neural network model is trained by pairing the input scene and rendering quality target with the actual optimal sampling rate. The deep learning framework TensorFlow is used to implement model training.

[0209] 6) Parameter tuning: During the training process, the model parameters need to be tuned to improve the prediction accuracy and generalization ability. The back propagation algorithm and gradient descent optimization method can be used to minimize the loss function of the model.

[0210] Dynamic adaptive sampling rate algorithm: Based on the output of the AI ​​model, a dynamic adaptive sampling rate algorithm is designed. The algorithm can automatically adjust the sampling rate according to factors such as input scene information, rendering quality, and device performance. In complex scenes, the sampling rate is increased to improve rendering details and ensure image quality; in simple scenes, the sampling rate is reduced to speed up rendering.

[0211] 1) Input parameter acquisition: Obtain scene information, rendering quality, device performance and other parameters during the rendering process. These parameters may include scene information such as lighting conditions, object complexity, motion blur, rendering quality targets (such as image detail requirements), and device computing power and frame rate limits.

[0212] 2) AI model output: Input parameters are fed into the trained AI model to obtain its output. The AI ​​model will predict the optimal sampling rate based on the input scene information, rendering quality target, etc.

[0213] 3) Sampling rate adjustment strategy: Based on the output of the AI ​​model, a dynamic adaptive sampling rate algorithm is designed to automatically adjust the sampling rate. According to the scene information and rendering quality requirements, it is determined whether the sampling rate needs to be increased or decreased.

[0214] 4) Increase the sampling rate for complex scenes: In complex scenes, the rendering details can be improved by increasing the sampling rate. For example, for scenes with a lot of details and reflection / refraction effects, more details and light and shadow effects can be captured by increasing the sampling rate.

[0215] 5) Reduce the sampling rate for simple scenes: In simple scenes, the sampling rate can be reduced to speed up rendering while maintaining reasonable image quality. For example, for scenes with relatively uniform lighting and relatively simple objects, the sampling rate can be appropriately reduced to reduce the amount of calculation.

[0216] 6) Device performance considerations: Further adjust the sampling rate based on factors such as the device's computing power, frame rate limit, etc. On devices with lower performance, the sampling rate can be adjusted based on the device's performance limitations to ensure the stability of the rendering speed.

[0217] Real-time prediction and update: During real-time rendering, the AI ​​model is used to predict the optimal sampling rate in real time and dynamically update it during rendering. Through continuous feedback and learning, the AI ​​model is further optimized and adapted to changes in different scenarios and rendering requirements.

[0218] 1) Real-time prediction: During the real-time rendering process, the input scene information, rendering quality, device performance and other parameters are provided to the trained AI model. Based on these inputs, the AI ​​model predicts the optimal sampling rate in real time.

[0219] 2) Dynamic update: According to the prediction results of the AI ​​model, the sampling rate in the current rendering process is updated in a timely manner. This can be done by interacting with the interface of the rendering system and passing the new sampling rate value to the rendering engine to adjust the sampling rate of the current frame in real time.

[0220] 3) Continuous feedback and learning: Feedback information can be obtained by performing quality assessment on real-time rendering results. The actual rendering results generated by the rendering system are compared with the expected rendering quality to obtain differences and errors. These differences and errors are used as feedback signals for learning and to improve the prediction accuracy of the AI ​​model.

[0221] 4) Data collection and update: Use the feedback information obtained during real-time rendering to collect new high-quality rendering samples. Add these samples to the training data set for subsequent model updates. By collecting more samples and retraining the AI ​​model, the model's predictive ability is continuously optimized.

[0222] 5) Model update and iteration: Retrain the AI ​​model with new training data and update the model weights and parameters. Through continuous model update and iteration, the model can better adapt to changes in different scenarios and rendering requirements.

[0223] Hardware acceleration and distributed computing: Use existing hardware acceleration technologies, such as GPUs, and combine them with distributed computing to improve the computing performance of AI models. By accelerating the model inference process, efficient applications in real-time rendering can be achieved.

[0224] 1) Hardware acceleration: Use existing hardware acceleration technologies, such as graphics processing units (GPUs), to accelerate the calculation process of AI models. GPUs have powerful parallel computing capabilities and are suitable for accelerating deep learning calculations. Use GPU programming frameworks (such as CUDA or OpenCL) to write parallel computing codes to effectively utilize the performance of GPUs.

[0225] 2) Model inference optimization: Optimize the inference process of the AI ​​model to improve computing efficiency and real-time rendering speed. Optimization includes reducing the computational complexity of the model, reducing memory access, using mixed precision calculations, etc. By reducing unnecessary calculations and memory operations, the model inference speed is accelerated.

[0226] 3) Distributed computing: Combined with distributed computing technology, the computing tasks of the AI ​​model are assigned to multiple computing nodes for parallel execution to further improve computing performance. Distributed computing frameworks, such as the distributed training function of TensorFlow, can be used to distribute the model's inference process to multiple devices or computing nodes for calculation, thereby achieving parallel computing acceleration.

[0227] 4) Software-hardware co-optimization: In the process of hardware acceleration and distributed computing, software-hardware co-optimization is performed. This includes model design and algorithm selection that adapt to the hardware architecture to give full play to the advantages of hardware acceleration. At the same time, by optimizing data transmission and communication, the latency and overhead of distributed computing are reduced.

[0228] 5) Performance evaluation and tuning: Determine the effectiveness of hardware acceleration and distributed computing solutions through performance evaluation and tuning. You can use performance analysis tools and monitoring tools, NVIDIA's Nsight and TensorBoard, to analyze the computing performance and bottlenecks of the model and further optimize the system's throughput and computing efficiency.

[0229] Feedback optimization: Compare the actual rendering effect with the expected target and further optimize the accuracy of the prediction model through continuous learning.

[0230] 1) Evaluation of actual rendering effect: In real-time rendering, the predicted sampling rate is used for rendering and the final rendering result is generated. For each frame of rendering result, image quality evaluation indicators (such as SSIM or PSNR) can be used for objective evaluation, or feedback can be obtained through user subjective evaluation.

[0231] 2) Compare with expected target: Compare the actual rendering effect with the expected target to determine the accuracy and error of the prediction model. If there is a difference between the actual rendering result and the expected target, it means that the model's prediction may not be accurate enough.

[0232] 3) Difference and error analysis: Compare the actual rendering results with the expected targets and further analyze the reasons for the differences and sources of errors. This can include locating different factors in the rendering process, such as specific scene types, rendering quality requirements, device performance changes, etc.

[0233] 4) Model parameter update: Based on the actual rendering results and feedback from the difference analysis, the parameters or structure of the model can be updated and adjusted. By using optimization algorithms such as gradient descent, back propagation is performed based on the error signal, and the weights and parameters of the model are adjusted to make the prediction more accurate and optimize the rendering effect.

[0234] 5) Continuous learning and iteration: Through the iterative process of feedback optimization, new training data and model updates are continuously used to further improve the accuracy of predictions. New high-quality rendering samples can be collected periodically and the model can be retrained using these samples to adapt to changing scenes and rendering requirements.

[0235] The solution provided by this application has the following effects:

[0236] 1. Improve rendering efficiency: Through the application of intelligent prediction and dynamic adaptive sampling rate algorithm of AI model, the sampling rate can be automatically adjusted according to scene changes and rendering requirements to improve rendering efficiency. During real-time rendering, the sampling rate can be increased or decreased as needed to increase rendering speed.

[0237] 2. Improve image quality: Through the intelligent adjustment of the sampling rate by the AI ​​model, the sampling rate can be increased in complex scenes to improve rendering details and image quality, achieving more realistic rendering effects without sacrificing performance.

[0238] 3. Adaptability and flexibility: Due to the real-time prediction and update of the AI ​​model, this solution can adapt to the dynamic changes of different scenarios and rendering requirements.

[0239] Possess good adaptability and flexibility.

[0240] 4. Improve the practicality of AI technology application: By applying AI technology to real-time rendering, this solution can further promote the application of AI technology in the field of graphics rendering and improve the practicality and user experience of real-time rendering.

[0241] Exemplary Electronic Devices

[0242] Below, an electronic device according to an embodiment of the present application is described with reference to the drawings. Figure 2 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0243] like Figure 2 As shown, the electronic device 200 includes one or more processors 210 and a memory 220 .

[0244] The processor 210 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 200 to perform desired functions.

[0245] The memory 220 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 210 may run the program instructions to implement the dynamic adaptive sampling rate optimization method based on artificial intelligence and / or other desired functions of the various embodiments of the present application described above. Various contents such as category correspondences may also be stored in the computer-readable storage medium.

[0246] In one example, the electronic device 200 may further include: an input device 230 and an output device 240 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0247] In addition, the input device 230 may also include, for example, a keyboard, a mouse, an interface, etc. The output device 240 may output various information to the outside, including analysis results, etc. The output device 240 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0248] Of course, to simplify, Figure 2Only some of the components in the electronic device related to the present application are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application conditions.

[0249] Exemplary computer program products and computer-readable storage media

[0250] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the dynamic adaptive sampling rate optimization method based on artificial intelligence according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0251] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0252] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps of the dynamic adaptive sampling rate optimization method based on artificial intelligence according to various embodiments of the present application described in the above “Exemplary Method” section of this specification.

[0253] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0254] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A dynamic adaptive sampling rate optimization method based on artificial intelligence for real-time rendering system, characterized in that: The method includes: Building and training a neural network model, the neural network model is used to automatically predict the optimal sampling rate according to the input scene and rendering quality target; constructing a dynamically adaptive sampling rate algorithm for automatically adjusting the sampling rate based on the output of the neural network model; During the real-time rendering process, the neural network model is used to predict the optimal sampling rate in real time, and the rendering sampling rate is adjusted to the optimal sampling rate based on the dynamic adaptive sampling rate algorithm during rendering.

2. The method for optimizing the sampling rate based on artificial intelligence according to claim 1, characterized in that: The training neural network model comprises: Collect a preset number of high-quality rendering samples under different scenarios, device states, and rendering requirements as training data; The neural network model is trained based on the training data.

3. The method for optimizing the sampling rate based on dynamic adaptiveness based on artificial intelligence according to claim 2, characterized in that: The collecting of a preset number of high-quality rendering samples under different scenes, device states and rendering requirements as training data includes: Collect rendering samples under different scenarios, device states, and rendering requirements; Perform quality assessment on rendering samples and remove rendering samples that fail the quality assessment; Marks the rendering samples corresponding to the optimal sampling rate.

4. The method for optimizing the sampling rate based on dynamic adaptiveness based on artificial intelligence according to claim 1, characterized in that: The neural network model is a convolutional neural network.

5. The method for optimizing the sampling rate based on dynamic adaptiveness based on artificial intelligence according to claim 1, characterized in that: In the real-time rendering process, the neural network model is used to predict the optimal sampling rate in real time, and the rendering sampling rate is adjusted to the optimal sampling rate based on the dynamic adaptive sampling rate algorithm during rendering, including: In the real-time rendering process, the input scene information, rendering quality and device performance parameters are provided to the trained neural network model to obtain the predicted optimal sampling rate; Based on the dynamic adaptive sampling rate algorithm, the sampling rate in the current rendering process is updated in time to be the optimal sampling rate.

6. The method for optimizing the sampling rate based on dynamic adaptiveness based on artificial intelligence according to claim 1, characterized in that: The principles of disease prediction by the neural network model include: In complex scenarios, increase the sampling rate; In simple scenarios, reduce the sampling rate; Further adjust the sampling rate based on the computing power and frame rate limitations of the device.

7. The method for optimizing the sampling rate based on dynamic adaptiveness based on artificial intelligence according to claim 1, characterized in that: Also includes: Use hardware acceleration and distributed computing to improve computing speed.

8. The method for optimizing the sampling rate based on dynamic adaptiveness based on artificial intelligence according to claim 1, characterized in that: Also includes: Get feedback; The neural network model is retrained based on the feedback information.

9. An electronic device, characterized in that: include: A processor, and a memory for storing a program executable by the processor; The processor is used to implement the dynamic adaptive sampling rate optimization method based on artificial intelligence as described in any one of claims 1 to 8 by running the program in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to execute the dynamic adaptive sampling rate optimization method based on artificial intelligence as described in any one of claims 1 to 8.

Images