Image rendering method and device, electronic equipment and readable storage medium
By performing weight calculation of image blocks and evaluating GPU performance load parameters, the image blocks are balanced to multiple GPUs, solving the problem of low image rendering efficiency in the prior art and achieving more efficient image rendering.
Patent Information
- Application Number
- CN202411765657.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-03-04
AI Technical Summary
In the prior art, image rendering efficiency is low, mainly due to the performance and load imbalance of different GPUs, which leads to excessive load or excessive load of some GPUs.
By extracting data from the scene model to be rendered, multiple image blocks and their weights are obtained, and the performance and load parameters of multiple GPUs are calculated. Based on these parameters, the image blocks are allocated to each GPU, making the load of each GPU more balanced. Image blocks are rendered in parallel through multiple GPUs and the intermediate rendering data are merged to obtain the final rendering result.
By balancing the image blocks, some problems of excessive or too small load of GPUs are avoided, and the efficiency of image rendering is improved.
Smart Images
Figure CN119557100A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to an image rendering method, device, electronic device and readable storage medium. Background Art
[0002] With the development of virtual reality technology, the requirements for image rendering quality are increasing. In related technologies, in order to improve the rendering quality of images, multiple graphics processors (Graphics Processing Unit, GPU) are used to segment the image and then render the segmented images through different GPUs. In related technologies, different GPUs have different performances and rendering loads. There are cases where a GPU with lower performance has a larger load, or a GPU with higher performance has a smaller load, resulting in lower image rendering efficiency.
[0003] It can be seen that the related art has the problem of low image rendering efficiency. Summary of the invention
[0004] The embodiments of the present application provide an image rendering method, an apparatus, an electronic device, and a readable storage medium to solve the problem of low image rendering efficiency in the related art.
[0005] To solve the above problems, this application is implemented as follows:
[0006] In a first aspect, an embodiment of the present application provides an image rendering method, comprising:
[0007] Extract data from the scene model to be rendered to obtain multiple image blocks and the weight corresponding to each image block;
[0008] Calculating a first parameter corresponding to each GPU in a plurality of image processors GPU, wherein the plurality of GPUs are GPUs for performing image rendering, and the first parameter is used to characterize the performance and / or load of the GPU;
[0009] Allocating the plurality of image blocks to the plurality of GPUs respectively based on the first parameter;
[0010] Rendering the image blocks based on the weights corresponding to each image block by the multiple GPUs to obtain multiple intermediate rendering data;
[0011] The plurality of intermediate rendering data are merged through the first target GPU to obtain a rendering result corresponding to the scene model.
[0012] In a second aspect, an embodiment of the present application further provides an image rendering device, including:
[0013] An extraction module is used to extract data from the scene model to be rendered, and obtain multiple image blocks and a weight corresponding to each image block;
[0014] A calculation module, used to calculate a first parameter corresponding to each GPU in a plurality of image processors GPU, wherein the plurality of GPUs are GPUs used for image rendering, and the first parameter is used to characterize the performance and / or load of the GPU;
[0015] an allocating module, configured to allocate the plurality of image blocks to the plurality of GPUs respectively based on the first parameter;
[0016] A rendering module, configured to render the image blocks based on the weights corresponding to each image block through the multiple GPUs to obtain multiple intermediate rendering data;
[0017] The merging module is used to merge the multiple intermediate rendering data through the first target GPU to obtain a rendering result corresponding to the scene model.
[0018] In a third aspect, an embodiment of the present application further provides an electronic device, including a transceiver and a processor.
[0019] The transceiver is used to extract data from the scene model to be rendered to obtain multiple image blocks and a weight corresponding to each image block;
[0020] The processor is used to calculate a first parameter corresponding to each GPU in a plurality of image processors GPU, wherein the plurality of GPUs are GPUs used for image rendering, and the first parameter is used to characterize the performance and / or load of the GPU;
[0021] The processor is further configured to allocate the plurality of image blocks to the plurality of GPUs respectively based on the first parameter;
[0022] The processor is further configured to render the image blocks based on the weights corresponding to each image block through the multiple GPUs to obtain multiple intermediate rendering data;
[0023] The processor is further configured to merge the plurality of intermediate rendering data through the first target GPU to obtain a rendering result corresponding to the scene model.
[0024] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the image rendering prediction method described in the first aspect above are implemented.
[0025] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image rendering method described in the first aspect above are implemented.
[0026] In a sixth aspect, an embodiment of the present application further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the image rendering method as described in the first aspect above.
[0027] In an embodiment of the present application, data is extracted from the scene model to be rendered to obtain multiple image blocks and a weight corresponding to each image block; a first parameter corresponding to each GPU in multiple image processors GPUs is calculated, the multiple GPUs are GPUs for image rendering, and the first parameter is used to characterize the performance and / or load of the GPU; based on the first parameter, the multiple image blocks are respectively allocated to the multiple GPUs; the image blocks are rendered by the multiple GPUs based on the weights corresponding to each image block to obtain multiple intermediate rendering data; the multiple intermediate rendering data are merged by the first target GPU to obtain the rendering result corresponding to the scene model. In this way, by calculating the first parameter of the standard GPU performance and / or load, and then allocating multiple image blocks based on the first parameter, the image blocks allocated to each GPU are more balanced, avoiding the situation where too many image blocks are allocated to a GPU and the load is too large, thereby improving the efficiency of image rendering. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0029] Figure 1 is a flowchart of an image rendering method provided by an embodiment of the present application;
[0030] Figure 2 is a schematic diagram of a first target GPU merging intermediate rendering data provided by an embodiment of the present application;
[0031] Figure 3 This is a multi-GPU parallel rendering flow chart provided in an embodiment of the present application;
[0032] Figure 4 is a structural diagram of an image rendering device provided in an embodiment of the present application;
[0033] Figure 5It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0035] See also Figure 1 , Figure 1 is a flowchart of an image rendering method provided by an embodiment of the present application, such as Figure 1 As shown, the following steps are included:
[0036] Step 101: extract data from the scene model to be rendered to obtain multiple image blocks and a weight corresponding to each image block.
[0037] The above scene model can be a three-dimensional model or a two-dimensional model. The scene model can include object models, position vectors, light source information and camera information in the scene, so that the scene model can ensure integrity. The scene model also includes material information at different positions in the model, such as texture, reflectivity and refractive index, etc., to ensure the integrity and authenticity of the scene.
[0038] The above image block is used to represent the pixel situation of an area in the scene model, and may include multiple pixels and the materials corresponding to the pixels. The weight corresponding to the above image block is used to represent the probability of the materials corresponding to different pixels in the image block.
[0039] In an embodiment of the present application, data is extracted from a scene model to be rendered to obtain multiple image blocks to be rendered by the GPU. The GPU can render the image blocks based on weights corresponding to the image blocks to achieve image rendering of the scene model.
[0040] Step 102: Calculate a first parameter corresponding to each GPU in a plurality of graphics processing units (GPUs), where the plurality of GPUs are GPUs for performing image rendering, and the first parameter is used to characterize the performance and / or load of the GPU.
[0041] It should be noted that when multiple GPUs are used to render a scene model, different types of GPUs have different performances, and their ability to render images also differs. At the same time, when multiple GPUs are used to render a scene model, there may be other tasks that need to be performed, that is, there are other loads, and a high load will also affect the image rendering capability. Therefore, in this application, a first parameter corresponding to each GPU in a plurality of GPUs is calculated, and the performance and / or load of the GPU is characterized by the first parameter, so that when the image blocks are distributed through the first parameter, they can be distributed to multiple GPUs in a more balanced manner, thereby ensuring the efficiency of each GPU in rendering the image blocks.
[0042] Step 103: Allocate the multiple image blocks to the multiple GPUs respectively based on the first parameters.
[0043] In some embodiments, allocating multiple image blocks based on the first parameter may be allocating multiple image blocks simultaneously based on the first parameter, or may be allocating multiple image blocks sequentially based on the first parameter. It should be noted that different allocation methods may be selected according to the specific image block allocation process.
[0044] In some embodiments, multiple image blocks are allocated simultaneously based on a first parameter, and the multiple image blocks can be evenly allocated to multiple GPUs based on the first parameter, so that a GPU with a higher first parameter among the multiple GPUs is allocated more image blocks, and a GPU with a lower first parameter among the multiple GPUs is allocated fewer image blocks. For example, the number of multiple image blocks is calculated based on a preset algorithm to obtain multiple groups of image blocks in decreasing numbers, each group of image blocks corresponding to a GPU, and then the multiple groups of image blocks are sorted in descending order based on the number, and the descending groups of image blocks are allocated to different GPUs.
[0045] For example, the number of image blocks is 10 and the number of GPUs is 4. The image blocks are divided into 4 groups through a preset algorithm. The number of image blocks in each group is 4, 3, 2, and 1 respectively. A group of 4 image blocks is assigned to the GPU with the highest first parameter, a group of 3 image blocks is assigned to the GPU with the second highest second parameter, and so on, until the image blocks of all groups are assigned.
[0046] In some embodiments, multiple image blocks are allocated in sequence based on the first parameter, and the resource parameters required for rendering each image block can also be calculated, and the multiple image blocks are allocated to multiple GPUs based on the first parameter and the resource parameters of the image blocks, so that the target ratio corresponding to each GPU is the same, and the target ratio is the ratio of the sum of the resource parameters of all image blocks allocated to the GPU to the first parameter corresponding to the GPU.
[0047] Step 104: Render the image blocks by using the multiple GPUs based on the weights corresponding to each image block to obtain multiple intermediate rendering data.
[0048] Step 105: Merge the plurality of intermediate rendering data through the first target GPU to obtain a rendering result corresponding to the scene model.
[0049] It should be noted that when multiple GPUs render image blocks in parallel, in order to reduce the time and system delays for subsequent data transmission, after each GPU completes rendering, it temporarily saves its rendering results (i.e., intermediate rendering data) in its video memory, and then merges all the intermediate rendering data through the first target GPU to obtain the rendering results corresponding to the scene model, which can effectively reduce the time and system delays for subsequent data transmission.
[0050] Among them, Figure 2 As shown, each image block corresponds to a material, such as glass material, mirror material, and diffuse reflection material. After the GPU allocates the image block, each GPU starts to render its own allocated image block at the same time. Each GPU processes the rendering task independently and uses its computing power to complete the rendering of each image block to obtain the intermediate rendering data. Finally, the intermediate rendering data is merged through the first target GPU.
[0051] In some embodiments, the first target GPU may be the GPU with the lowest first parameter among multiple GPUs. In this case, the first target GPU has a lower load than other GPUs and can better perform the task of merging intermediate rendering data. Furthermore, according to the system performance and resource utilization during the merging process, the load and priority of the first target GPU are dynamically adjusted so that the first target GPU is allocated fewer image blocks or no image blocks, so as to make full use of the computing resources of the first target GPU.
[0052] In some implementations, after the plurality of intermediate rendering data are merged through the first target GPU to obtain a rendering result corresponding to the scene model, the rendering structure may be saved to the cloud for subsequent reading and use.
[0053] In an embodiment of the present application, data is extracted from the scene model to be rendered to obtain multiple image blocks and a weight corresponding to each image block; a first parameter corresponding to each GPU in multiple image processors GPUs is calculated, the multiple GPUs are GPUs for image rendering, and the first parameter is used to characterize the performance and / or load of the GPU; based on the first parameter, the multiple image blocks are respectively allocated to the multiple GPUs; the image blocks are rendered by the multiple GPUs based on the weights corresponding to each image block to obtain multiple intermediate rendering data; the multiple intermediate rendering data are merged by the first target GPU to obtain the rendering result corresponding to the scene model. In this way, by calculating the first parameter of the standard GPU performance and / or load, and then allocating multiple image blocks based on the first parameter, the image blocks allocated to each GPU are more balanced, avoiding the situation where too many image blocks are allocated to a GPU and the load is too large, thereby improving the efficiency of image rendering.
[0054] In one embodiment, calculating the first parameter corresponding to each GPU in the plurality of graphics processing units GPUs includes:
[0055] Obtain at least one of a performance parameter and a load parameter corresponding to each GPU;
[0056] Calculate a first parameter corresponding to each GPU based on at least one of a corresponding performance parameter and a load parameter;
[0057] The performance parameter includes at least one of a computing capability parameter, a video memory size, a video memory bandwidth, and a current video memory usage rate;
[0058] The load parameter includes at least one of the GPU task running time, task computing amount and task video memory usage rate.
[0059] In some implementations, when the first parameter corresponding to each GPU is calculated only by using the performance parameter or load parameter corresponding to the GPU, the performance parameter or load parameter may be directly set as the first parameter.
[0060] In some implementations, when the first parameter corresponding to each GPU is calculated using the performance parameter and load parameter corresponding to the GPU, the performance parameter and the load parameter may be weighted to obtain the first parameter. Specifically, the weighted calculation may be implemented using the following formula:
[0061] GPUscore(i)=ω0·GPU(i)+ω1·GPUwork(i)
[0062] GPU in the formula score (i) represents the first parameter corresponding to the i-th GPU, GPU(i) represents the performance parameter corresponding to the i-th GPU, GPUwork (i) represents the load parameter corresponding to the i-th GPU, w0 is the weight coefficient of the performance parameter, and w1 is the weight coefficient of the load parameter.
[0063] In some implementations, when the first parameter corresponding to each GPU is calculated using the performance parameter and load parameter corresponding to the GPU, in addition to the above formula, the product of the performance parameter and the load parameter may also be used as the first parameter.
[0064] In an embodiment of the present application, at least one of the performance parameters and load parameters corresponding to each GPU is obtained; based on at least one of the corresponding performance parameters and load parameters, a first parameter corresponding to each GPU is calculated, so that the first parameter can characterize the performance and / or load of the GPU.
[0065] In one embodiment, the performance parameter is calculated as follows:
[0066] Based on at least one of the computing capability parameter, the video memory size, the video memory bandwidth, and the current video memory usage rate, calculate and obtain a first intermediate parameter corresponding to each GPU;
[0067] Calculate the performance parameter of each GPU, where the performance parameter of each GPU is a quotient of a difference between the first intermediate parameter of each GPU and a minimum first intermediate parameter and a first target difference, where the first target difference is a difference between a maximum first intermediate parameter and a minimum first intermediate parameter corresponding to the multiple GPUs.
[0068] In some embodiments, based on at least one of the computing power parameter, video memory size, video memory bandwidth, and current video memory usage, a first intermediate parameter corresponding to each GPU is calculated. The first intermediate parameter may be obtained by weighted calculation of at least two of the computing power parameter, video memory size, video memory bandwidth, and current video memory usage.
[0069] For example, weight values are preset for the computing power parameter, video memory size, video memory bandwidth and current video memory usage, respectively, and then a weighted sum of the computing power parameter, video memory size, video memory bandwidth and current video memory usage is calculated based on the preset weight values, and the weighted sum is used as the first intermediate parameter.
[0070] In some embodiments, the first intermediate parameter corresponding to each GPU is calculated based on at least one of the computing power parameter, the video memory size, the video memory bandwidth, and the current video memory usage rate. The first intermediate parameter can also be calculated by multiplying at least two of the computing power parameter, the video memory size, the video memory bandwidth, and the current video memory usage rate.
[0071] In some implementations, the computing capability parameter, the video memory size, the video memory bandwidth, or the current video memory usage rate may also be directly set as the first intermediate parameter.
[0072] Furthermore, after the first intermediate parameter is calculated, the first intermediate parameter needs to be normalized to achieve the performance parameter after data optimization. Specifically, in the embodiment of the present application, the first target difference between the maximum first intermediate parameter and the minimum first intermediate parameter corresponding to multiple GPUs is calculated, and then the quotient of the difference between the first intermediate parameter of each GPU and the minimum first intermediate parameter and the first target difference is set as the performance parameter.
[0073] The above process can be realized by the following formula:
[0074]
[0075] In the formula, GPU(i) is the performance parameter of the i-th GPU, GPU ori (i) is the first intermediate parameter of the i-th GPU, GPU min is the smallest first intermediate parameter, GPU max is the largest first intermediate parameter.
[0076] In some embodiments, in addition to the above formula to implement processing of the first intermediate parameter, the first intermediate parameter can also be processed by zero first mean normalization, specifically, calculating the first standard deviation and the first mean of all the first intermediate parameters, and dividing the difference between the first intermediate parameter and the first mean corresponding to each GPU by the first standard deviation to obtain the performance parameter.
[0077] The above process can be realized by the following formula:
[0078]
[0079] GPU in the formula σ is the first standard deviation, GPU μ is the first mean.
[0080] Furthermore, other methods may be used to achieve normalization of the first intermediate parameter, such as nonlinear normalization.
[0081] In the embodiment of the present application, based on at least one of the computing capability parameter, video memory size, video memory bandwidth, and current video memory usage, the first intermediate parameter corresponding to each GPU is calculated; the performance parameter of each GPU is calculated, and the performance parameter of each GPU is the quotient of the difference between the first intermediate parameter of each GPU and the minimum first intermediate parameter and the first target difference, and the first target difference is the difference between the maximum first intermediate parameter and the minimum first intermediate parameter corresponding to multiple GPUs. In this way, the performance parameter corresponding to each GPU is calculated by at least one of the computing capability parameter, video memory size, video memory bandwidth, and current video memory usage.
[0082] In one embodiment, the load parameter is calculated as follows:
[0083] Calculating a second intermediate parameter corresponding to each GPU, where the second intermediate parameter is the task running time, the task computing amount, or the task video memory usage rate, or the product of at least two of the task running time, the task computing amount, and the task video memory usage rate;
[0084] Calculate the load parameter of each GPU, where the load parameter of each GPU is the quotient of the difference between the second intermediate parameter of each GPU and the minimum second intermediate parameter and the second target difference, and the second target difference is the difference between the maximum second intermediate parameter and the minimum second intermediate parameter corresponding to the multiple GPUs.
[0085] The above calculation of the second intermediate parameter corresponding to each GPU can be expressed by the following formula:
[0086] GPUload(i)=taskt*taskc*taskm
[0087] GPU in the formula load (i) is the second intermediate parameter, task t is the task running time, task c is the task computation amount, task m It is the task video memory usage.
[0088] In some implementations, in addition to the formula, the second intermediate parameter corresponding to each GPU may be calculated by weighted calculation of at least two of the task running time, the task computing amount, and the task video memory usage rate.
[0089] For example, weight values are preset for task running time, task computing amount and task video memory usage respectively, and then a weighted sum of the task running time, task computing amount and task video memory usage is calculated based on the preset weight values, and the weighted sum is used as the second intermediate parameter.
[0090] Furthermore, after the second intermediate parameter is calculated, the second intermediate parameter needs to be normalized to achieve the performance parameter after data optimization. Specifically, in the embodiment of the present application, the second target difference between the maximum second intermediate parameter and the minimum second intermediate parameter corresponding to multiple GPUs is calculated, and then the quotient of the difference between the second intermediate parameter of each GPU and the minimum second intermediate parameter and the second target difference is set as the performance parameter.
[0091] The above process can be realized by the following formula:
[0092]
[0093] GPU in the formula work (i) is the load parameter, GPU' ori (i) is the second intermediate parameter, GPU' min is the smallest second intermediate parameter, GPU' max is the largest second intermediate parameter.
[0094] In some embodiments, in addition to the above formula to implement the processing of the second intermediate parameter, the second intermediate parameter can also be processed by zero second mean normalization, specifically, the second standard deviation and the second mean of all the second intermediate parameters are calculated, and the difference between the second intermediate parameter corresponding to each GPU and the second mean is divided by the second standard deviation to obtain the performance parameter.
[0095] The above process can be realized by the following formula:
[0096]
[0097] GPU in the formula σ is the second standard deviation, GPU' μ is the second mean.
[0098] Furthermore, other methods may be used to achieve normalization of the second intermediate parameter, such as nonlinear normalization.
[0099] In an embodiment of the present application, the second intermediate parameter corresponding to each GPU is calculated, and the second intermediate parameter is the task running time, the task calculation amount, or the task memory usage rate, or the product of at least two of the task running time, the task calculation amount, and the task memory usage rate; the load parameter of each GPU is calculated, and the load parameter of each GPU is the quotient of the difference between the second intermediate parameter of each GPU and the minimum second intermediate parameter and the second target difference, and the second target difference is the difference between the maximum second intermediate parameter and the minimum second intermediate parameter corresponding to multiple GPUs. In this way, the load parameter corresponding to each GPU is calculated by at least one of the task running time, the task calculation amount, and the task memory usage rate.
[0100] In one embodiment, allocating the plurality of image blocks to the plurality of GPUs respectively based on the first parameter includes:
[0101] sorting the multiple image blocks based on the weight corresponding to each image block;
[0102] The sorted plurality of image blocks are allocated to the plurality of GPUs based on the first parameter.
[0103] It should be noted that the greater the weight corresponding to an image block, the greater the resource consumption required to render the image block. When allocating image blocks, it is necessary to allocate image blocks with larger weights to GPUs with stronger rendering capabilities as much as possible, so that the image blocks allocated to each GPU are more balanced.
[0104] In an embodiment of the present application, multiple image blocks are sorted based on the weight corresponding to each image block; and the sorted multiple image blocks are distributed to multiple GPUs based on a first parameter, so that the image blocks sliced by each GPU are more balanced.
[0105] In some embodiments, the sorted multiple image blocks are allocated to the multiple GPUs by sorting the multiple image blocks in descending order, and allocating an image block to different GPUs in order of the first parameter. After allocating the image block to the GPU with the lowest first parameter, the image block is allocated to the GPU with the highest first parameter, and the cycle is repeated until all image blocks are allocated.
[0106] In one embodiment, allocating the sorted plurality of image blocks to the plurality of GPUs comprises:
[0107] In the case of allocating a first image block among the plurality of image blocks after sorting, allocating the first image block to a GPU having the highest first parameter among the plurality of GPUs;
[0108] In the case of allocating a non-first image block among the sorted multiple image blocks, the non-first image block is allocated to a second target GPU, where the second target GPU is the GPU with the highest second parameter among the multiple GPUs, and the second parameter is a parameter recalculated by the multiple GPUs after the image blocks have been allocated.
[0109] It should be noted that after the GPU allocates an image block, the corresponding load of the GPU changes and the second parameter needs to be recalculated. The second parameter can represent the performance and / or load of the GPU after the image block is allocated, and the GPU to which the next image block is allocated is determined by the second parameter.
[0110] In the process of calculating the second parameter, only the second parameter of the image block that has been allocated needs to be recalculated, while the GPU load of the unallocated image block does not change, and the first parameter of the GPU of the unallocated image block is the same as the second parameter. In this way, only the second parameter of the image block that has been allocated is calculated, and the second parameter is calculated quickly. In this way, the second parameter of the GPU is calculated after each image block is allocated, and the execution is repeated until all image blocks are allocated.
[0111] In an embodiment of the present application, when the first image block among the plurality of image blocks after sorting is allocated, the first image block is allocated to the GPU with the highest first parameter among the plurality of GPUs; when the non-first image block among the plurality of image blocks after sorting is allocated, the non-first image block is allocated to the second target GPU, the second target GPU is the GPU with the highest second parameter among the plurality of GPUs, and the second parameter is the parameter recalculated by the plurality of GPUs after the image blocks have been allocated. In this way, by updating the second parameter corresponding to the GPU, the second parameter can more accurately reflect the situation of the GPU after the image block is allocated, so that the GPU after the image block allocated based on the second parameter is more balanced, thereby improving rendering efficiency.
[0112] In one embodiment, the data extraction of the scene model to be rendered to obtain a plurality of image blocks and a weight corresponding to each image block includes:
[0113] Generating an initial scene image based on the scene model to be rendered;
[0114] Extracting features from the initial scene image to obtain a feature map and probabilities corresponding to different pixels in the feature map, wherein the feature map includes pixels of different materials;
[0115] Convert pixels of the same material between adjacent pixels in the feature map into image blocks to obtain the multiple image blocks;
[0116] The weight corresponding to each image block is calculated based on the probabilities corresponding to different pixels in the feature map.
[0117] The above-mentioned generation of the initial scene image based on the scene model to be rendered can be to directly divide the two-dimensional model into different areas and generate the initial scene image through the different areas; or it can be to extract three-dimensional coordinates, normals, texture and other data from the three-dimensional model, and project these data onto a two-dimensional plane to generate the initial scene image.
[0118] The feature map is a feature map of the initial scene image, and the relevant information of different pixels, such as the position of the pixel, the material information of the pixel, etc., can be determined through the feature map. By extracting the feature map, the image block can be obtained through the feature map. In some embodiments, a convolutional neural network can be used to process the initial scene image to obtain a feature map and the probabilities corresponding to different pixels in the feature map.
[0119] The above-mentioned conversion of pixels of the same material between adjacent pixels in the feature map into image blocks to obtain the multiple image blocks can effectively reduce the number of image blocks, so that when the GPU renders the image blocks, it can reduce the occupancy of GPU resources while ensuring the rendering effect, thereby improving the rendering efficiency.
[0120] In an embodiment of the present application, an initial scene image is generated based on a scene model to be rendered; features are extracted from the initial scene image to obtain a feature map and probabilities corresponding to different pixels in the feature map, the feature map including pixels of different materials; pixels of the same material between adjacent pixels in the feature map are converted into image blocks to obtain multiple image blocks; and the weight corresponding to each image block is calculated based on the probabilities corresponding to different pixels in the feature map.
[0121] In some embodiments, feature extraction is performed on the initial scene image to obtain a feature map, and the probabilities corresponding to different pixels in the feature map can be calculated by a normalized exponential function (Softmax function) after the feature map is extracted through a convolutional neural network.
[0122] In some embodiments, in addition to calculating the weight corresponding to each image block based on the probabilities corresponding to different pixels in the feature map, multiple image blocks can also be processed by a normalized exponential function to calculate the probabilities of different image blocks, and the probabilities of the image blocks can be directly set as the weights corresponding to the image blocks.
[0123] In one embodiment, extracting features from the initial scene image to obtain a feature map and probabilities corresponding to different pixels in the feature map include:
[0124] Normalizing the initial scene image to obtain a normalized image matrix;
[0125] Extracting material features from the normalized image matrix to obtain material features;
[0126] Upsampling and downsampling the material features, and fusing the upsampling result and the downsampling result to obtain the feature map;
[0127] The feature map is calculated by a normalized exponential function to obtain the probabilities corresponding to different pixels in the feature map.
[0128] The above-mentioned normalization of the initial scene image to obtain a normalized image matrix can specifically be to scale the pixel value range to between [0, 1] or [-1, 1] to ensure the consistency of the data in the initial scene image and the stability of the model. In some embodiments, linear normalization or zero-mean normalization can be used to normalize the initial scene image to obtain a normalized image matrix. In other embodiments, the initial scene image can also be normalized by a normalization layer in a neural network to obtain a normalized image matrix.
[0129] It should be noted that in the present application, the image block corresponds to a single material to facilitate GPU rendering. In the process of processing the feature map, it is necessary to extract different material features at different positions in the feature map to achieve an image block of a single material.
[0130] In some embodiments, material features can be extracted through a residual network (ResNet). Specifically, the normalized image matrix I is used as the input of ResNet to calculate the feature map distribution F of the scene. res , the process can be expressed as follows:
[0131] Fres=ResNet(I)
[0132] In this way, the material features are extracted through the residual network.
[0133] Furthermore, the extracted material features can be further optimized. In some embodiments, the attention mechanism is used to make the network focus on the important areas in the image, enhance the representation ability of various materials, and obtain the enhanced material features F by calculation. att , the process can be expressed as follows:
[0134] Fatt=Attention(Fres)
[0135] Among them, Attention() is a neural network with attention mechanism.
[0136] The material features are upsampled and downsampled, and the upsampling results and downsampling results are fused to obtain the feature map, wherein the upsampling result is used to enhance image details and material changes, and the downsampling result is used to capture global information and a large range of material structures. The information of the two is fused to obtain a feature map, thereby making full use of the information of the two to improve classification accuracy. The process can be expressed as follows:
[0137] Ffuse=Fup+Fdown
[0138] In the formula, F fuse is the feature map, Fup is the upsampling result, F down is the downsampling result.
[0139] The above-mentioned feature map is calculated by the normalized exponential function to obtain the probabilities corresponding to different pixels in the feature map. The fused feature map can be processed by the Softmax function to output the probability P of each pixel position belonging to various materials. mat The process can be expressed as follows:
[0140] Pmat=Softmax(Wseg·Ffuse+bseg)
[0141] W in the formula seg is the weight of the classification layer, b seg is the bias of the classification layer, and both are calculated through the classification function.
[0142] In an embodiment of the present application, the initial scene image is normalized to obtain a normalized image matrix; material features are extracted from the normalized image matrix to obtain material features; the material features are upsampled and downsampled, and the upsampling results and the downsampling results are fused to obtain the feature map; the feature map is calculated using a normalized exponential function to calculate the probabilities corresponding to different pixels in the feature map.
[0143] In one embodiment, the calculating the weight corresponding to each image block based on the probabilities corresponding to different pixels in the feature map includes:
[0144] Calculating the probability and value of pixels in each image block;
[0145] A weight corresponding to each image block is calculated, where the weight corresponding to each image block is a quotient of the probability sum value and the number of pixels of the image block.
[0146] The above calculation process can be expressed by the following formula:
[0147]
[0148] In the formula, W block (i,j) is the weight corresponding to the image block, (i,j) and (i,j) are the pixel positions, block(i,j) is the image block where the pixel (i,j) is located, block_size is the number of pixels in the image block, P mat (x,y) is the probability of a pixel at position (x,y).
[0149] In some embodiments, in addition to calculating the weight corresponding to each image block based on the probabilities corresponding to different pixels in the feature map, multiple image blocks can also be processed by a normalized exponential function to calculate the probabilities of different image blocks, and the probabilities of the image blocks can be directly set as the weights corresponding to the image blocks.
[0150] For example, multiple image blocks are merged into an intermediate image, and the intermediate image is used as input to a normalized exponential function, and the normalized exponential function outputs the probabilities corresponding to different image blocks.
[0151] In the embodiment of the present application, the probability and value of the pixels in each image block are calculated; the weight corresponding to each image block is calculated, and the weight corresponding to each image block is the quotient of the probability and value and the number of pixels in the image block. In this way, the weight corresponding to the entire pixel block is calculated by the probability of the pixels in each image block.
[0152] This application also provides a multi-GPU parallel rendering flow chart, such as Figure 3 As shown, it includes a deep learning module, a GPU monitoring module and a dynamic merging module, wherein, after the scene model is loaded and the material parameters are set, feature extraction is performed through the deep learning model to obtain image blocks, and the weights corresponding to the image blocks are output; the GPU monitoring module is used to monitor the GPU performance to calculate the first parameters of different GPUs, and the GPUs are dynamically sorted according to the first parameters to facilitate the subsequent allocation of different image blocks based on the first parameters; the dynamic merging module is used to implement GPU parallel rendering to render all image blocks, and then the intermediate rendering data is merged through the first target GPU to obtain a merged result.
[0153] See also Figure 4 , Figure 4 is a structural diagram of an image rendering device provided in an embodiment of the present application, such as Figure 4 As shown, the image rendering device 400 includes:
[0154] An extraction module 401 is used to extract data from the scene model to be rendered, and obtain a plurality of image blocks and a weight corresponding to each image block;
[0155] A calculation module 402 is used to calculate a first parameter corresponding to each GPU in a plurality of graphics processors GPU, wherein the plurality of GPUs are GPUs for performing image rendering, and the first parameter is used to characterize the performance and / or load of the GPU;
[0156] An allocating module 403, configured to allocate the plurality of image blocks to the plurality of GPUs respectively based on the first parameter;
[0157] A rendering module 404, configured to render the image blocks based on the weights corresponding to each image block through the multiple GPUs to obtain multiple intermediate rendering data;
[0158] The merging module 405 is used to merge the multiple intermediate rendering data through the first target GPU to obtain a rendering result corresponding to the scene model.
[0159] In one embodiment, the calculation module 402 includes:
[0160] An acquisition unit, configured to acquire at least one of a performance parameter and a load parameter corresponding to each GPU;
[0161] A first calculation unit, configured to calculate a first parameter corresponding to each GPU based on at least one of a corresponding performance parameter and a load parameter;
[0162] The performance parameter includes at least one of a computing capability parameter, a video memory size, a video memory bandwidth, and a current video memory usage rate;
[0163] The load parameter includes at least one of the GPU task running time, task computing amount and task video memory usage rate.
[0164] In one embodiment, the performance parameter is calculated as follows:
[0165] Based on at least one of the computing capability parameter, the video memory size, the video memory bandwidth, and the current video memory usage rate, calculate and obtain a first intermediate parameter corresponding to each GPU;
[0166] Calculate the performance parameter of each GPU, where the performance parameter of each GPU is a quotient of a difference between the first intermediate parameter of each GPU and a minimum first intermediate parameter and a first target difference, where the first target difference is a difference between a maximum first intermediate parameter and a minimum first intermediate parameter corresponding to the multiple GPUs.
[0167] In one embodiment, the load parameter is calculated as follows:
[0168] Calculating a second intermediate parameter corresponding to each GPU, where the second intermediate parameter is the task running time, the task computing amount, or the task video memory usage rate, or the product of at least two of the task running time, the task computing amount, and the task video memory usage rate;
[0169] Calculate the load parameter of each GPU, where the load parameter of each GPU is the quotient of the difference between the second intermediate parameter of each GPU and the minimum second intermediate parameter and the second target difference, and the second target difference is the difference between the maximum second intermediate parameter and the minimum second intermediate parameter corresponding to the multiple GPUs.
[0170] In one embodiment, the allocation module 403 includes:
[0171] A sorting unit, configured to sort the multiple image blocks based on a weight corresponding to each image block;
[0172] An allocating unit is used to allocate the sorted plurality of image blocks to the plurality of GPUs based on the first parameter.
[0173] In one embodiment, the allocation unit comprises:
[0174] A first allocation subunit is configured to allocate, when allocating a first image block among the plurality of image blocks after sorting, the first image block to a GPU having the highest first parameter among the plurality of GPUs;
[0175] A second allocation subunit is used to allocate a non-first image block among the multiple image blocks after sorting to a second target GPU, where the second target GPU is the GPU with the highest second parameter among the multiple GPUs, and the second parameter is a parameter recalculated by the multiple GPUs after the image blocks have been allocated.
[0176] In one embodiment, the extraction module 401 includes:
[0177] A generating unit, configured to generate an initial scene image based on the scene model to be rendered;
[0178] An extraction unit, configured to perform feature extraction on the initial scene image to obtain a feature map and probabilities corresponding to different pixels in the feature map, wherein the feature map includes pixels of different materials;
[0179] A conversion unit, configured to convert pixels of the same material between adjacent pixels in the feature map into image blocks to obtain the plurality of image blocks;
[0180] The second calculation unit is used to calculate the weight corresponding to each image block based on the probabilities corresponding to different pixels in the feature map.
[0181] In one embodiment, the extraction unit comprises:
[0182] A processing subunit, used for normalizing the initial scene image to obtain a normalized image matrix;
[0183] An extraction subunit, used for extracting material features from the normalized image matrix to obtain material features;
[0184] A sampling subunit, used for upsampling and downsampling the material features, and fusing the upsampling result and the downsampling result to obtain the feature map;
[0185] The first calculation subunit is used to calculate the feature map by using a normalized exponential function to obtain probabilities corresponding to different pixels in the feature map.
[0186] In one embodiment, the second computing unit comprises:
[0187] A second calculation subunit, used to calculate the probability and value of pixels in each image block;
[0188] The third calculation subunit is used to calculate the weight corresponding to each image block, where the weight corresponding to each image block is a quotient of the probability sum value and the number of pixels of the image block.
[0189] The image rendering device provided in the embodiment of the present application is capable of implementing each process of each embodiment of the above-mentioned image rendering method, and the technical features correspond one to one and can achieve the same technical effect. To avoid repetition, they will not be described here.
[0190] It should be noted that the image rendering device in the embodiment of the present application can be a device, or a component, an integrated circuit, or a chip in an electronic device.
[0191] For details, see Figure 5 An embodiment of the present application further provides an electronic device, including a bus 501, a transceiver 502, an antenna 503, a bus interface 504, a processor 505 and a memory 506.
[0192] Wherein, the transceiver 502 is used to obtain eye movement data, and the eye movement data includes eye movement images of multiple consecutive time steps;
[0193] The processor 505 is used to extract data from the scene model to be rendered to obtain multiple image blocks and a weight corresponding to each image block;
[0194] The processor 505 is further used to calculate a first parameter corresponding to each GPU in a plurality of image processors GPU, wherein the plurality of GPUs are GPUs used for image rendering, and the first parameter is used to characterize the performance and / or load of the GPU;
[0195] The processor 505 is further configured to allocate the plurality of image blocks to the plurality of GPUs respectively based on the first parameter;
[0196] The processor 505 is further configured to render the image blocks based on the weights corresponding to each image block through the multiple GPUs to obtain multiple intermediate rendering data;
[0197] The processor 505 is further configured to merge the plurality of intermediate rendering data through the first target GPU to obtain a rendering result corresponding to the scene model.
[0198] In one embodiment, calculating the first parameter corresponding to each GPU in the plurality of graphics processing units GPUs includes:
[0199] Obtain at least one of a performance parameter and a load parameter corresponding to each GPU;
[0200] Calculate a first parameter corresponding to each GPU based on at least one of a corresponding performance parameter and a load parameter;
[0201] The performance parameter includes at least one of a computing capability parameter, a video memory size, a video memory bandwidth, and a current video memory usage rate;
[0202] The load parameter includes at least one of the GPU task running time, task computing amount and task video memory usage rate.
[0203] In one embodiment, the performance parameter is calculated as follows:
[0204] Based on at least one of the computing capability parameter, the video memory size, the video memory bandwidth, and the current video memory usage rate, calculate and obtain a first intermediate parameter corresponding to each GPU;
[0205] Calculate the performance parameter of each GPU, where the performance parameter of each GPU is a quotient of a difference between the first intermediate parameter of each GPU and a minimum first intermediate parameter and a first target difference, where the first target difference is a difference between a maximum first intermediate parameter and a minimum first intermediate parameter corresponding to the multiple GPUs.
[0206] In one embodiment, the load parameter is calculated as follows:
[0207] Calculating a second intermediate parameter corresponding to each GPU, where the second intermediate parameter is the task running time, the task computing amount, or the task video memory usage rate, or the product of at least two of the task running time, the task computing amount, and the task video memory usage rate;
[0208] Calculate the load parameter of each GPU, where the load parameter of each GPU is the quotient of the difference between the second intermediate parameter of each GPU and the minimum second intermediate parameter and the second target difference, and the second target difference is the difference between the maximum second intermediate parameter and the minimum second intermediate parameter corresponding to the multiple GPUs.
[0209] In one embodiment, allocating the plurality of image blocks to the plurality of GPUs respectively based on the first parameter includes:
[0210] sorting the multiple image blocks based on the weight corresponding to each image block;
[0211] The sorted plurality of image blocks are allocated to the plurality of GPUs based on the first parameter.
[0212] In one embodiment, allocating the sorted plurality of image blocks to the plurality of GPUs comprises:
[0213] In the case of allocating a first image block among the plurality of image blocks after sorting, allocating the first image block to a GPU having the highest first parameter among the plurality of GPUs;
[0214] In the case of allocating a non-first image block among the sorted multiple image blocks, the non-first image block is allocated to a second target GPU, where the second target GPU is the GPU with the highest second parameter among the multiple GPUs, and the second parameter is a parameter recalculated by the multiple GPUs after the image blocks have been allocated.
[0215] In one embodiment, the data extraction of the scene model to be rendered to obtain a plurality of image blocks and a weight corresponding to each image block includes:
[0216] Generating an initial scene image based on the scene model to be rendered;
[0217] Extracting features from the initial scene image to obtain a feature map and probabilities corresponding to different pixels in the feature map, wherein the feature map includes pixels of different materials;
[0218] Convert pixels of the same material between adjacent pixels in the feature map into image blocks to obtain the multiple image blocks;
[0219] The weight corresponding to each image block is calculated based on the probabilities corresponding to different pixels in the feature map.
[0220] In one embodiment, extracting features from the initial scene image to obtain a feature map and probabilities corresponding to different pixels in the feature map include:
[0221] Normalizing the initial scene image to obtain a normalized image matrix;
[0222] Extracting material features from the normalized image matrix to obtain material features;
[0223] Upsampling and downsampling the material features, and fusing the upsampling result and the downsampling result to obtain the feature map;
[0224] The feature map is calculated by a normalized exponential function to obtain the probabilities corresponding to different pixels in the feature map.
[0225] In one embodiment, the calculating the weight corresponding to each image block based on the probabilities corresponding to different pixels in the feature map includes:
[0226] Calculating the probability and value of pixels in each image block;
[0227] A weight corresponding to each image block is calculated, where the weight corresponding to each image block is a quotient of the probability sum value and the number of pixels of the image block.
[0228] exist Figure 5 In the embodiment, the bus architecture (represented by bus 501) is shown, and bus 501 may include any number of interconnected buses and bridges, and bus 501 links various circuits including one or more processors represented by processor 505 and memory represented by memory 506. Bus 501 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 504 provides an interface between bus 501 and transceiver 502. Transceiver 502 may be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by processor 505 is transmitted on a wireless medium via antenna 503, and further, antenna 503 also receives data and transmits the data to processor 505.
[0229] The processor 505 is responsible for managing the bus 501 and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory 506 can be used to store data used by the processor 505 when performing operations.
[0230] Optionally, the processor 505 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD).
[0231] An embodiment of the present application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned image rendering method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0232] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the embodiment of the above-mentioned image rendering method is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0233] The embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the embodiment of the above-mentioned image rendering method are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.
[0234] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0235] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.
[0236] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. An image rendering method, characterized in that: include: Extract data from the scene model to be rendered to obtain multiple image blocks and the weight corresponding to each image block; Calculating a first parameter corresponding to each GPU in a plurality of image processors GPU, wherein the plurality of GPUs are GPUs for performing image rendering, and the first parameter is used to characterize the performance and / or load of the GPU; Allocating the plurality of image blocks to the plurality of GPUs respectively based on the first parameter; Rendering the image blocks based on the weights corresponding to each image block by the multiple GPUs to obtain multiple intermediate rendering data; The plurality of intermediate rendering data are merged through the first target GPU to obtain a rendering result corresponding to the scene model.
2. The method according to claim 1, characterized in that The calculating of the first parameter corresponding to each GPU in the plurality of graphics processing units GPUs comprises: Obtain at least one of a performance parameter and a load parameter corresponding to each GPU; Calculate a first parameter corresponding to each GPU based on at least one of a corresponding performance parameter and a load parameter; The performance parameter includes at least one of a computing capability parameter, a video memory size, a video memory bandwidth, and a current video memory usage rate; The load parameter includes at least one of the GPU task running time, task computing amount and task video memory usage rate.
3. The method according to claim 2, characterized in that The performance parameters are calculated as follows: Based on at least one of the computing capability parameter, the video memory size, the video memory bandwidth, and the current video memory usage rate, calculate and obtain a first intermediate parameter corresponding to each GPU; Calculate the performance parameter of each GPU, where the performance parameter of each GPU is a quotient of a difference between the first intermediate parameter of each GPU and a minimum first intermediate parameter and a first target difference, where the first target difference is a difference between a maximum first intermediate parameter and a minimum first intermediate parameter corresponding to the multiple GPUs.
4. The method according to claim 2, characterized in that The load parameters are calculated as follows: Calculating a second intermediate parameter corresponding to each GPU, where the second intermediate parameter is the task running time, the task computing amount, or the task video memory usage rate, or the product of at least two of the task running time, the task computing amount, and the task video memory usage rate; Calculate the load parameter of each GPU, where the load parameter of each GPU is the quotient of the difference between the second intermediate parameter of each GPU and the minimum second intermediate parameter and the second target difference, and the second target difference is the difference between the maximum second intermediate parameter and the minimum second intermediate parameter corresponding to the multiple GPUs.
5. The method according to claim 1, characterized in that The allocating the plurality of image blocks to the plurality of GPUs respectively based on the first parameter comprises: sorting the multiple image blocks based on the weight corresponding to each image block; The sorted plurality of image blocks are allocated to the plurality of GPUs based on the first parameter.
6. The method according to claim 5, characterized in that The step of allocating the sorted plurality of image blocks to the plurality of GPUs comprises: In the case of allocating a first image block among the plurality of image blocks after sorting, allocating the first image block to a GPU having the highest first parameter among the plurality of GPUs; In the case of allocating a non-first image block among the sorted multiple image blocks, the non-first image block is allocated to a second target GPU, where the second target GPU is the GPU with the highest second parameter among the multiple GPUs, and the second parameter is a parameter recalculated by the multiple GPUs after the image blocks have been allocated.
7. The method according to claim 1, characterized in that The step of extracting data from the scene model to be rendered to obtain a plurality of image blocks and a weight corresponding to each image block includes: Generating an initial scene image based on the scene model to be rendered; Extracting features from the initial scene image to obtain a feature map and probabilities corresponding to different pixels in the feature map, wherein the feature map includes pixels of different materials; Convert pixels of the same material between adjacent pixels in the feature map into image blocks to obtain the multiple image blocks; The weight corresponding to each image block is calculated based on the probabilities corresponding to different pixels in the feature map.
8. The method according to claim 7, characterized in that The extracting features of the initial scene image to obtain a feature map and probabilities corresponding to different pixels in the feature map includes: Normalizing the initial scene image to obtain a normalized image matrix; Extracting material features from the normalized image matrix to obtain material features; Upsampling and downsampling the material features, and fusing the upsampling result and the downsampling result to obtain the feature map; The feature map is calculated by a normalized exponential function to obtain the probabilities corresponding to different pixels in the feature map.
9. The method according to claim 7, characterized in that The calculating the weight corresponding to each image block based on the probabilities corresponding to different pixels in the feature map includes: Calculating the probability and value of pixels in each image block; The weight corresponding to each image block is calculated according to the probability and value of the pixels in each image block, and the weight corresponding to each image block is the quotient of the probability and value and the number of pixels in the image block.
10. An image rendering device, characterized in that: include: An extraction module is used to extract data from the scene model to be rendered, and obtain multiple image blocks and a weight corresponding to each image block; A calculation module, used to calculate a first parameter corresponding to each GPU in a plurality of image processors GPU, wherein the plurality of GPUs are GPUs used for image rendering, and the first parameter is used to characterize the performance and / or load of the GPU; an allocating module, configured to allocate the plurality of image blocks to the plurality of GPUs respectively based on the first parameter; A rendering module, configured to render the image blocks based on the weights corresponding to each image block through the multiple GPUs to obtain multiple intermediate rendering data; The merging module is used to merge the multiple intermediate rendering data through the first target GPU to obtain a rendering result corresponding to the scene model.
11. An electronic device, characterized in that: Including transceiver and processor, The transceiver is used to extract data from the scene model to be rendered to obtain multiple image blocks and a weight corresponding to each image block; The processor is used to calculate a first parameter corresponding to each GPU in a plurality of image processors GPU, wherein the plurality of GPUs are GPUs used for image rendering, and the first parameter is used to characterize the performance and / or load of the GPU; The processor is further configured to allocate the plurality of image blocks to the plurality of GPUs respectively based on the first parameter; The processor is further configured to render the image blocks based on the weights corresponding to each image block through the multiple GPUs to obtain multiple intermediate rendering data; The processor is further configured to merge the plurality of intermediate rendering data through the first target GPU to obtain a rendering result corresponding to the scene model.
12. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps in the image rendering method according to any one of claims 1 to 9 when executed by the processor.
13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image rendering method according to any one of claims 1 to 9 are implemented.
14. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the image rendering method according to any one of claims 1 to 9.