A composite distributed rendering method based on depth map algorithm and block-based algorithm

Abstract

This application discloses a composite distributed rendering method based on depth map algorithms and block-based algorithms, relating to the field of image processing technology. The method includes: dividing a scene image into several image regions; dividing the image regions into blocks based on a block-based algorithm to obtain multiple first image blocks; classifying each first image block into a first type of image block and a second type of image block based on the information complexity value corresponding to each first image block; obtaining the rendering priority corresponding to each second type of image block based on a depth map algorithm; rendering each second type of image block individually based on the rendering priority; and performing block-based rendering on each first type of image block based on a block-based rendering algorithm to finally obtain the rendered image. This application classifies the image blocks of the rendered image according to information complexity values ​​and uses different rendering methods for different image blocks to fully utilize rendering resources and improve rendering quality and efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically a composite distributed rendering method based on depth map algorithm and block algorithm. Background Technology

[0002] In computer graphics and computer vision, rendering technology is the crucial process for converting 3D models into 2D color images. With the continuous improvement of computer hardware and graphics processing capabilities, especially in applications such as Virtual Reality (VR), Augmented Reality (AR), game development, and film special effects production, the demand for high-quality rendering is increasing. In these scenarios, accurate and efficient rendering methods are particularly important. Since standalone depth map algorithms or tile-based algorithms have certain inefficiencies in rendering, combining depth map algorithms with tile-based algorithms is an effective way to achieve composite distributed rendering.

[0003] When computer hardware is limited, rendering the entire scene image results in excessive computation and low rendering efficiency. Therefore, it's necessary to render the scene image in regions. A composite distributed rendering method based on depth map algorithms and block-based algorithms generally involves steps such as scene block division, depth map generation, distributed computation, and result synthesis. Although this composite distributed rendering method offers advantages in improving rendering efficiency and image quality, it still has some shortcomings. Summary of the Invention

[0004] The purpose of this application is to provide a composite distributed rendering method based on depth map algorithm and block algorithm to solve the technical problem that existing composite distributed rendering methods are prone to causing boundary distortion of the rendered image.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] A composite distributed rendering method based on depth map algorithm and block-based algorithm, comprising:

[0007] The scene image is divided into several image regions, and the image regions are further divided into blocks to obtain multiple first image blocks;

[0008] Based on the edge detection algorithm, the information complexity value corresponding to each first image block is obtained; the information complexity value is used to characterize at least the amount of information complexity in the corresponding first image block;

[0009] Based on the information complexity value, each first image block is classified into a first type of image block and a second type of image block; the information complexity value of the first type of image block is higher than that of the second type of image block.

[0010] Based on the depth map algorithm, the rendering priority corresponding to each first image patch is obtained; the rendering priority is used to characterize the difficulty of rendering the corresponding second type of image patch;

[0011] Based on the rendering priority, each second-type image block is rendered individually, and based on the block rendering algorithm, each first-type image block is rendered in blocks.

[0012] The rendered image is obtained based on the rendering results of each first-class image block and each second-class image block within all image regions.

[0013] As a specific solution in this application, the step of obtaining the information complexity value corresponding to each first image block based on the edge detection algorithm includes:

[0014] Based on an edge detection algorithm, the texture complexity value of the first image block is obtained; the texture complexity value is used to characterize at least the magnitude of the texture complexity in the first image block.

[0015] Obtain the depth complexity value of the first image block; the depth complexity value is used to characterize at least the magnitude of the difference in depth values ​​among the pixels in the first image block;

[0016] Based on the texture complexity value and the depth complexity value, the information complexity value of the first image block is obtained.

[0017] As a specific solution in this application, the step of obtaining the texture complexity value of the first image patch based on the edge detection algorithm includes:

[0018] An edge detection image is obtained from the first image block based on an edge detection algorithm;

[0019] Based on the edge detection image, multiple variance values ​​are obtained; the variance value is the variance of the slope between each adjacent pixel in any edge line in the edge detection image;

[0020] Based on the edge detection image, information entropy is obtained; the information entropy is the information entropy of the average gray value of all edge lines in the edge detection image.

[0021] Based on the information entropy and various variance values, the texture complexity value of the first image patch is obtained.

[0022] As a specific solution in this application, the formula for calculating the texture complexity value of the first image patch based on the information entropy and various variance values ​​is as follows:

[0023]

[0024] Where f is the texture complexity value corresponding to the first image patch; n represents the number of edge lines in the edge detection image corresponding to the first image patch; H represents the variance of the slope between adjacent pixels of the i-th edge line in the edge detection image; H represents the information entropy of the average gray value of all edge lines in the edge detection image.

[0025] As a specific solution in this application, obtaining the depth complexity value of the first image patch includes:

[0026] From the edge detection image, obtain the depth value corresponding to the pixel point in each edge line;

[0027] Based on each depth value, the depth complexity value of the first image block is obtained; the depth complexity value is the average of the standard deviations of the depth values ​​corresponding to each pixel in each edge line.

[0028] As a specific solution in this application, the step of obtaining the rendering priority corresponding to each second-type image patch based on the depth map algorithm includes:

[0029] Based on the depth map algorithm, the depth map of each second-class image patch is obtained;

[0030] Based on the depth map, key edge contours are obtained from the edge detection image; the key edge contours are edge contours with complex depth variations in the second type of image patch.

[0031] Based on the key edge contours, obtain the rendering level values ​​corresponding to each key edge contour;

[0032] Based on the information complexity value and the rendering level value, obtain the level importance value of the rendering layer corresponding to each key edge contour;

[0033] The rendering priority of the second type of image patch is obtained based on the importance values ​​of each level.

[0034] As a specific solution in this application, the step of obtaining key edge contours from the edge detection image based on the depth map includes:

[0035] Based on the depth map, the depth values ​​corresponding to the pixels in each edge line are obtained from the edge detection image;

[0036] Based on each depth value, obtain the standard deviation of the pixels in each edge line;

[0037] Arrange the standard deviations in ascending order to obtain the first sequence;

[0038] Divide the first sequence to obtain the second sequence;

[0039] The contour formed by the edge lines corresponding to the standard deviation in the second sequence is taken as the key edge contour.

[0040] As a specific solution in this application, the step of dividing the first sequence to obtain the second sequence includes:

[0041] Based on the first sequence, obtain the difference sequence;

[0042] Based on the difference sequence, obtain the maximum value;

[0043] Based on the maximum value, a first standard deviation and a second standard deviation are obtained; the first standard deviation and the second standard deviation are the standard deviations of the maximum value obtained in the first sequence, and the second standard deviation is greater than the first standard deviation;

[0044] The second standard deviation and the standard deviations in the first sequence that follow the second standard deviation are combined to form the second sequence.

[0045] As a specific solution in this application, the step of obtaining the rendering priority of the second type of image patch based on the importance values ​​of each level includes:

[0046] The visual cone spatial boundary is obtained from the edge detection image;

[0047] Based on the visual cone space boundary, the probability that the key edge contour is located in the visual cone space is obtained;

[0048] Based on the importance values ​​of each level and the probability, the rendering priority of the second type of image patch is obtained.

[0049] As a specific solution in this application, obtaining the probability that the key edge contour lies in the visual cone space based on the visual cone space boundary includes:

[0050] Based on the key edge contour, a first distance and a first area are obtained; the first distance is the average depth value of each pixel corresponding to the key edge contour; the first area is the area of ​​the key edge contour.

[0051] Based on the visual cone spatial boundary, a second distance and a second area are obtained; the second distance is the depth value of the far clipping plane in the visual cone spatial boundary; the second area is the area of ​​the near clipping plane in the visual cone spatial boundary.

[0052] Based on the first distance, the second distance, the first area, and the second area, the probability that the key edge contour is located in the visual cone space is obtained.

[0053] Compared with the prior art, the beneficial effects of this application are:

[0054] This application first divides the scene image into regions using existing methods and performs preliminary block segmentation of the image regions. Then, it uses information such as contour shape, color, and depth in the image blocks to obtain the information complexity of each image block. Next, based on the information complexity, each first image block is binary classified to obtain a first-class image block with higher complexity and a second-class image block with lower complexity. For the first-class image blocks with higher complexity, a block rendering algorithm is used to perform block rendering to improve rendering efficiency. For the second-class image blocks with lower complexity, a depth map algorithm is used to obtain the corresponding rendering priority, and rendering is performed separately based on the rendering priority. This aims to make full use of rendering resources and improve rendering quality. By merging the multiple rendered image regions, the complete rendering of the scene image is performed by region even when computer hardware conditions are insufficient. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating a composite distributed rendering method based on depth map algorithm and block algorithm proposed in an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] The terms "first," "second," etc., in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. For example, the first sequence and the second sequence mentioned below belong to different sequences. It should be understood that such names can be used interchangeably where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules in the embodiments of this application is merely a logical division. In actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be ignored or not performed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, and the indirect coupling or communication connection between modules may be electrical or other similar forms. None of these are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0058] Before understanding the composite distributed rendering method based on depth map algorithm and block algorithm proposed in the embodiments of this application, the steps of the existing composite distributed rendering method based on depth map algorithm and block algorithm are as follows:

[0059] Scene segmentation: First, the scene image is divided into multiple small blocks based on a segmentation algorithm. Each small block contains a portion of the objects and geometric information in the scene. This segmentation can be achieved by dividing the scene's view window into multiple blocks, with each block handled by an independent rendering node.

[0060] Depth Map Generation: For each block, a corresponding depth map is generated based on a depth map algorithm. The depth map records the depth information of each pixel in the scene image. By calculating the depth information of all objects in each block, the depth map for that block can be generated. This step is crucial for ensuring consistency when multiple rendering nodes render the same scene.

[0061] Task allocation: The rendering task for each block is assigned to a different rendering node. Each rendering node is responsible for rendering one or more blocks of the image.

[0062] Rendering execution: Each rendering node performs rendering according to its assigned task. Each rendering node only processes the block it is responsible for, generating the image data for that block.

[0063] Result merging: Finally, the block image data rendered by each rendering node are merged to form a complete scene image.

[0064] It should be noted that the above steps are all mature existing technologies, so they will not be elaborated on here.

[0065] To address the technical problem of boundary distortion in rendered images caused by existing composite distributed rendering methods, this application proposes an embodiment of a composite distributed rendering method based on depth map algorithms and block-based algorithms. The overall technical concept of this method is as follows: based on node characteristics, depth map algorithms are used to render low-complexity nodes, while block-based rendering algorithms are used to render high-complexity nodes in blocks. This improves the overall rendering speed and efficiency while effectively solving the problem of insufficient computational power. Here, a node can represent an object or a local region in the image. For example, consider a 3D rendering scene with a map and N airplanes. During flight, due to the high speed of the airplanes, different scenes from the map, such as numerous buildings, mountains, and trees, need to be continuously loaded. This generates a large amount of computation. Therefore, the corresponding nodes of the map need to use a chunking algorithm to improve the rendering efficiency of those nodes. Since the scene of a single airplane is relatively small and the amount of data is also relatively small, the airplane can be rendered separately. Similarly, multiple airplanes can be rendered separately because the computational load of a single model is not very large. In this case, a depth map rendering method is used to merge the N airplanes from multiple servers into a complete image using depth maps.

[0066] Based on the above overall technical concept, the implementation process of the composite distributed rendering method based on depth map algorithm and block algorithm proposed in this application includes:

[0067] (1) Rendering task allocation on the management end

[0068] Scene file loading: Based on the input scene file, the system loads different scene files through different modules, converting the content of the local file into content that the program can recognize.

[0069] Scene file analysis: The system has the capability to perform in-depth analysis of various scene files. By comprehensively analyzing the structure, content, and parameters of scene files, the system can quickly identify key nodes, data flows, and interaction relationships within the scene, providing an accurate data foundation for subsequent task allocation and load balancing.

[0070] Node type analysis: The system classifies the nodes in the scenario by type and provides a data foundation for the load balancing algorithm by comprehensively evaluating different types of nodes, ensuring that the algorithm can accurately allocate nodes to different servers.

[0071] Load balancing algorithm allocation: The system employs an advanced load balancing algorithm that dynamically adjusts task allocation strategies based on scene file analysis and node type analysis results. By rationally allocating tasks and distributing the rendering methods and tasks across servers, the system achieves load balancing, preventing overload of individual servers and improving the overall system's processing capacity and response speed.

[0072] Task allocation: Based on the results of scenario file analysis, node type analysis, and load balancing algorithm allocation, the system reasonably allocates tasks to each server node.

[0073] (2) Rendering task allocation on the management end

[0074] Scene file loading: Based on the input scene file, the system loads different scene files through different modules, converting the content of the local file into content that the program can recognize.

[0075] Node type analysis: The system classifies the nodes in the scenario into different types. By comprehensively evaluating different types of nodes, it provides a data foundation for the load balancing algorithm, ensuring that the algorithm can accurately allocate nodes to different servers.

[0076] Waiting for tasks: After the server starts up and performs the necessary initialization, it enters a waiting state and waits for the management to assign tasks.

[0077] Execution of tasks: After the server receives the task from the management end, it will parse the received information, determine which nodes to render using the depth map algorithm, which scenes to render in blocks, and specifically which block(s) to render and calculate.

[0078] (3) Rendering data acquisition

[0079] After the server rendering task is completed, the depth map data of the corresponding task is obtained, the data is compressed and serialized, and then transmitted over the network to the designated merging service for merging.

[0080] (4) Rendering data merging

[0081] Rendering data merging is one of the key steps in a distributed rendering system. It is responsible for receiving rendering data from multiple rendering servers, decompressing and deserializing this data, and finally merging this data into the final rendering result based on depth and color information.

[0082] (5) Merged data shows

[0083] The merged data is then displayed by the display processing module, thus completing the rendering and display process for one frame of data.

[0084] The following section, with reference to the accompanying drawings, details the implementation process of a composite distributed rendering method based on depth map algorithm and block-based algorithm proposed in this application. Figure 1 As shown, the composite distributed rendering method based on depth map algorithm and block algorithm includes steps S100 to S500.

[0085] Step S100: Divide the scene image into several image regions, segment the image regions into blocks, and obtain multiple first image blocks.

[0086] In the embodiments of this application, the scene image is acquired in advance. First, due to the performance limitations of the graphics processing unit, rendering the entire image is somewhat difficult. Therefore, the scene image is divided into several image regions. For example, the scene image is evenly divided into four image regions, each of which is a rectangular region. Subsequently, each image region can be rendered separately by four graphics processing units, and the rendering results of the four image regions are synthesized by the central graphics processing unit to finally complete the rendering of the entire image. Then, each image region is further divided into blocks based on a block-based algorithm (that is, each image region of the scene image is divided into multiple first image blocks). As mentioned above, dividing each image region into blocks is a mature technology, which will not be elaborated here.

[0087] Step S200: Based on the edge detection algorithm, obtain the information complexity value corresponding to each first image block.

[0088] In this embodiment, the information complexity value is used at least to characterize the level of information complexity in the corresponding first image block. It should be noted that the more complex the information contained in a first image block, the more likely it is that key boundary information within that first image block will be lost during subsequent rendering, thus easily causing boundary distortion in the rendered image. In other words, in this embodiment, the more complex the information contained in a first image block, the more attention should be paid to its boundary information during subsequent rendering to avoid boundary distortion after rendering.

[0089] It should be clear that, in the embodiments of this application, any reasonable method can be used to obtain the information complexity value corresponding to each first image block based on the edge detection algorithm. For example, in one embodiment of this application, step S200, obtaining the information complexity value corresponding to each first image block based on the edge detection algorithm, includes steps S210 to S230.

[0090] Step S210: Based on the edge detection algorithm, obtain the texture complexity value of the first image block.

[0091] In embodiments of this application, the texture complexity value is used at least to characterize the magnitude of texture complexity in the first image patch.

[0092] It should be clear that, in the embodiments of this application, any reasonable method can be used to obtain the texture complexity value of the first image patch based on the edge detection algorithm. For example, the number of edge lines in the edge detection image corresponding to the first image patch can be selected as the texture complexity value of the first image patch. That is, the more edge lines in the edge detection image corresponding to the first image patch, the greater the texture complexity of the first image patch; the fewer edge lines in the edge detection image corresponding to the first image patch, the smaller the texture complexity of the first image patch. In order to accurately characterize the texture complexity in the first image patch, in one embodiment of this application, step S210, the step of obtaining the texture complexity value of the first image patch based on the edge detection algorithm, includes steps S211 to S214.

[0093] Step S211: Obtain an edge detection image from the first image block based on the edge detection algorithm.

[0094] It is important to understand that, in the embodiments of this application, the edge detection algorithm can be any algorithm capable of obtaining an edge detection image from the first image patch. For example, the edge detection algorithm can be the Prewitt algorithm, the Sobel algorithm, the Laplacian algorithm, and the Canny algorithm, etc. Obtaining the corresponding edge detection image from a certain image (i.e., the first image patch) based on an edge detection algorithm is a mature technology and will not be elaborated here.

[0095] Step S212: Based on the edge detection image, obtain multiple variance values.

[0096] In this embodiment, the variance value is the variance of the slope between adjacent pixels of any edge line in the edge detection image. It is easy to understand that the smaller the variance of the slope between adjacent pixels of an edge line, the closer the edge line is to a straight line; conversely, the larger the variance of the slope between adjacent pixels of an edge line, the more tortuous the edge line is. Compared to straight lines, tortuous lines correspond to more complex object contours, that is, greater texture complexity.

[0097] Step S213: Obtain information entropy based on the edge detection image.

[0098] In this embodiment, the information entropy is the information entropy of the average grayscale value of all edge lines in the edge detection image. It is easy to understand that if the difference in the average grayscale value of each edge line in the edge detection image is greater, it indicates a greater color difference in the corresponding first image block, that is, a greater texture complexity in the corresponding first image block; conversely, if the difference in the average grayscale value of each edge line in the edge detection image is smaller, it indicates a smaller color difference in the corresponding first image block, that is, a smaller texture complexity in the corresponding first image block.

[0099] Step S214: Based on the information entropy and each variance value, obtain the texture complexity value of the first image block.

[0100] In embodiments of this application, the texture complexity value of the first image patch can be obtained based on the information entropy and each variance value in any reasonable manner. For example, the texture complexity value of the first image patch can be the sum of the information entropy and each variance value, or the sum of the average of each variance value and the information entropy. In a specific embodiment of this application, step S214, the calculation formula for obtaining the texture complexity value of the first image patch based on the information entropy and each variance value, is as follows:

[0101]

[0102] Where f is the texture complexity value corresponding to the first image patch; n represents the number of edge lines in the edge detection image corresponding to the first image patch; H represents the variance of the slope between adjacent pixels of the i-th edge line in the edge detection image; H represents the information entropy of the average gray value of all edge lines in the edge detection image.

[0103] In the embodiments of this application, the texture complexity value of the first image block can be directly used as the information complexity value corresponding to the first image block.

[0104] It is important to understand that the complexity of information within each pre-divided first image block is not only reflected in the complexity of the image texture, but also in the hierarchical information of the target objects in the rendered scene within the actual augmented reality technology. Both together constitute the overall information complexity of the image block. In other words, the depth and hierarchy of the image block also have a significant impact on rendering quality and efficiency. Based on this, step S220 can also be executed in the embodiments of this application.

[0105] Step S220: Obtain the depth complexity value of the first image patch.

[0106] In embodiments of this application, the depth complexity value is used at least to characterize the magnitude of the difference in depth values ​​among individual pixels in the first image block.

[0107] In the embodiments of this application, the depth complexity value of the first image block can be obtained in any reasonable manner. For example, step S220, which involves obtaining the depth complexity value of the first image block, includes steps S221 and S222.

[0108] Step S221: Obtain the depth value corresponding to the pixel point in each edge line from the edge detection image.

[0109] It should be noted that obtaining the depth value corresponding to each pixel in the edge detection image is a mature technology, which will not be elaborated here.

[0110] Step S222: Based on each depth value, obtain the depth complexity value of the first image block.

[0111] In the embodiments of this application, the depth complexity value of the first image patch can be obtained based on each depth value using any reasonable method. For example, the depth complexity value is the average of the standard deviations of the depth values ​​corresponding to each pixel in each edge line. That is, in this embodiment, step S222, the calculation formula for obtaining the depth complexity value of the first image patch based on each depth value is as follows:

[0112]

[0113] in, The depth complexity value represents the depth complexity of the first image patch; n represents the number of edge lines in the edge detection image corresponding to the first image patch. It represents the standard deviation of the depth values ​​corresponding to each pixel point in the i-th edge line of the edge detection image.

[0114] Step S230: Based on the texture complexity value and the depth complexity value, obtain the information complexity value of the first image block.

[0115] In embodiments of this application, the information complexity value of the first image block can be obtained based on the texture complexity value and the depth complexity value in any reasonable manner. For example, the information complexity value of the first image block can be the product or sum of the texture complexity value and the depth complexity value.

[0116] Step S300: Based on the information complexity value, classify each first image block into a first type of image block and a second type of image block.

[0117] By obtaining the complexity values ​​of each image patch as described above, we can further analyze the pixel information within the image patch, evaluate the detailed information within the patch, and improve the overall rendering efficiency of the foreground and background. For example, for complex and constantly changing background information, we need to perform patch rendering based on a segmentation algorithm. For simpler or repetitive foreground information, we can combine a depth map algorithm to obtain different depth level data, perform distributed rendering of repetitive and simple foreground information, and finally obtain a complete foreground image through fusion. This ensures that the scene information in the image patch is well preserved during the rendering process, guaranteeing rendering efficiency and reducing unnecessary waste of rendering resources.

[0118] In this embodiment, each first image block is classified into two categories based on its information complexity value. First image blocks with higher information complexity values ​​are assigned to the first category, while those with lower information complexity values ​​are assigned to the second category. At this point, the information complexity value of the first category of image blocks is higher than that of the second category.

[0119] It should be clear that, in the embodiments of this application, any reasonable method can be used to classify each first image block into a first type of image block and a second type of image block. For example, a binary classification algorithm can be used to perform binary classification on the information complexity value of each first image block, and the first image block can be classified according to the classification result to obtain the first type of image block and the second type of image block.

[0120] Step S400: Based on the depth map algorithm, obtain the rendering priority corresponding to each second type of image patch.

[0121] In embodiments of this application, the rendering priority is used at least to characterize the difficulty of rendering the corresponding second type of image patch. It is readily understood that in subsequent rendering processes, resources can be configured for rendering the second type of image patch based on its rendering difficulty, thereby avoiding the loss of detailed information while improving rendering efficiency.

[0122] It is important to understand that, in the embodiments of this application, the rendering priority corresponding to the second type of image patch can be determined in any way based on the depth map algorithm. For example, the information complexity value of the second type of image patch can be directly used as the rendering priority corresponding to the second type of image patch. That is, if the information complexity value of the second type of image patch is higher (i.e., the higher the complexity), the higher the rendering priority corresponding to the second type of image patch; if the information complexity value of the second type of image patch is lower (i.e., the lower the complexity), the lower the rendering priority corresponding to the second type of image patch.

[0123] After obtaining the information complexity value of each second-type image block, simply using the information complexity value as data support for adjusting rendering resources may not necessarily improve rendering efficiency. Since the segmentation in step S100 is uniform, although the size of each second-type image block is the same, the complexity and hierarchical information of its internal information are different. To further analyze the pixel information in the second-type image blocks, reduce the possibility of losing boundary details during subsequent scene rendering, and ensure that scene information in the image blocks is well preserved during rendering, thereby guaranteeing rendering efficiency and reducing unnecessary waste of rendering resources, in the embodiments of this application, step S400, based on a depth map algorithm, obtains the rendering priority corresponding to each second-type image block, which may include steps S410 to S450.

[0124] Step 410: Based on the depth map algorithm, obtain the depth map of each second type of image patch.

[0125] In the embodiments of this application, depth maps of each second-class image patch can be obtained based on a depth map algorithm.

[0126] Step S420: Based on the depth map, obtain the key edge contours from the edge detection image.

[0127] In the embodiments of this application, the key edge contour is the edge contour with complex depth variations in the second type of image block.

[0128] It should be clear that, in the embodiments of this application, any reasonable method can be used to obtain the key edge contour from the edge detection image. For example, the contour formed by edge lines with large differences in corresponding depth values ​​among pixels can be selected as the key edge contour. In order to accurately obtain the key edge contour, in one embodiment of this application, step S420, based on the depth map, obtains the key edge contour from the edge detection image, including steps S421 to S425.

[0129] Step S421: Based on the depth map, obtain the depth value corresponding to the pixel point in each edge line from the edge detection image.

[0130] It should be clear that obtaining the depth value corresponding to the pixel point in each edge line from the edge detection image based on the depth map is a mature technology, which will not be elaborated here.

[0131] Step S422: Based on each depth value, obtain the standard deviation of the pixels in each edge line.

[0132] It is important to understand that obtaining the standard deviation based on multiple values ​​(i.e., various depth values) is also a mature technique, which will not be elaborated on here.

[0133] Step S423: Arrange the standard deviations in ascending order to obtain the first sequence.

[0134] It is important to understand that sorting multiple values ​​(i.e., various standard deviations) to obtain the corresponding sequence (i.e., the first sequence) is a mature technique, which will not be elaborated here.

[0135] Step S424: Divide the first sequence to obtain the second sequence.

[0136] In embodiments of this application, the second sequence is the sequence formed by the larger standard deviations of the values ​​in the first sequence. In embodiments of this application, the first sequence can be divided in any reasonable way to obtain the second sequence. For example, in one embodiment of this application, the last 50% of the first sequence can be divided into the second sequence. Alternatively, step S424, dividing the first sequence to obtain the second sequence, includes steps S426 to S429.

[0137] Step S426: Based on the first sequence, obtain the difference sequence.

[0138] It is important to understand that obtaining the corresponding difference sequence based on a certain sequence (i.e., the first sequence) is a mature technique, which will not be elaborated here.

[0139] Step S427: Obtain the maximum value based on the difference sequence.

[0140] It should be noted that obtaining the maximum value from a certain sequence (i.e., the difference sequence) is also a mature technique, which will not be elaborated here.

[0141] Step S428: Based on the maximum value, obtain the first standard deviation and the second standard deviation.

[0142] In this embodiment, the first standard deviation and the second standard deviation are the standard deviations of the maximum value obtained in the first sequence, and the second standard deviation is greater than the first standard deviation.

[0143] Step S429: Combine the second standard deviation and the standard deviations in the first sequence that are located after the second standard deviation to form the second sequence.

[0144] Step S425: The contour formed by the edge lines corresponding to the standard deviation in the second sequence is taken as the key edge contour.

[0145] It should be clear that obtaining the contour formed by these edge lines based on each edge line is a mature technology, which will not be elaborated here.

[0146] Step S430: Based on the key edge contours, obtain the rendering level values ​​corresponding to each key edge contour.

[0147] In the embodiments of this application, the rendering level value corresponding to the key edge contour refers to the rendering level corresponding to the key edge contour, which can generally be obtained before rendering. Specifically, step S430, based on the key edge contour, obtains the rendering level value corresponding to each key edge contour, which may include steps S431 to S433.

[0148] Step S431: Based on the second type of image block, obtain the target object and the rendering layer corresponding to the target object.

[0149] In the embodiments of this application, the target object refers to the object that needs to be rendered in the second type of image block. Before rendering, obtaining the target object and rendering layer corresponding to a certain image (i.e., the second type of image block) is a mature technology, which will not be elaborated here.

[0150] Step S432: Based on the contour matching algorithm, obtain the similarity between each target object and each key edge contour.

[0151] It is important to understand that obtaining the similarity between two contours based on contour matching algorithms is a mature technology, which will not be elaborated on here.

[0152] S433: Based on the similarity, obtain the rendering level value corresponding to the key edge contour.

[0153] In this embodiment, the rendering layer value corresponding to the rendering layer of the target object with the highest similarity to the key edge contour can be used as the rendering layer value corresponding to the key edge contour. It is easy to understand that if the rendering layer value of a key edge contour is larger, the rendering of the target object corresponding to the key edge contour is more complex, that is, the rendering difficulty is greater.

[0154] Step S440: Based on the information complexity value and the rendering level value, obtain the level importance value of the rendering layer corresponding to each key edge contour.

[0155] In the embodiments of this application, the hierarchy importance value of the rendering layer corresponding to each key edge contour can be obtained in any reasonable manner based on the information complexity value and the rendering hierarchy value. For example, the hierarchy importance value of the rendering layer corresponding to the key edge contour can be the product or sum of the information complexity value and the rendering hierarchy value.

[0156] Step S450: Based on the importance values ​​of each level, obtain the rendering priority of the second type of image patch.

[0157] In embodiments of this application, the average value of the importance values ​​at each level can be directly used as the rendering priority of the second type of image patch. That is, the higher the average value of the importance values ​​at each level, the higher the rendering priority of the second type of image patch.

[0158] It is important to note that when rendering a scene, nearby objects may occlude distant objects. Therefore, starting rendering from the lowest depth level ensures that rendered objects fill the depth buffer, avoiding display errors caused by occlusion during subsequent rendering. If the rendering priority of the second type of image patch is determined solely based on the level importance value, this rendering priority may be affected by the user's perspective. If the key edge contour is not within the user's good visual range, the rendering effect will not have a significant impact on the user's vision. Based on this, in order to obtain a more accurate rendering priority, in one embodiment of this application, step S450, which involves obtaining the rendering priority of the second type of image patch based on the importance values ​​of each level, includes steps S451 to S453.

[0159] Step S451: Obtain the visual cone spatial boundary from the edge detection image.

[0160] It's important to understand that visual cone space refers to a cone-shaped spatial region from the observer's perspective. Typically, visual cone space consists of a near clipping plane, a far clipping plane, a left clipping plane, a right clipping plane, a top clipping plane, and a bottom clipping plane. The near clipping plane is the plane closest to the camera and defines the front boundary of the visual cone space; the far clipping plane is the plane farthest from the camera and defines the rear boundary of the visual cone space; the left, right, top, and bottom clipping planes define the left, right, top, and bottom boundaries of the visual cone space, forming a three-dimensional "cone." Objects within the visual cone space are usually visible, while objects outside the visual cone space are invisible. Since visual cone space is a mature technology, it will not be discussed further here.

[0161] Step S452: Based on the visual cone space boundary, obtain the probability that the key edge contour is located in the visual cone space.

[0162] In the embodiments of this application, any reasonable method can be used to obtain the probability that the key edge contour is located in the visual cone space based on the visual cone space boundary. For example: step S452, obtaining the probability that the key edge contour is located in the visual cone space based on the visual cone space boundary, includes steps S454 to S456.

[0163] Step S454: Based on the key edge contour, obtain the first distance and the first area.

[0164] In this embodiment, the first distance is the average depth value of each pixel corresponding to the key edge contour. The first area is the area of ​​the key edge contour.

[0165] Step S455: Based on the visual cone spatial boundary, obtain the second distance and the second area.

[0166] In this embodiment, the second distance is the depth value of the far clipping plane in the visual cone spatial boundary, and the depth value of the far clipping plane is the average depth value of all pixels on the far clipping plane. The second area is the area of ​​the near clipping plane in the visual cone spatial boundary.

[0167] Step S456: Based on the first distance, the second distance, the first area, and the second area, obtain the probability that the key edge contour is located in the visual cone space.

[0168] It should be clear that, in the embodiments of this application, any reasonable method can be used to obtain the probability that the key edge contour is located in the visual cone space based on the first distance, the second distance, the first area, and the second area. For example, in step S456, the formula for calculating the probability that the key edge contour is located in the visual cone space based on the first distance, the second distance, the first area, and the second area can be as follows:

[0169]

[0170] in, This indicates the probability that the key edge contour lies within the visual cone space; () represents a normalization function, used to normalize the values ​​within the parentheses to the range [0, 1]. Indicates the first area; Indicates the second area; Indicates the first distance; Indicates the second distance.

[0171] In this embodiment, if The larger the ratio, the more likely the rendering layer corresponding to the key edge contour is within the range of the near clipping plane; if The larger the ratio, the more likely the rendering layer corresponding to the key edge contour is within the visible range of the visual cone. In other words, if... The larger the value, the greater the probability that the rendering layer containing the key edge contour is located in the visual cone space. In other words, the rendering layer corresponding to the key edge contour is more important for the current rendering scene.

[0172] Step S453: Based on the importance values ​​of each level and the probability, obtain the rendering priority of the second type of image patch.

[0173] In the embodiments of this application, the rendering priority of the second type of image patch can be obtained in any reasonable way based on the importance values ​​of each level and the probability. For example, in step S453, the calculation formula for obtaining the rendering priority of the second type of image patch based on the importance values ​​of each level and the probability is as follows:

[0174]

[0175] Where Q represents the rendering priority of the second type of image patch; m represents the number of key edge contours corresponding to the second type of image patch; This represents the hierarchical importance value corresponding to the x-th key edge contour of the second type of image patch; This represents the probability corresponding to the x-th key edge contour of the second type of image patch.

[0176] Step S500: Based on the rendering priority, each second-type image block is rendered individually, and based on the block rendering algorithm, each first-type image block is rendered in blocks.

[0177] In the embodiments of this application, each second-type image block can be rendered individually based on rendering priority. During this process, the rendering layer with higher rendering priority should be allocated more rendering resources and use high-quality rendering techniques (e.g., high-resolution textures, complex lighting). Since rendering each second-type image block based on rendering priority is a mature technique, it will not be elaborated upon here.

[0178] In the embodiments of this application, for each first type of image block, based on the block rendering algorithm, each first type of image block can be further divided into smaller blocks, and the smaller blocks obtained by division can be rendered using the same rendering method as the aforementioned second type of image blocks.

[0179] Step S600: Based on the rendering results of each first-class image block and each second-class image block in all image regions, obtain the rendered image.

[0180] In the embodiments of this application, the rendering results of each first-class image block and each second-class image block within all image areas are merged. Rendering result merging is one of the key steps of the distributed rendering system. It is responsible for receiving rendering data from multiple rendering servers, decompressing and deserializing the data, and finally merging the data into the final rendered image based on depth and color information.

[0181] The embodiment of the composite distributed rendering method based on depth map algorithm and block-based algorithm proposed in this application firstly divides the image region of the scene image into preliminary blocks according to existing methods; then, it uses information such as contour shape, color, and depth in the image blocks to obtain the information complexity of each image block; then, based on the information complexity value, each first image block is classified into a first type of image block and a second type of image block; based on the depth map algorithm, the rendering priority of each second type of image block is obtained; finally, based on the rendering priority, each second type of image block is rendered separately, and based on the block-based rendering algorithm, each first type of image block is rendered in blocks. The rendering resources are allocated through adaptive block-based results, thereby making full use of rendering resources and improving rendering quality and efficiency.

[0182] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0183] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the methods, apparatuses, and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0184] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or modules may be electrical, mechanical, or other forms.

[0185] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0186] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0187] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0188] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video optical disc), or a semiconductor medium (e.g., solid-state drive (SSD)).

[0189] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles of this application.

Claims

1. A composite distributed rendering method based on a depth map algorithm and a block algorithm, characterized in that, The method comprises the following steps: dividing a scene image into a plurality of image regions, dividing the image regions into blocks, and obtaining a plurality of first image blocks; obtaining information complexity values corresponding to the first image blocks based on an edge detection algorithm; the information complexity values are used at least to represent the information complexity of the corresponding first image blocks; classifying the first image blocks into first-type image blocks and second-type image blocks based on the information complexity values; the information complexity value of the first-type image blocks is higher than that of the second-type image blocks; obtaining rendering priorities corresponding to the second-type image blocks based on a depth map algorithm; the rendering priorities are used at least to represent the difficulty of rendering the corresponding second-type image blocks; individually rendering the second-type image blocks based on the rendering priorities, and block rendering the first-type image blocks based on a block rendering algorithm; obtaining a rendered image based on the rendering results of the first-type image blocks and the second-type image blocks in all the image regions; the method comprises the following steps: obtaining a texture complexity value of the first image block based on the edge detection algorithm; the texture complexity value is used at least to represent the texture complexity of the first image block; obtaining a depth complexity value of the first image block; the depth complexity value is used at least to represent the difference in depth values of the pixels in the first image block; obtaining the information complexity value of the first image block based on the texture complexity value and the depth complexity value; the method comprises the following steps: obtaining an edge detection image based on the first image block by using the edge detection algorithm; obtaining a plurality of variance values based on the edge detection image; the variance value is the variance of the slopes between adjacent pixels in any edge line in the edge detection image; obtaining an information entropy based on the edge detection image; the information entropy is the information entropy of the average gray values of all the edge lines in the edge detection image; obtaining the texture complexity value of the first image block based on the information entropy and the variance values.

2. The composite distributed rendering method based on the depth map algorithm and the block algorithm of claim 1, wherein, The calculation formula for obtaining the texture complexity value of the first image block based on the information entropy and the variance values is as follows: Wherein, f represents the texture complexity value corresponding to the first image block; n represents the number of edge lines in the edge detection image corresponding to the first image block; Wherein, f represents the texture complexity value corresponding to the first image block; n represents the number of edge lines in the edge detection image corresponding to the first image block; 3. The composite distributed rendering method based on the depth map algorithm and the block algorithm of claim 1, wherein, the method comprises the following steps: obtaining the depth values of the pixels in each edge line based on the edge detection image; obtaining the depth complexity value of the first image block based on the depth values; the depth complexity value is the average of the standard deviations of the depth values of the pixels in each edge line.

4. The composite distributed rendering method based on the depth map algorithm and the block algorithm according to any one of claims 1 to 3, characterized in that, the method comprises the following steps: obtaining a depth map of each second-type image block based on a depth map algorithm; obtaining a key edge contour based on the edge detection image and the depth map; the key edge contour is an edge contour with complex depth changes in the second-type image block; obtaining a rendering level value corresponding to each key edge contour based on the key edge contour. Based on the information complexity value and the rendering level value, a level importance value of a rendering layer corresponding to each key edge contour is obtained; Based on each level importance value, a rendering priority of the second type of image block is obtained.

5. The composite distributed rendering method based on the depth map algorithm and the block algorithm of claim 4, wherein, The key edge contour is obtained based on the depth map and the edge detection image, including: Based on the depth map, a depth value corresponding to each pixel in each edge line is obtained from the edge detection image; Based on each depth value, a standard deviation of each pixel in each edge line is obtained; Each standard deviation is arranged in order from small to large to obtain a first sequence; The first sequence is divided to obtain a second sequence; The contour composed of the edge line corresponding to the standard deviation in the second sequence is taken as the key edge contour.

6. The composite distributed rendering method based on the depth map algorithm and the block algorithm of claim 5, wherein, The first sequence is divided to obtain a second sequence, including: Based on the first sequence, a difference sequence is obtained; Based on the difference sequence, a maximum value is obtained; Based on the maximum value, a first standard deviation and a second standard deviation are obtained; the first standard deviation and the second standard deviation are the standard deviations in the first sequence that obtain the maximum value, and the second standard deviation is greater than the first standard deviation; The second standard deviation and the standard deviations after the second standard deviation in the first sequence are combined to form the second sequence.

7. The composite distributed rendering method based on the depth map algorithm and the block algorithm of claim 4, wherein, Based on each level importance value, a rendering priority of the second type of image block is obtained, including: From the edge detection image, a visual pyramid space boundary is obtained; Based on the visual pyramid space boundary, a probability that the key edge contour is located in the visual pyramid space is obtained; Based on each level importance value and the probability, a rendering priority of the second type of image block is obtained.

8. The composite distributed rendering method based on the depth map algorithm and the block algorithm of claim 7, wherein, Based on the visual pyramid space boundary, a probability that the key edge contour is located in the visual pyramid space is obtained, including: Based on the key edge contour, a first distance and a first area are obtained; the first distance is an average depth value of each pixel corresponding to the key edge contour; the first area is an area of the key edge contour; Based on the visual pyramid space boundary, a second distance and a second area are obtained; the second distance is a depth value of a far clipping plane in the visual pyramid space boundary; the second area is an area of a near clipping plane in the visual pyramid space boundary; Based on the first distance, the second distance, the first area and the second area, a probability that the key edge contour is located in the visual pyramid space is obtained.

Images