An ultra-high-definition picture block slicing, parallel rendering and output display method

Abstract

This invention relates to the field of computer graphics processing and display technology, and discloses a method for parallel rendering and output display of ultra-high-definition images through block slicing. The method includes: acquiring an ultra-high-definition image; calculating a global density distribution map using a low-resolution Mipmap layer; and performing non-uniform slicing using a dynamic quadtree algorithm to obtain basic slices; using the boundaries of the basic slices as a reference, dynamically expanding the width outward according to the maximum filter kernel radius to obtain extended slices containing overlapping regions; allocating the extended slices to multiple pipelines for independent parallel rendering; calculating fusion weights using overlapping pixels and performing weighted mixing processing to obtain merged image data; and using a memory page table mapping mechanism to zero-copy map the merged image data to the output frame buffer, and extracting data within the visible area and submitting it to the display controller for local refresh. This invention solves the problems of uneven load, edge tearing, and memory bandwidth congestion, achieving extremely low-latency roaming of ultra-high-definition images.

Description

Technical Field

[0001] This invention relates to the field of computer graphics processing and display technology, and in particular to a method for parallel rendering and output display of ultra-high-definition images by slicing and dividing them into blocks. Background Technology

[0002] With the rapid development of display technology, ultra-high-definition images such as 8K and 16K have been widely used in fields such as large-scale video walls, virtual reality, and high-precision medical remote sensing images. Ultra-high-definition images contain massive amounts of pixel data. In modern GPU rendering pipelines, large images are typically sliced ​​into blocks and then processed in parallel by multiple rendering pipelines. However, existing parallel rendering methods have significant technical drawbacks:

[0003] First, due to the uneven distribution of complexity and detail across different regions of an image, direct slicing leads to an extremely unbalanced computational load across the rendering pipelines, causing severe pipeline congestion and rendering delays. Second, modern advanced rendering algorithms (such as anti-aliasing and flooding) require cross-pixel sampling. Traditional hard slicing causes the loss of contextual information at the slice edges, easily resulting in out-of-bounds errors or sample truncation during final merging. It also inevitably produces visual artifacts such as Mach band effects due to discontinuities in the first derivative of the image or abrupt transition seams. Finally, in the image merging and output stages, traditional methods require a full physical copy of massive pixel data to the output frame buffer via the bus. This causes significant memory bandwidth congestion and instantaneous peaks, making it difficult to achieve smooth interaction with ultra-high resolution images on ordinary terminal devices. Summary of the Invention

[0004] This invention provides a method for parallel rendering and output display of ultra-high-definition images by segmenting and slicing. By adaptive dynamic segmentation based on content density, redundant edge nonlinear fusion based on filter kernel, and zero-copy viewport local refresh technology, it solves the problems of uneven computational load, visual tearing at the edges, and congestion of video memory bandwidth in parallel rendering of ultra-high-definition images, and achieves efficient, seamless, and extremely low-latency display and interaction of ultra-high resolution images.

[0005] This invention provides a method for parallel rendering and output display of ultra-high-definition images through block slicing, including:

[0006] S1. Obtain the ultra-high-definition image to be rendered, and use the low-resolution Mipmap layer of the ultra-high-definition image to calculate the global density distribution map. Based on the global density distribution map, use the dynamic quadtree algorithm to perform non-uniform slice segmentation on the ultra-high-definition image to obtain multiple basic slices.

[0007] S2. For each base slice, expand the dynamic width outward based on the boundary of the base slice to obtain the corresponding extended slice; wherein, the dynamic width is determined according to the maximum filter kernel radius used by the rendering algorithm to be executed, and the extended slice includes the base slice area and the overlapping area extended outward.

[0008] S3. The multiple extended slices are assigned to multiple rendering pipelines for independent parallel rendering to obtain multiple corresponding rendered slices. The overlapping pixels corresponding to the overlapping area in the adjacent rendered slices are extracted. The fusion weight is calculated based on the overlapping pixels to perform weighted mixing processing on the color values ​​of adjacent rendered slices at the same coordinate position to obtain merged image data.

[0009] S4. Using the video memory page table mapping mechanism, the merged image data is zero-copy mapped to the output frame buffer of the current device's physical video memory, and the visible area of ​​the current device is obtained. The merged image data belonging to the visible area is extracted from the physical video memory and submitted to the display controller of the current device for local refresh and output display.

[0010] Furthermore, S1 specifically includes:

[0011] S101. Obtain the ultra-high-definition image to be rendered, and extract the low-resolution Mipmap layer of the specified level of the ultra-high-definition image by downsampling.

[0012] S102. Based on the pixel data of the low-resolution Mipmap layer, calculate the local information entropy or high-frequency gradient density by traversing a local window of a preset size, and map the calculation results of each local window into a two-dimensional matrix to construct a global density distribution map corresponding to the ultra-high-definition image.

[0013] S103. Establish the coordinate mapping relationship between the global density distribution map and the ultra-high-definition image based on the downsampling ratio coefficient, and use the ultra-high-definition image as the root node to perform a top-down recursive space partitioning using the dynamic quadtree algorithm.

[0014] S104. In each partitioning iteration, the density integral value corresponding to the currently partitioned sub-region in the global density distribution map is queried according to the coordinate mapping relationship, and it is determined whether the density integral value corresponding to the sub-region partitioned each time meets the preset load balancing condition. If it meets the condition, the partitioning of the branch is stopped, and the final leaf node region is used as the base slice to obtain multiple base slices.

[0015] Furthermore, in S102, the specific process of calculating the local information entropy is as follows:

[0016] The probability distribution of image grayscale values ​​within the local window is statistically analyzed, and the local information entropy H of the local window is calculated using the Shannon information entropy formula. The calculation formula is as follows:

[0017]

[0018] Where i represents the grayscale level of the pixel. This represents the probability that a pixel with grayscale value i will appear within the current local window.

[0019] The specific process for calculating the high-frequency gradient density is as follows:

[0020] The pixels within the local window are convolved using an edge detection operator to obtain the gradient magnitude of each pixel. The gradient magnitudes exceeding a preset high-frequency threshold within the local window are then accumulated to obtain the high-frequency gradient density value of the local window.

[0021] Furthermore, in S104, the preset load balancing conditions specifically include:

[0022] Determine whether the density integral value corresponding to the current sub-region on the global density distribution map is less than or equal to a preset single-threaded computing capability threshold, or whether the quadtree partitioning depth of the current sub-region has reached a preset maximum depth threshold.

[0023] If the density integral value is greater than the preset single-thread computing capability threshold but does not reach the maximum depth threshold, it is determined that the preset load balancing condition is not met, and the quadtree quadrating operation is continued to be performed on the current sub-region.

[0024] If the density integral value is less than or equal to the preset single-threaded computing capability threshold, or if the quadtree partitioning depth reaches the maximum depth threshold, then the preset load balancing condition is satisfied, triggering the recursive termination condition of the current branch and solidifying it into a leaf node.

[0025] Furthermore, S2 specifically includes:

[0026] S201. Analyze the rendering state of the current rendering computation pipeline and extract the post-processing or filtering algorithms involving spatial sampling in the rendering algorithm to be executed.

[0027] S202. Obtain the size N×N of the filter kernel matrix of each extracted post-processing or filtering algorithm, where N is an odd number. Calculate the corresponding sampling radius R according to the formula R=(N-1) / 2. Compare the sampling radii of all post-processing or filtering algorithms to be executed and select the maximum value. The pixel value of the dynamic width D is set to or Where C is the preset safety tolerance pixel value;

[0028] S203. For each of the base slices, extract its absolute physical coordinates in the ultra-high-definition image, and calculate the outward expansion coordinates using boundary clamping logic in conjunction with the dynamic width; wherein, the expanded upper left corner coordinates are updated to... The expanded bottom right corner coordinates are updated to W and H represent the total width and total height of the ultra-high-definition image, respectively. These are the top-left and bottom-right coordinates of the base slice in the ultra-high-definition image, respectively.

[0029] S204. Based on the outward expansion coordinates, extract pixel blocks from the ultra-high-definition image as the extended slices, and mark the outwardly extended pixel areas as the overlapping areas.

[0030] Furthermore, S3 specifically includes:

[0031] S301. Using a multi-threaded asynchronous scheduling mechanism, the multiple extended slices are allocated to multiple rendering computation pipelines for independent parallel rendering, and multiple corresponding rendered slices are output.

[0032] S302. Extract overlapping pixels corresponding to the overlapping region from adjacent rendered slices by coordinate matching, and calculate the Euclidean distance from the overlapping pixel to the boundary of its original base slice.

[0033] S303. Using a nonlinear smooth step function based on a cubic polynomial, calculate the fusion weights for eliminating boundary gaps based on the Euclidean distance.

[0034] S304. Calculate the final output color value at the same coordinate position using the fusion weights. After traversing the overlapping region pixel by pixel and performing the weighted mixing process, combine the original color data of the non-overlapping region with the color data of the mixed overlapping region to obtain the complete ultra-high-definition merged image data; wherein, the output color value... The calculation formula is:

[0035]

[0036] in, The output color value of the overlapping pixel P in the rendered slice A. This represents the output color value of the overlapping pixel P within the rendered slice B. These are the fusion weights of the overlapping pixel point P with respect to the rendered slices A and B, respectively.

[0037] Furthermore, S303 specifically includes:

[0038] Using the nonlinear smooth step function based on a cubic polynomial, the attenuation fusion weight of overlapping pixel point P with respect to the rendered slice A is calculated. The calculation formula is as follows:

[0039]

[0040] Wherein, overlapping pixel P belongs to the boundary overlapping area of ​​adjacent rendered slice A and rendered slice B; D is the dynamic width; Let P be the vertical Euclidean distance from the overlapping pixel point P to the original base slice boundary of the rendered slice A. The range of values ​​is ;

[0041] Determine the blending weight of the overlapping pixel P with respect to the rendered slice B. , .

[0042] Furthermore, S4 specifically includes:

[0043] S401. Using the video memory page table mapping mechanism, the logical address space where the merged image data is located is zero-copy mapped to the output frame buffer of the current device's physical video memory; wherein, the zero-copy mapping modifies the underlying video memory page table to remap the pointer of the physical memory page storing the merged image data to the effective virtual address of the output frame buffer.

[0044] S402. Real-time acquisition of the viewport status parameters of the current device, construction of a two-dimensional bounding box based on the viewport status parameters, and definition of the rectangular area covered by the projection of the two-dimensional bounding box into the global two-dimensional coordinate system of the merged image data as the visible area; wherein, the viewport status parameters include the scaling factor, viewport translation coordinates, and the physical resolution of the current device screen;

[0045] S403. Perform an intersection test between the coordinate range of the visible area and the pre-constructed spatial index tree corresponding to the basic slice, filter out the target slice block that spatially overlaps with the visible area, activate and extract the merged image data corresponding to the target slice block from the physical video memory, and use it as the local merged image data.

[0046] S404. Submit the extracted local merged image data to the display controller of the current device to drive the display panel to perform local refresh and output display. At the same time, listen to the roaming interaction operation of the current device. When a translation or zoom causes a local position shift in the visible area, extract the local merged image data moved into the new visible area according to the new visible area after the shift, and submit it to the display controller for local refresh.

[0047] The present invention also provides an ultra-high-definition image segmentation and parallel rendering output display device, based on the ultra-high-definition image segmentation and parallel rendering output display method described above, the device comprising:

[0048] The acquisition module is used to acquire the ultra-high-definition image to be rendered, and to calculate the global density distribution map using the low-resolution Mipmap layer of the ultra-high-definition image. Based on the global density distribution map, the dynamic quadtree algorithm is used to perform non-uniform slice segmentation on the ultra-high-definition image to obtain multiple basic slices.

[0049] An extension module is used to extend the dynamic width outward based on the boundary of each base slice to obtain a corresponding extended slice; wherein, the dynamic width is determined according to the maximum filter kernel radius used by the rendering algorithm to be executed, and the extended slice includes the base slice area and the overlapping area extended outward;

[0050] The rendering module is used to allocate multiple extended slices to multiple rendering computation pipelines for independent parallel rendering to obtain multiple corresponding rendered slices, extract overlapping pixels corresponding to the overlapping area in adjacent rendered slices, calculate fusion weights based on the overlapping pixels, and perform weighted mixing processing on the color values ​​of adjacent rendered slices at the same coordinate position to obtain merged image data.

[0051] The display module is used to use the video memory page table mapping mechanism to zero-copy map the merged image data to the output frame buffer of the current device's physical video memory, obtain the visible area of ​​the current device, extract the merged image data belonging to the visible area from the physical video memory, and submit it to the display controller of the current device for local refresh and output display.

[0052] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0053] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0054] The beneficial effects of this invention are as follows:

[0055] This invention extracts low-resolution Mipmap layers and combines them with a dynamic quadtree algorithm for non-uniform slicing, ensuring balanced computational load across rendering pipelines and eliminating latency caused by parallel rendering. It adaptively expands the overlapping region of the slices with dynamic width based on the maximum filter kernel radius and uses a non-linear smooth step function to calculate fusion weights for weighted mixing. This preserves sufficient contextual information of adjacent pixels while ensuring smooth transitions at boundaries, eliminating slice boundary gaps and visual artifacts caused by parallel rendering. Finally, it combines zero-copy mapping of memory page tables with a viewport-driven local refresh mechanism to avoid substantial physical transport of massive pixel data, reducing memory bandwidth congestion and display controller throughput pressure. This enables extremely low-latency roaming and smooth interaction of ultra-high resolution images even on ordinary consumer-grade terminal devices with limited memory capacity. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of a method flow according to an embodiment of the present invention.

[0057] Figure 2 This is a schematic diagram of the device structure according to an embodiment of the present invention.

[0058] Figure 3 This is a schematic diagram of the internal structure of a computer device according to an embodiment of the present invention.

[0059] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0060] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0061] like Figure 1 As shown, this invention provides a method for parallel rendering and output display of ultra-high-definition images through block slicing, including:

[0062] S1. Obtain the ultra-high-definition image to be rendered, and calculate the global density distribution map using the low-resolution Mipmap layer of the ultra-high-definition image. Based on the global density distribution map, use the dynamic quadtree algorithm to perform non-uniform slice segmentation on the ultra-high-definition image to obtain multiple basic slices.

[0063] In a preferred embodiment, step S1 specifically includes the following sub-steps:

[0064] S101. Obtain the ultra-high-definition image to be rendered, and generate or extract the low-resolution Mipmap layer of the ultra-high-definition image.

[0065] Because ultra-high-definition images (such as images with a resolution of 8K or 16K) have massive amounts of pixel data, directly performing full image traversal and feature calculation at the original resolution would consume enormous computing resources and cause severe rendering delays. Therefore, this embodiment first obtains the original ultra-high-definition image to be rendered, and then downsamples it using a hardware texture sampler or preprocessing algorithm to construct a multi-level mipmap pyramid.

[0066] Preferably, based on the current hardware's computing power and a preset performance threshold, a low-resolution mipmap layer at a specific level (e.g., Level 3 or Level 4) is selected as the data source for subsequent calculations. Using a low-resolution layer allows for the preservation of the original image's macroscopic structural features and pixel distribution trends with extremely low GPU memory and computing power overhead.

[0067] S102. Based on the pixel data of the low-resolution Mipmap layer, calculate the local information entropy or high-frequency gradient density, and construct a global density distribution map corresponding to the ultra-high-definition image.

[0068] After obtaining the selected low-resolution Mipmap layer, the image data of that layer is traversed using a sliding window of a preset size (e.g., a 3×3 or 5×5 pixel window), and the image complexity within each window region is calculated. The image complexity can be measured by local information entropy or high-frequency gradient density.

[0069] In one specific embodiment, high-frequency gradient density is used as the metric. An edge detection operator (such as the Sobel operator or the Laplacian operator) is used to perform convolution operations on the pixels within the window to calculate the gradient magnitude of each pixel. The gradient magnitudes exceeding a preset high-frequency threshold within the window are then accumulated to obtain the high-frequency gradient density value for that region.

[0070] In another specific embodiment, local information entropy is used as the metric. The probability distribution of image grayscale values ​​within the statistical window is calculated, and the entropy value is obtained using the Shannon information entropy formula:

[0071]

[0072] in, This represents the probability that a pixel with grayscale value i will appear within the current local window.

[0073] The complexity values ​​of all local windows obtained through traversal calculation are mapped to a two-dimensional matrix to generate a global density distribution map. Regions with higher values ​​in this distribution map represent areas with dense geometric details or complex textures in the corresponding locations in the ultra-high-definition original image (i.e., areas with high rendering computation), while regions with lower values ​​represent areas with flat transitions in the corresponding locations in the original image (i.e., areas with low rendering computation).

[0074] S103. Establish the coordinate mapping relationship between the global density distribution map and the ultra-high-definition image, and use the entire ultra-high-definition image as the root node to perform a top-down recursive space partitioning using a dynamic quadtree algorithm.

[0075] Since the global density distribution map is calculated based on the low-resolution Mipmap layer, the coordinate mapping relationship between the two is first established according to the downsampling ratio coefficient (for example, if the density map is calculated using a Mipmap layer scaled down to 1 / 8 of its width and height, then a 1×1 pixel on the density map corresponds to an 8×8 pixel area in the original image).

[0076] Subsequently, the dynamic quadtree algorithm is initialized, treating the complete ultra-high-definition image as the root node of the quadtree. In each partitioning iteration, the image region corresponding to the current node is uniformly divided into four sub-regions, and the density integral values ​​(i.e., the estimated rendering computation) of the corresponding four sub-regions are queried in the global density distribution map according to the above coordinate mapping relationship.

[0077] S104. Determine whether the density integral value corresponding to each sub-region meets the preset load balancing condition. If it does, stop dividing the branch and use the final leaf node region as the base slice to obtain a set of base slices of different sizes.

[0078] The preset load balancing conditions mainly include: determining whether the sum of density integrals corresponding to the current sub-region on the global density distribution map is less than or equal to a preset single-threaded computing capacity threshold. Or, has the current quadtree partition depth reached the maximum allowed depth? .

[0079] If the sum of the density integrals of the current sub-region is still greater than Furthermore, the maximum depth has not been reached, indicating that the details in this area are too dense. Rendering it as a whole using a single pipeline would still cause pipeline congestion. Therefore, it is necessary to continue performing a quadtree division operation on this sub-region. If the density integral sum of the current sub-region is less than or equal to... If the maximum depth has been reached, the recursive termination condition for that branch is triggered, and the partitioning stops. At this point, the sub-region is solidified as a leaf node of the quadtree.

[0080] After traversing the entire quadtree, the image regions corresponding to all generated leaf nodes constitute multiple basic slices. This allows complex areas to be divided into smaller but more numerous basic slices, while flat, solid-color areas are retained as larger basic slices. This ensures that the computation time required for each output basic slice during subsequent parallel rendering is approximately equal.

[0081] S2. For each base slice, expand the dynamic width outward based on the boundary of the base slice to obtain the corresponding extended slice; wherein, the dynamic width is determined according to the maximum filter kernel radius used by the rendering algorithm to be executed, and the extended slice includes the base slice area and the overlapping area extended outward.

[0082] In a preferred embodiment, step S2 specifically includes the following sub-steps:

[0083] S201. Analyze the rendering state of the current rendering pipeline, and traverse to extract the parameter information of each post-processing or filtering algorithm involving spatial sampling in the rendering algorithm to be executed.

[0084] In modern GPU rendering pipelines for ultra-high-definition images, multiple algorithms that require sampling adjacent pixels are typically layered to achieve high-quality visual effects (such as anti-aliasing SMAA / FXAA, ambient occlusion SSAO, depth of field DepthofField, Gaussian blur, or bloom). First, the shader instruction set to be executed in the current frame or slice is analyzed to extract a list of all algorithms requiring cross-pixel sampling, and the corresponding filter kernel size or maximum texture sampling offset for these algorithms is obtained.

[0085] S202. Calculate the sampling radius corresponding to each algorithm, and select the maximum value as the maximum filter kernel radius to determine the dynamic width D.

[0086] For any extracted filtering algorithm, assuming its filter kernel matrix size is N×N (where N is usually an odd number), the required sampling radius R is (N-1) / 2. Compare the sampling radii of all algorithms to be executed in the current rendering pipeline. Take the maximum value among them. .

[0087] To ensure that pixels at the slice edge still receive sufficient context information and do not experience out-of-bounds errors or sampling truncation even in the most extreme filtering scenarios, the pixel value of the dynamic width D is strictly set to be equal to the maximum filter kernel radius. (In some redundant designs, it can also be set as) Where C is a preset minimum safety tolerance pixel, typically 1 or 2). This adaptive mechanism ensures that the overlapping area is neither too small, causing edge tearing, nor too large, unnecessarily consuming valuable video memory bandwidth.

[0088] S203. For each of the basic slices, extract its absolute physical coordinates in the ultra-high-definition image, and perform an expansion operation on the boundary coordinates in combination with the dynamic width D.

[0089] Specifically, assume the total width of the original ultra-high-definition image is W, and the total height is H. For any basic slice in the set of basic slices, let its coordinates in the top-left corner of the original image be H. The coordinates of the lower right corner are .

[0090] When expanding the dynamic width D outwards from the boundary of the base slice, to prevent the expanded coordinates from exceeding the actual physical boundary of the original ultra-high-definition image, boundary clamping logic is used to calculate the new coordinates of the expanded slice:

[0091] The expanded top-left corner coordinates are updated as follows: ;

[0092] The expanded bottom right corner coordinates are updated as follows: .

[0093] Through the above clamping calculation, the actual pixel capture range corresponding to the extended slice in the ultra-high-definition original image is precisely defined.

[0094] S204. Based on the calculated outward coordinates, extract pixel blocks from the video memory data of the ultra-high-definition image as the extended slice, and mark the outwardly extended pixel area as the overlapping area.

[0095] Data is extracted based on the new coordinate range calculated according to S203, and the resulting image block is the complete extended slice. The extended slice physically consists of two parts: the inner core area is the original base slice area, and the annular area surrounding the core area with a maximum width of D is the overlapping area.

[0096] Traverse all the basic slices in the basic slice set and perform the operations from S201 to S204 above to finally generate an extended slice set consisting of multiple extended slices with adaptive overlapping regions.

[0097] S3. The multiple extended slices are assigned to multiple rendering pipelines for independent parallel rendering to obtain multiple corresponding rendered slices. The overlapping pixels corresponding to the overlapping area in the adjacent rendered slices are extracted. The fusion weight is calculated based on the overlapping pixels to perform weighted mixing processing on the color values ​​of adjacent rendered slices at the same coordinate position to obtain merged image data.

[0098] In a preferred embodiment, step S3 specifically includes the following sub-steps:

[0099] S301. Using a multi-threaded asynchronous scheduling mechanism, the multiple extended slices are allocated to multiple rendering computation pipelines for independent parallel rendering, and multiple corresponding rendered slices are output.

[0100] In ultra-high-definition image rendering scenarios, the main control CPU or GPU scheduler constructs an asynchronous task queue. Each extended slice in the extended slice set generated in step S2 is treated as an independent rendering task and distributed to multiple available computing units or rendering pipelines at the underlying level.

[0101] Since the quadtree algorithm ensured a balanced estimated rendering computation for each basic slice in step S1, and sufficient context pixels (overlapping areas) were adaptively added to each slice in step S2, each rendering pipeline can achieve truly independent rendering without synchronization waiting between pipelines or cross-memory access when performing complex post-processing or filtering algorithms (such as anti-aliasing, flooding, etc.). After rendering is complete, each pipeline outputs its corresponding result to a temporary buffer in video memory, forming multiple rendered slices.

[0102] S302. In the output merging stage, overlapping pixels corresponding to the overlapping area in adjacent rendered slices are extracted by coordinate matching, and the Euclidean distance from the overlapping pixel to the boundary of its original base slice is calculated.

[0103] Because adjacent extended slices have an overlapping region of width D in physical coordinates, when these rendered slices are stitched together into a complete image, there will be two (or more) overlapping pixels from different pipelines at the same absolute physical coordinate position. For any overlapping pixel P(x,y), let it belong to the boundary overlap region of adjacent rendered slices A and B. Calculate the Euclidean distance from pixel P to the original base slice boundary of slice A. And the Euclidean distance to the original basic slice boundary of slice B. .

[0104] Based on geometric relationships, constraints exist. (In an orthogonal grid system) the distance d reflects the degree to which the pixel is biased toward a non-overlapping area of ​​a slice core.

[0105] S303. Using a nonlinear smooth step function, calculate the fusion weights for eliminating boundary gaps based on the Euclidean distance.

[0106] Traditional linear blending can easily lead to discontinuities in the first derivative of an image when processing high-frequency detail at image boundaries, resulting in visual effects such as Mach banding or abrupt transition seams. To address this, this embodiment introduces a nonlinear smoothing step function based on a cubic polynomial to calculate the fusion weights. .

[0107] The distance from pixel P to the original base slice boundary of the rendered slice A For example (where) The attenuation fusion weight of this pixel with respect to slice A The calculation formula is:

[0108]

[0109] Where D is the dynamic width of the overlapping region determined in step S2; The vertical Euclidean distance from the current overlapping pixel to the boundary of the original base slice A; this nonlinear function in When (i.e., close to the core boundary), the derivative is 0, and the weight approaches 1; When the outermost edge of the overlapping region is reached, the derivative is 0, and the weights approach 0. This ensures that weight changes occur at the boundary. Continuous (the first derivative is continuous).

[0110] Similarly, the fusion weight of this pixel with respect to slice B can be The calculation yields the result, or, to ensure weight conservation, the result is taken directly. .

[0111] S304. Using the calculated fusion weights, the color values ​​of adjacent rendered slices at the same coordinate position are weighted and mixed to obtain merged image data.

[0112] Obtain the output color value of the overlapping pixel P in the rendered slice A. (Typically includes RGB channels) and the output color values ​​in the rendered slice B. .

[0113] The final output color value at this coordinate position is calculated using the following Alpha blending formula. :

[0114]

[0115] By traversing the overlapping regions pixel by pixel and performing the aforementioned weighted blending process, pixels from different pipelines with slight rendering differences at their edges are seamlessly merged, eliminating slice boundary gaps and visual artifacts caused by parallel rendering. Subsequently, the original color data of the non-overlapping regions is combined with the blended color data of the overlapping regions to obtain a complete and continuous ultra-high-definition merged image.

[0116] S4. Using the video memory page table mapping mechanism, the merged image data is zero-copy mapped to the output frame buffer of the current device's physical video memory, and the visible area of ​​the current device is obtained. The merged image data belonging to the visible area is extracted from the physical video memory and submitted to the display controller of the current device for local refresh and output display.

[0117] In a preferred embodiment, step S4 specifically includes the following sub-steps:

[0118] S401. Using the video memory page table mapping mechanism, the logical address space where the merged image data generated in step S3 is located is mapped to the output frame buffer of the current device's physical video memory with a "zero copy".

[0119] Traditional image merging and output processes typically require physically copying a large amount of pixel data from a dedicated temporary video memory area for rendering to the output frame buffer bound to the display controller via the bus. For ultra-high-definition images, this full copy causes significant video memory bandwidth congestion and instantaneous peaks. To address this issue, this embodiment employs a low-level zero-copy technique. Since each rendered slice has already undergone edge blending and been written to the physical blocks of video memory in step S3, by directly modifying the GPU's low-level video memory page table, the pointers to the physical memory pages storing the merged image data are remapped to the effective virtual addresses of the output frame buffer. This ensures that the massive amount of pixel data does not undergo any substantial data transfer on the physical video memory medium; only the address pointers are redirected. This mechanism eliminates the bandwidth bottleneck caused by writing ultra-high-definition large images back to video memory.

[0120] S402. Obtain the viewport status parameters of the current device in real time, and calculate and determine the visible area of ​​the current device in the global coordinate system of the merged image data.

[0121] Because the total resolution of an ultra-high-definition image (e.g., 15360×8640) is usually much larger than the physical resolution of a current device screen (e.g., a standard 3840×2160 4K screen), a user can only observe a portion of the ultra-high-definition image at any given time.

[0122] The system monitors user interactions in real time to obtain the current device's viewport status parameters. These parameters include, but are not limited to, the user's zoom level, viewport translation coordinates, and the physical resolution of the current device screen. Based on these parameters, a two-dimensional bounding box is constructed. The precise rectangular area covered by the projection of this bounding box onto the global two-dimensional coordinate system of the merged image data is defined as the visible area.

[0123] S403. Using a spatial indexing mechanism, based on the coordinate range of the visible area, accurately extract locally merged image data belonging to the visible area from the physical memory.

[0124] To avoid scanning the entire ultra-high-definition image, a spatial index tree (such as a quadtree bounding box index or a hash grid index) corresponding to the base slices is maintained in memory. The coordinate range of the visible area calculated in S402 is intersected with this spatial index to quickly filter out slice blocks that spatially overlap with the visible area.

[0125] Subsequently, only the merged image data corresponding to the intersecting portions is activated and extracted from the output frame buffer of the physical video memory. Hidden image data that is outside the visible area and does not intersect is kept in a silent resident state and does not participate in subsequent reading and transmission bus usage.

[0126] S404. The extracted local merged image data is submitted to the display controller of the current device to drive the display panel to perform local refresh and output display.

[0127] The display controller only needs to read the pixel data within the visible area and convert it into timing signals (such as HDMI, DisplayPort, or eDP signals) required by the display panel for output display. When the user performs continuous roaming interactions such as panning or zooming, causing a local position shift in the visible area, it only needs to incrementally update the slice data that has moved into the new visible area and submit it to the display controller for local refresh, without having to re-render and submit the entire ultra-high-definition image. This viewport-driven on-demand local refresh mechanism greatly reduces the throughput pressure on the display controller, enabling extremely low-latency roaming and smooth interaction of ultra-high-resolution images even on ordinary consumer-grade terminal devices with limited video memory.

[0128] like Figure 2 As shown, the present invention also provides an ultra-high-definition image segmentation and parallel rendering output display device, based on the ultra-high-definition image segmentation and parallel rendering output display method described above, the device comprising:

[0129] The acquisition module 1 is used to acquire the ultra-high-definition image to be rendered, and to calculate the global density distribution map using the low-resolution Mipmap layer of the ultra-high-definition image. Based on the global density distribution map, the dynamic quadtree algorithm is used to perform non-uniform slice segmentation on the ultra-high-definition image to obtain multiple basic slices.

[0130] Extension module 2 is used to extend the dynamic width outward based on the boundary of each base slice to obtain a corresponding extended slice; wherein, the dynamic width is determined according to the maximum filter kernel radius used by the rendering algorithm to be executed, and the extended slice includes the base slice area and the overlapping area extended outward;

[0131] Rendering module 3 is used to allocate multiple extended slices to multiple rendering computation pipelines for independent parallel rendering to obtain multiple corresponding rendered slices, extract overlapping pixels corresponding to the overlapping area in adjacent rendered slices, calculate fusion weights based on the overlapping pixels, and perform weighted mixing processing on the color values ​​of adjacent rendered slices at the same coordinate position to obtain merged image data.

[0132] Display module 4 is used to use the video memory page table mapping mechanism to zero-copy map the merged image data to the output frame buffer of the physical video memory of the current device, obtain the visible area of ​​the current device, extract the merged image data belonging to the visible area from the physical video memory, and submit it to the display controller of the current device for local refresh and output display.

[0133] Each of the above modules is used to perform the corresponding steps in the above-described ultra-high-definition image block slicing parallel rendering output display method. The specific implementation method is as described in the above method embodiment, and will not be repeated here.

[0134] like Figure 3 As shown, the present invention also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores all data required for the process of the ultra-high-definition image tile-slicing parallel rendering output display method. The network interface is used for communication with external terminals via a network connection. The computer program is executed by the processor to implement the ultra-high-definition image tile-slicing parallel rendering output display method.

[0135] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0136] An embodiment of this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-described ultra-high-definition image block slicing parallel rendering output display methods.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by hardware related to computer program instructions. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. Any references to memory, storage, databases, or other media provided in this application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), such as dynamic RAM (used as main storage) or static RAM (commonly used as cache memory). By way of illustration and not limitation, RAM has various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), and Rambus DRAM (RDRAM).

[0138] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0139] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for parallel rendering and output display of ultra-high-definition images through block slicing, characterized in that, include: S1. Obtain the ultra-high-definition image to be rendered, and use the low-resolution Mipmap layer of the ultra-high-definition image to calculate the global density distribution map. Based on the global density distribution map, use the dynamic quadtree algorithm to perform non-uniform slice segmentation on the ultra-high-definition image to obtain multiple basic slices. S2. For each base slice, expand the dynamic width outward based on the boundary of the base slice to obtain the corresponding extended slice; wherein, the dynamic width is determined according to the maximum filter kernel radius used by the rendering algorithm to be executed, and the extended slice includes the base slice area and the overlapping area extended outward. S3. The multiple extended slices are assigned to multiple rendering pipelines for independent parallel rendering to obtain multiple corresponding rendered slices. The overlapping pixels corresponding to the overlapping area in the adjacent rendered slices are extracted. The fusion weight is calculated based on the overlapping pixels to perform weighted mixing processing on the color values ​​of adjacent rendered slices at the same coordinate position to obtain merged image data. S4. Using the video memory page table mapping mechanism, the merged image data is zero-copy mapped to the output frame buffer of the current device's physical video memory, and the visible area of ​​the current device is obtained. The merged image data belonging to the visible area is extracted from the physical video memory and submitted to the display controller of the current device for local refresh and output display.

2. The ultra-high-definition image segmentation and parallel rendering output display method according to claim 1, characterized in that, S1 specifically includes: S101. Obtain the ultra-high-definition image to be rendered, and extract the low-resolution Mipmap layer of the specified level of the ultra-high-definition image by downsampling. S102. Based on the pixel data of the low-resolution Mipmap layer, calculate the local information entropy or high-frequency gradient density by traversing a local window of a preset size, and map the calculation results of each local window into a two-dimensional matrix to construct a global density distribution map corresponding to the ultra-high-definition image. S103. Establish the coordinate mapping relationship between the global density distribution map and the ultra-high-definition image based on the downsampling ratio coefficient, and use the ultra-high-definition image as the root node to perform a top-down recursive space partitioning using the dynamic quadtree algorithm. S104. In each partitioning iteration, the density integral value corresponding to the currently partitioned sub-region in the global density distribution map is queried according to the coordinate mapping relationship, and it is determined whether the density integral value corresponding to the sub-region partitioned each time meets the preset load balancing condition. If it meets the condition, the partitioning of the branch is stopped, and the final leaf node region is used as the base slice to obtain multiple base slices.

3. The ultra-high-definition image segmentation and parallel rendering output display method according to claim 2, characterized in that, In step S102, the specific process for calculating the local information entropy is as follows: The probability distribution of image grayscale values ​​within the local window is statistically analyzed, and the local information entropy H of the local window is calculated using the Shannon information entropy formula. The calculation formula is as follows: Where i represents the grayscale level of the pixel. This represents the probability that a pixel with grayscale value i will appear within the current local window. The specific process for calculating the high-frequency gradient density is as follows: The pixels within the local window are convolved using an edge detection operator to obtain the gradient magnitude of each pixel. The gradient magnitudes exceeding a preset high-frequency threshold within the local window are then accumulated to obtain the high-frequency gradient density value of the local window.

4. The ultra-high-definition image block slicing parallel rendering output display method according to claim 2, characterized in that, In step S104, the preset load balancing conditions specifically include: Determine whether the density integral value corresponding to the current sub-region on the global density distribution map is less than or equal to a preset single-threaded computing capability threshold, or whether the quadtree partitioning depth of the current sub-region has reached a preset maximum depth threshold. If the density integral value is greater than the preset single-thread computing capability threshold but does not reach the maximum depth threshold, it is determined that the preset load balancing condition is not met, and the quadtree quadrating operation is continued to be performed on the current sub-region. If the density integral value is less than or equal to the preset single-threaded computing capability threshold, or if the quadtree partitioning depth reaches the maximum depth threshold, then the preset load balancing condition is satisfied, triggering the recursive termination condition of the current branch and solidifying it into a leaf node.

5. The ultra-high-definition image segmentation and parallel rendering output display method according to claim 1, characterized in that, S2 specifically includes: S201. Analyze the rendering state of the current rendering computation pipeline and extract the post-processing or filtering algorithms involving spatial sampling in the rendering algorithm to be executed. S202. Obtain the size N×N of the filter kernel matrix of each extracted post-processing or filtering algorithm, where N is an odd number. Calculate the corresponding sampling radius R according to the formula R=(N-1) / 2. Compare the sampling radii of all post-processing or filtering algorithms to be executed and select the maximum value. The pixel value of the dynamic width D is set to or Where C is the preset safety tolerance pixel value; S203. For each of the base slices, extract its absolute physical coordinates in the ultra-high-definition image, and calculate the outward expansion coordinates using boundary clamping logic in conjunction with the dynamic width; wherein, the expanded upper left corner coordinates are updated to... The expanded bottom right corner coordinates are updated to W and H represent the total width and total height of the ultra-high-definition image, respectively. These are the top-left and bottom-right coordinates of the base slice in the ultra-high-definition image, respectively. S204. Based on the outward expansion coordinates, extract pixel blocks from the ultra-high-definition image as the extended slices, and mark the outwardly extended pixel areas as the overlapping areas.

6. The ultra-high-definition image block slicing parallel rendering output display method according to claim 1, characterized in that, S3 specifically includes: S301. Using a multi-threaded asynchronous scheduling mechanism, the multiple extended slices are allocated to multiple rendering computation pipelines for independent parallel rendering, and multiple corresponding rendered slices are output. S302. Extract overlapping pixels corresponding to the overlapping region from adjacent rendered slices by coordinate matching, and calculate the Euclidean distance from the overlapping pixel to the boundary of its original base slice. S303. Using a nonlinear smooth step function based on a cubic polynomial, calculate the fusion weights for eliminating boundary gaps based on the Euclidean distance. S304. Calculate the final output color value at the same coordinate position using the fusion weights. After traversing the overlapping region pixel by pixel and performing the weighted mixing process, combine the original color data of the non-overlapping region with the color data of the mixed overlapping region to obtain the complete ultra-high-definition merged image data; wherein, the output color value... The calculation formula is: in, The output color value of the overlapping pixel P in the rendered slice A. This represents the output color value of the overlapping pixel P within the rendered slice B. These are the fusion weights of the overlapping pixel point P with respect to the rendered slices A and B, respectively.

7. The ultra-high-definition image segmentation and parallel rendering output display method according to claim 6, characterized in that, Specifically, S303 includes: Using the nonlinear smooth step function based on a cubic polynomial, the attenuation fusion weight of overlapping pixel point P with respect to the rendered slice A is calculated. The calculation formula is as follows: Wherein, overlapping pixel P belongs to the boundary overlapping area of ​​adjacent rendered slice A and rendered slice B; D is the dynamic width; Let P be the vertical Euclidean distance from the overlapping pixel point P to the original base slice boundary of the rendered slice A. The range of values ​​is ; Determine the blending weight of the overlapping pixel P with respect to the rendered slice B. , .

8. The ultra-high-definition image segmentation and parallel rendering output display method according to claim 1, characterized in that, S4 specifically includes: S401. Using the video memory page table mapping mechanism, the logical address space where the merged image data is located is zero-copy mapped to the output frame buffer of the current device's physical video memory; wherein, the zero-copy mapping modifies the underlying video memory page table to remap the pointer of the physical memory page storing the merged image data to the effective virtual address of the output frame buffer. S402. Real-time acquisition of the viewport status parameters of the current device, construction of a two-dimensional bounding box based on the viewport status parameters, and definition of the rectangular area covered by the projection of the two-dimensional bounding box into the global two-dimensional coordinate system of the merged image data as the visible area; wherein, the viewport status parameters include the scaling factor, viewport translation coordinates, and the physical resolution of the current device screen; S403. Perform an intersection test between the coordinate range of the visible area and the pre-constructed spatial index tree corresponding to the basic slice, filter out the target slice block that spatially overlaps with the visible area, activate and extract the merged image data corresponding to the target slice block from the physical video memory, and use it as the local merged image data. S404. Submit the extracted local merged image data to the display controller of the current device to drive the display panel to perform local refresh and output display. At the same time, listen to the roaming interaction operation of the current device. When a translation or zoom causes a local position shift in the visible area, extract the local merged image data moved into the new visible area according to the new visible area after the shift, and submit it to the display controller for local refresh.

9. A device for parallel rendering and output display of ultra-high-definition images by segmentation and slicing, based on the method for parallel rendering and output display of ultra-high-definition images by segmentation and slicing according to any one of claims 1 to 8, characterized in that, The device includes: The acquisition module is used to acquire the ultra-high-definition image to be rendered, and to calculate the global density distribution map using the low-resolution Mipmap layer of the ultra-high-definition image. Based on the global density distribution map, the dynamic quadtree algorithm is used to perform non-uniform slice segmentation on the ultra-high-definition image to obtain multiple basic slices. An extension module is used to extend the dynamic width outward based on the boundary of each base slice to obtain a corresponding extended slice; wherein, the dynamic width is determined according to the maximum filter kernel radius used by the rendering algorithm to be executed, and the extended slice includes the base slice area and the overlapping area extended outward; The rendering module is used to allocate multiple extended slices to multiple rendering computation pipelines for independent parallel rendering to obtain multiple corresponding rendered slices, extract overlapping pixels corresponding to the overlapping area in adjacent rendered slices, calculate fusion weights based on the overlapping pixels, and perform weighted mixing processing on the color values ​​of adjacent rendered slices at the same coordinate position to obtain merged image data. The display module is used to use the video memory page table mapping mechanism to zero-copy map the merged image data to the output frame buffer of the current device's physical video memory, obtain the visible area of ​​the current device, extract the merged image data belonging to the visible area from the physical video memory, and submit it to the display controller of the current device for local refresh and output display.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images