A holographic zoo off-screen rendering optimization method and device under low computing power equipment

Abstract

This application provides an optimization method and apparatus for off-screen rendering of a holographic zoo on low-computing-power devices, relating to the field of real-time rendering technology. It involves creating a dual buffer for the off-screen rendering pipeline of the holographic zoo; performing motion saliency detection on the area to be rendered to obtain a directly reused area and a region to be determined; for each pixel block in the region to be determined, when the cumulative reuse count of that pixel block in the second buffer exceeds a dynamic threshold, the pixel block is marked as a forced redraw area; a complete off-screen redraw is performed on the forced redraw area; pixel data is copied from the first buffer for the directly reused area; linear blending is performed on the remaining regions to be determined; the pixel data in the first buffer is updated, the cumulative reuse count of redrawn pixel blocks is reset to zero, and the cumulative reuse count of unredrawn pixel blocks is increased by one. This application can achieve efficient reuse of the off-screen rendering frame buffer, thereby reducing computational power consumption while eliminating image ghosting.

Description

Technical Field

[0001] This application relates to the field of real-time rendering technology, and more specifically, to a method and apparatus for optimizing off-screen rendering of a holographic zoo on low-computing-power devices. Background Technology

[0002] Real-time rendering technology, as the core support for building immersive 3D visual scenes, is widely used in scenarios such as holographic displays and virtual interactions. Real-time rendering technology can complete model drawing, lighting and shadow calculation and image output in real time, providing users with continuous and realistic dynamic visual effects. On low-computing-power terminal devices, efficient real-time rendering technology is the key to ensuring the stable operation of lightweight applications such as holographic zoos. It can improve the smoothness of scenes under limited hardware conditions and has important application value in promoting the development of immersive holographic experiences towards portability and popularization.

[0003] In order to reduce computational overhead, existing low-computing-power devices generally use frame buffer data reuse to reduce redundant rendering calculations when performing holographic off-screen rendering. However, in holographic zoo scenes, dynamic targets move frequently and have complex trajectories. Directly reusing frame buffer data can easily cause problems such as image ghosting and motion blur, resulting in a decline in visual experience. If conventional methods are used to eliminate ghosting, a large amount of additional image calculation is required, which will further occupy the device's limited graphics processing resources and reduce the rendering frame rate. Current technology is difficult to balance computational power saving and image clarity, and lacks a reasonable frame buffer reuse control mechanism. Therefore, how to achieve efficient reuse of the off-screen rendering frame buffer, thereby reducing computational power consumption while eliminating image ghosting, has become a problem faced by the industry. Summary of the Invention

[0004] This application provides a method and apparatus for optimizing off-screen rendering of a holographic zoo on low-computing-power devices, which can achieve efficient reuse of off-screen rendering frame buffers, thereby reducing computing power consumption while eliminating image ghosting.

[0005] In a first aspect, this application provides an off-screen rendering optimization method for a holographic zoo on low-computing-power devices, the off-screen rendering optimization method for a holographic zoo comprising the following steps: Create a double buffer for the off-screen rendering pipeline of the holographic zoo, where the first buffer stores the complete rendering result of the previous frame and the second buffer stores the cumulative reuse count of each pixel block in the previous frame; When rendering the current frame, motion saliency detection is performed on the area to be rendered. The exhibition background area that is static is marked as a directly reused area, and the animal area that is moving is marked as an area to be determined. For each pixel block in the region to be determined, the cumulative reuse count of that pixel block in the second buffer is read. When the cumulative reuse count exceeds the dynamic threshold, the pixel block is marked as a region to be forced to redraw. Perform a full off-screen redraw for the forced redraw area, copy pixel data from the first buffer for the directly reused area, and linearly blend the pixel data from the previous frame with the partial redraw result of the current frame for the remaining areas to be determined. After rendering is complete, update the pixel data in the first buffer, reset the cumulative reuse count of the redrawn pixel blocks to zero, and increment the cumulative reuse count of the unredrawn pixel blocks by one.

[0006] In this embodiment, a dual-buffer system is created for the off-screen rendering pipeline of the holographic zoo. The first buffer stores the complete rendering result of the previous frame, and the second buffer stores the cumulative reuse count of each pixel block in the previous frame. Specifically, this includes: The size of the pixel blocks is determined based on the complexity of the exhibit and animal models in the holographic zoo. The rendering target is divided into a grid-like array of pixels based on the size of the pixel blocks; A first off-screen rendering texture is allocated as a first buffer for the pixel block array. The first off-screen rendering texture is used to store the complete color and depth data of each pixel block in the previous frame. A second off-screen rendering texture is allocated as a second buffer for the pixel block array. The second off-screen rendering texture is used to store the cumulative number of times each pixel block was reused in the previous frame in the form of integer data.

[0007] In this embodiment, when rendering the current frame, motion saliency detection is performed on the area to be rendered. Areas where the detection result indicates a stationary exhibition hall background are marked as directly reusable areas, while areas where the detection result indicates moving animals are marked as areas to be determined. Specifically, this includes: Obtain the animal skeleton animation data and camera view transformation parameters for the current frame; The animal skeleton animation data and camera view transformation parameters of the previous frame are compared with the corresponding data of the current frame to generate the motion vector field of the animal model in screen space. The motion vector field is subjected to region growing and morphological filtering to generate a motion saliency mask; Based on the motion saliency mask, the areas to be rendered with mask values ​​below the first threshold are marked as static exhibition hall background areas, i.e., directly reused areas; The areas to be rendered with a mask value higher than the second threshold are marked as moving animal areas, i.e., areas to be determined.

[0008] In this embodiment, for each pixel block in the region to be determined, the cumulative reuse count of that pixel block in the second buffer is read. When the cumulative reuse count exceeds a dynamic threshold, the pixel block is marked as a region to be forcibly redrawn. Specifically, this includes: Traverse all pixel blocks covered by the region to be determined, and obtain the index address of each pixel block in the second buffer; Read the cumulative reuse count of each pixel block according to the index address; The accumulated reuse count value is compared with a preset dynamic threshold, which is adjusted according to the rendering load of the current frame and the power consumption and temperature status of the device. If the cumulative reuse count is greater than the dynamic threshold, the pixel block is marked as a forced redraw region in the region mask.

[0009] In this embodiment, performing a complete off-screen redraw on the forced redraw area specifically includes: Get the set of pixel coordinates covered by the mask of the forced redraw region; Construct the bounding box of the forced redraw region in screen space based on the set of pixel block coordinates; Calculate the clipping plane offset of the view frustum based on the bounding box of the forced redraw region in screen space; Based on the clipping plane offset of the view frustum, the clipping plane of the original view frustum is asymmetrically shrunk to generate a redrawn view frustum; Rendering nodes that are completely outside the redrawn view frustum are marked as culled nodes, and rendering nodes that are partially or completely inside the redrawn view frustum are marked as nodes to be rendered. Perform a complete vertex transformation and rasterization process for all nodes to be rendered, generate pixel data for the redraw area, and write it to the first buffer.

[0010] In this embodiment, the linear blending of the pixel data from the previous frame with the partial redrawing result of the current frame for the remaining areas to be determined specifically includes: Obtain the list of pixel indexes covered by the remaining area to be determined; Read the pixel block data of the previous frame corresponding to the pixel block index list from the first buffer; Read the cumulative reuse count of each pixel block from the pixel block index list in the second buffer; A pixel reuse confidence field is constructed for each pixel block based on the cumulative reuse count of each pixel block and the cumulative reuse count of its neighboring pixel blocks. The pixel reuse confidence field is used to characterize the reliability of the reused data of each pixel point. The animal model corresponding to the remaining region to be determined is divided into an edge vertex set and an internal vertex set. The edge vertex set is processed by the full shader pipeline, and the internal vertex set is processed by the simplified shader pipeline to generate the partial redraw result of the current frame. Based on the pixel reuse confidence field of each pixel block, edge-preserving guided filtering fusion is performed on the pixel block data of the previous frame and the partial redrawing result of the current frame to generate mixed pixel data.

[0011] In this embodiment, after rendering is completed, the pixel data in the first buffer is updated, and the cumulative reuse count of the redrawn pixel blocks is reset to zero. The cumulative reuse count of the non-redrawn pixel blocks is increased by one. Specifically, this includes: Submit the final pixel data of the forced redraw region and linear blend region generated in this rendering to the graphics application programming interface, and update the corresponding pixel blocks in the first buffer. For all pixels marked as forced redraw areas and participating in linear blending in this rendering, reset the cumulative reuse count in the second buffer to the initial value of zero; For all pixel blocks marked as directly reusable regions in this rendering, increase the cumulative reuse count in the second buffer by one from the original value. After the second buffer is updated, the pointers of the first and second buffers are swapped, so that the first buffer points to the rendering result of the current frame and the second buffer points to the cumulative reuse count of the current frame.

[0012] In this embodiment, the animal model corresponding to the remaining region to be determined is divided into an edge vertex set and an internal vertex set. Full shader pipeline processing is performed on the edge vertex set, and simplified shader pipeline processing is performed on the internal vertex set. The specific steps to generate the partial redraw result for the current frame include: Obtain the vertex buffer data and triangle facets of the animal model corresponding to the remaining regions to be determined; Traverse each triangular facet of the animal model and identify the triangular facets located at the edges of the animal model's outline; The vertices contained in the triangular facets at the edge of the animal model outline are labeled as the edge vertex set, and the remaining vertices of the animal model are labeled as the interior vertex set; A full vertex shader is run for each vertex in the edge vertex set, and a simplified vertex shader is run for each vertex in the interior vertex set, wherein the simplified vertex shader only performs model-view projection transformation; A full fragment shader is run on the fragments covered by the edge vertex set, and a simplified fragment shader is run on the fragments covered by the internal vertex set. The simplified fragment shader samples color values ​​from the previous frame buffer and offsets the sampled coordinates according to the motion vector field of the current frame. The rendering results of the edge vertex set and the internal vertex set are merged to generate the partial redraw result of the current frame.

[0013] In this embodiment, the edge-preserving guided filtering fusion of the previous frame pixel block data and the current frame partial redrawing result is performed based on the pixel reuse confidence field of each pixel block to generate the mixed pixel data. Specifically, this includes: Obtain the pixel reuse confidence field for each pixel block, downsample the pixel reuse confidence field to generate a multi-layer confidence pyramid, and simultaneously construct a multi-layer image pyramid for the pixel block data of the previous frame and the partial redrawing result of the current frame respectively. Starting from the top layer of the multi-layer confidence pyramid, traverse layer by layer. For each pixel in the current layer, determine the local window radius of the guided filter based on the confidence value of that pixel. Extract the pixel block data of the previous frame corresponding to the current layer from the multi-layer image pyramid of the previous frame pixel block data, and extract the redraw result of the current frame corresponding to the current layer from the multi-layer image pyramid of the current frame partial redraw result. Within the radius of the local window, calculate the local covariance matrix and local mean vector of the pixel block data of the previous frame corresponding to the current layer and the partial redrawing result of the current frame corresponding to the current layer. The linear transformation parameters of the guided filter are calculated based on the local covariance matrix and the local mean vector. The linear transformation parameters are then applied to the pixel block data of the previous frame corresponding to the current layer to generate the fusion result of the current layer. The fusion result of the current layer is upsampled, and the upsampled fusion result is used as the guide image for the next layer of fusion. The above process is repeated until the bottom layer of the multi-layer confidence pyramid is reached to generate the final mixed pixel data.

[0014] Secondly, this application provides a holographic zoo off-screen rendering optimization device for low-computing-power devices. This holographic zoo off-screen rendering optimization device includes an off-screen rendering optimization unit, which comprises: The dual-buffer creation module is used to create dual buffers for the off-screen rendering pipeline of the holographic zoo. The first buffer stores the complete rendering result of the previous frame, and the second buffer stores the cumulative reuse count of each pixel block in the previous frame. The motion saliency detection module is used to perform motion saliency detection on the area to be rendered when rendering the current frame. The exhibition hall background area that is detected as stationary is marked as a directly reusable area, and the animal area that is detected as moving is marked as an area to be judged. The forced redraw region marking module is used to read the cumulative reuse count of each pixel block in the second buffer for each pixel block in the region to be judged. When the cumulative reuse count exceeds the dynamic threshold, the pixel block is marked as a forced redraw region. The partition rendering module is used to perform a complete off-screen redraw for the forced redraw area, copy pixel data from the first buffer for the directly reused area, and linearly blend the pixel data of the previous frame with the partial redraw result of the current frame for the remaining areas to be determined. The rendering data update module is used to update the pixel data in the first buffer after rendering is completed, reset the cumulative reuse count of the redrawn pixel blocks to zero, and increase the cumulative reuse count of the non-redrawn pixel blocks by one.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: A dual-buffer system is created for the off-screen rendering pipeline of the holographic zoo. The first buffer stores the complete rendering result of the previous frame, and the second buffer stores the cumulative reuse count of each pixel block in the previous frame. When rendering the current frame, motion saliency detection is performed on the area to be rendered. The exhibit background area that is detected as stationary is marked as a directly reused area, and the animal area that is detected as moving is marked as a pending area. For each pixel block in the pending area, the cumulative reuse count of that pixel block in the second buffer is read. When the cumulative reuse count exceeds a dynamic threshold, the pixel block is marked as a forced redraw area. A complete off-screen redraw is performed on the forced redraw area. For the directly reused area, pixel data is copied from the first buffer. For the remaining pending areas, the pixel data of the previous frame is linearly blended with the partial redraw result of the current frame. After rendering is completed, the pixel data in the first buffer is updated, the cumulative reuse count of the redrawn pixel blocks is reset to zero, and the cumulative reuse count of the unredrawn pixel blocks is increased by one.

[0016] Therefore, in this application, after rendering is completed, the pixel data in the first buffer is updated, and the cumulative reuse count of the redrawn pixel blocks is reset to zero, while the cumulative reuse count of the unredrawn pixel blocks is increased by one. Firstly, by creating a dual buffer for the off-screen rendering pipeline to store the rendering result of the previous frame and the cumulative reuse count of pixel blocks respectively, and configuring the buffer by dividing the pixel block array according to model complexity, fine-grained recording and management of frame buffer reuse status can be achieved, while avoiding image anomalies caused by indiscriminate reuse of frame buffers. Secondly, a motion saliency mask is generated based on the difference between animal skeletal animation and camera viewpoint parameters, thereby dividing the static background directly reuse area and the moving animal undetermined area. This can retain the advantages of frame buffer reuse in static areas and reduce redundant rendering computational overhead while accurately locating moving areas prone to ghosting, achieving a preliminary balance between computational saving and targeted rendering. Furthermore, by reading the cumulative reuse count of pixel blocks and combining it with a dynamic threshold to determine the forced redraw area, It can promptly trigger redrawing of long-reused pixel blocks, avoiding ghosting and motion blur issues caused by continuous reuse of frame data. Moreover, it only redraws local areas, without significantly increasing the device's computing power burden. Furthermore, it simplifies the rendering process by shrinking the view frustum for the forced redrawing area, and uses model vertex layering and inter-frame pixel linear blending for the remaining undetermined areas. This ensures the rendering accuracy and image clarity of the moving area edges, and further reduces graphics resource consumption by simplifying shader calculations, effectively avoiding the frame rate drop problem caused by conventional ghosting elimination methods. Finally, after rendering is completed, the reuse count of the redrawing area is reset, the reuse count of the reused area is accumulated, and the buffer pointers are swapped, forming a closed-loop update mechanism for the inter-frame reuse state. This allows the frame buffer reuse strategy to adapt to dynamic scene changes, continuously achieving efficient reuse of off-screen rendering frame buffers on low-computing-power devices. While steadily reducing computing power consumption, it completely eliminates image ghosting and motion blur, ensuring a smooth visual experience in the holographic zoo scene.

[0017] In summary, the technical solution adopted in this application can achieve efficient reuse of off-screen rendering frame buffers, thereby reducing computing power consumption while eliminating image ghosting. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this embodiment of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is an exemplary flowchart of an off-screen rendering optimization method for a holographic zoo on a low-computing-power device, as provided in this application. Figure 2 This is an exemplary flowchart of off-screen rendering for low-computing-power devices provided in this application; Figure 3 This is a flowchart illustrating the motion saliency detection process provided in this application; Figure 4 This is a flowchart illustrating the guided filter fusion process provided in this application; Figure 5 This is a module structure diagram of a holographic zoo off-screen rendering optimization device under low computing power according to the present application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] This application provides a method and apparatus for optimizing off-screen rendering of a holographic zoo on low-computing-power devices. The core of this method is to create a dual-buffer system for the off-screen rendering pipeline of the holographic zoo. The first buffer stores the complete rendering result of the previous frame, and the second buffer stores the cumulative reuse count of each pixel block in the previous frame. When rendering the current frame, motion saliency detection is performed on the area to be rendered. Areas with static exhibit backgrounds are marked as directly reused areas, and areas with moving animals are marked as areas to be determined. For each pixel block in the area to be determined, the cumulative reuse count of that pixel block in the second buffer is read. When the cumulative reuse count exceeds a dynamic threshold, the pixel block is marked as a forced redraw area. A complete off-screen redraw is performed on the forced redraw area. Pixel data is copied from the first buffer for the directly reused areas. For the remaining areas to be determined, the pixel data from the previous frame is linearly mixed with the partial redraw result of the current frame. After rendering is complete, the pixel data in the first buffer is updated, and the cumulative reuse count of the redrawn pixel blocks is reset to zero, while the cumulative reuse count of the unredrawn pixel blocks is increased by one.

[0022] Example 1: To better understand the above technical solution, the following will provide a detailed description of the technical solution in conjunction with the accompanying drawings and specific implementation methods. (Refer to...) Figure 1 As shown in the figure, this is an exemplary flowchart of a holographic zoo off-screen rendering optimization method under a low computing power device according to this embodiment of the present application. The holographic zoo off-screen rendering optimization method includes the following steps: In step S1, a double buffer is created for the off-screen rendering pipeline of the holographic zoo, wherein the first buffer stores the complete rendering result of the previous frame, and the second buffer stores the cumulative reuse count of each pixel block in the previous frame.

[0023] In this embodiment, creating a dual-buffer system for the off-screen rendering pipeline of the holographic zoo, where the first buffer stores the complete rendering result of the previous frame and the second buffer stores the cumulative reuse count of each pixel block in the previous frame, can be achieved through the following steps: The size of the pixel blocks is determined based on the complexity of the exhibit and animal models in the holographic zoo. The rendering target is divided into a grid-like array of pixels based on the size of the pixel blocks; A first off-screen rendering texture is allocated as a first buffer for the pixel block array. The first off-screen rendering texture is used to store the complete color and depth data of each pixel block in the previous frame. A second off-screen rendering texture is allocated as a second buffer for the pixel block array. The second off-screen rendering texture is used to store the cumulative number of times each pixel block was reused in the previous frame in the form of integer data.

[0024] It should be noted that, in this application, the pixel block size refers to the minimum rendering unit specification of the screen; the pixel block array refers to a set of grid-like pixel units formed by uniformly dividing the off-screen rendering target according to a preset pixel block size; the first off-screen rendering texture refers to a graphics hardware texture resource used to store the complete rendering result of the previous frame (including color data and depth data of each pixel block); the second off-screen rendering texture refers to a graphics hardware texture resource used to store, in unsigned integer data format, the cumulative number of times each pixel block was not redrawn in the previous frame but directly reused; the cumulative reuse count refers to the number of times a single pixel block has not undergone a redraw operation for multiple consecutive frames and has directly reused the pixel data of historical frames.

[0025] In the specific implementation, firstly, the static meshes of all pavilion models in the holographic zoo scene are traversed to obtain the median of the projected area of ​​each pavilion model in screen space. Simultaneously, all animal models are traversed to obtain the bounding box volume of each bone. The average bounding box volume of all bones is calculated as a representation of the animal model complexity. The median of the projected area of ​​the pavilion models in screen space is then weighted and summed with the animal model complexity representation, where the weight coefficient for pavilion models is set to 0.3 and the weight coefficient for animal models is set to 0.7. The weighted sum is mapped to a preset range of pixel block sizes, with a lower limit of 8 pixels and an upper limit of 64 pixels. The side length of the pixel block is calculated using linear interpolation and used as the pixel block size. Secondly, according to the determined pixel block size, the width and height directions of the off-screen rendering target are equally divided, and all the divided pixel units are arranged in a mesh pattern. The arranged set is used as a grid-like pixel array. Next, a graphics application programming interface (such as the OpenGL glGenTextures function) is called to allocate a floating-point texture with the same resolution as the off-screen rendering target and adapted to the data storage requirements of color and depth data to the pixel array. This floating-point texture is used as the first off-screen rendering texture to store the complete color and depth data of each pixel block in the previous frame, i.e., as the first buffer. Finally, the above graphics application programming interface is called again to allocate an unsigned integer texture with the same resolution as the off-screen rendering target and adapted to the data storage requirements of integer count data to the pixel array. This texture only stores integer count data and does not store color and depth information. This unsigned integer texture is used as the second off-screen rendering texture to store the cumulative reuse count of each pixel block in the previous frame in the form of integer data, i.e., as the second buffer.

[0026] It should be noted that in this application, references Figure 2 As shown in the figure, this is an exemplary flowchart of off-screen rendering for low-computing-power devices provided in this application embodiment. The low-computing-power device on the left is responsible for collecting user interaction signals. The exhibition hall scene in the middle includes the moving animal main body and the static exhibition hall background, which is the core rendering area. In the dual-buffer module on the right, the first buffer stores the complete rendering result of the current frame, and the second buffer stores the cumulative reuse count of each pixel block in the previous frame. Relying on the dual buffer, the inter-frame reuse and incremental update of off-screen rendering are realized, and the motion area and interaction area are accurately divided for rendering. While ensuring the smoothness of the screen, the rendering computing power consumption of low-computing-power devices is significantly reduced, adapting to the performance bottleneck of terminal devices.

[0027] In step S2, when rendering the current frame, motion saliency detection is performed on the area to be rendered. The exhibition background area where the detection result is static is marked as the directly reused area, and the animal area where the detection result is moving is marked as the area to be determined.

[0028] Preferably, in this embodiment, when rendering the current frame, motion saliency detection is performed on the area to be rendered. The exhibition hall background area, where the detection result is static, is marked as a directly reused area, and the animal area, where the detection result is moving, is marked as an area to be determined. (Refer to...) Figure 3 As shown in the figure, this is a schematic flowchart of motion saliency detection in some embodiments of this application. In this embodiment, motion saliency detection can be implemented using the following steps: In step S21, the animal skeleton animation data and camera view transformation parameters of the current frame are obtained; In step S22, the animal skeleton animation data and camera view transformation parameters of the previous frame are compared with the corresponding data of the current frame to generate the motion vector field of the animal model in screen space. In step S23, the motion vector field is subjected to region growing and morphological filtering to generate a motion saliency mask; In step S24, based on the motion saliency mask, the areas to be rendered with mask values ​​below the first threshold are marked as static exhibition hall background areas, i.e., directly reused areas. In step S25, the areas to be rendered with mask values ​​higher than the second threshold are marked as moving animal areas, i.e., areas to be determined.

[0029] It should be noted that the animal skeletal animation data mentioned in this application refers to the displacement and rotation data of the skeletal nodes that drive the animal model to produce posture changes; the camera perspective transformation parameters refer to the parameters related to the camera's position movement, perspective rotation, and projection transformation in the scene; the motion vector field refers to the set of vectors that cause displacement changes of each vertex on the surface of the animal model in screen space; the directly reusable area refers to the area of ​​the exhibition hall background that can directly reuse historical frame rendering data without motion changes; and the area to be determined refers to the area of ​​the animal model that has motion changes and requires further determination of the rendering strategy.

[0030] In specific implementation, firstly, the skeletal node transformation data of the animal model in the current frame is read from the animation rendering pipeline of the holographic zoo. Simultaneously, the view transformation and projection transformation parameters of the camera in the current frame are read from the camera control module. These two sets of data are used as the animal skeletal animation data and camera view transformation parameters for the current frame, respectively. Secondly, the animal skeletal animation data and camera view transformation parameters of the previous frame are retrieved from the cache. The corresponding data from the previous frame and the current frame are subjected to component-wise difference calculations. The resulting vertex vectors are integrated to form the motion vector field of the animal model in screen space. Next, a region growing algorithm is used to extract connected components from the regions with vector changes in the motion vector field. Then, morphological opening operations are used to remove noise from the extracted connected components. The processed grayscale data of the connected components is used as a motion saliency mask. Then, screen regions with values ​​below a first threshold in the motion saliency mask are classified as static exhibition background regions, i.e., directly reused regions. Finally, screen regions with values ​​above a second threshold in the motion saliency mask are classified as areas with moving animals, i.e., regions to be judged.

[0031] It should be noted that, in this application, the first threshold represents the mask value boundary for distinguishing whether a screen area is in a static state. It can be set in the following way: First, traverse all pixels in the area to be rendered, distinguish between pavilion background pixels and animal pixels by model ID, calculate the proportion of pavilion background pixels in the entire area to be rendered, divide this proportion into three intervals, and set the first threshold accordingly. For example: when the proportion of pavilion background pixels is ≥60%, the first threshold is set to 0.12. This value indicates that the background area has a high proportion, and appropriately increasing the threshold can reduce noise misjudgment and ensure that most static backgrounds are recognized; when 30% < pavilion background pixel proportion < 60%, the first threshold is set to 0.08, adapting to scenes with a balanced proportion of background and animal areas, balancing recognition accuracy and rendering efficiency; when the proportion of pavilion background pixels is ≤30%, the first threshold is set to 0.05, avoiding missed judgments of background areas due to a low background proportion, and using the statistically obtained background proportion matched with the value of the corresponding interval as the first threshold. It should also be noted that the second threshold in this application represents the mask numerical boundary for distinguishing whether there is obvious motion in the screen area. It can be set in the following way: First, extract the displacement data of the skeletal nodes of all animal models in the holographic zoo within 10 consecutive frames, calculate the maximum displacement modulus of each skeletal node, and then take the average of the maximum displacement modulus of all skeletal nodes as the average motion amplitude of the animal model. Combined with the screen resolution, the average motion amplitude is divided into three intervals and the second threshold is set accordingly. For example: when the relative value of the average motion amplitude is ≥ 1‰ of the relative value of the screen width, the second threshold is set to 0.55 to adapt to the scene of fast animal movement and ensure that all areas of fast-moving animals are identified; when the relative value of the screen width is 0.3‰ < the relative value of the average motion amplitude is < 1‰ of the relative value of the screen width, the second threshold is set to 0.45 to adapt to the scene of slow animal movement and balance the recognition accuracy and redundancy; when the relative value of the average motion amplitude is ≤ 0.3‰ of the relative value of the screen width, the second threshold is set to 0.35 to adapt to the scene of slight animal movement and avoid misjudging the area of ​​slightly moving animals as background. The calculated average motion amplitude of the animal is matched with the value of the corresponding interval as the second threshold.

[0032] In step S3, for each pixel block in the region to be determined, the cumulative reuse count of the pixel block in the second buffer is read. When the cumulative reuse count exceeds the dynamic threshold, the pixel block is marked as a region to be forcibly redrawn.

[0033] In this embodiment, for each pixel block in the region to be determined, the cumulative reuse count of that pixel block in the second buffer is read. When the cumulative reuse count exceeds the dynamic threshold, the pixel block is marked as a region to be forcibly redrawn. This can be achieved by the following steps: Traverse all pixel blocks covered by the region to be determined, and obtain the index address of each pixel block in the second buffer; Read the cumulative reuse count of each pixel block according to the index address; The accumulated reuse count value is compared with a preset dynamic threshold, which is adjusted according to the rendering load of the current frame and the power consumption and temperature status of the device. If the cumulative reuse count is greater than the dynamic threshold, the pixel block is marked as a forced redraw region in the region mask.

[0034] It should be noted that, in this application, the pixel block index address represents the coordinate identifier of the pixel block in the texture space of the second buffer; the cumulative reuse count value represents the number of times the pixel block has continuously reused historical frame data since the last redraw; the dynamic threshold represents the critical value of reuse count that is adapted and adjusted according to the device operating state; the region mask represents the identifier texture data used to mark regions with different rendering strategies; and the forced redraw region represents the set of pixel blocks that need to perform a complete off-screen redraw.

[0035] In specific implementation, firstly, all pixel blocks covered by the region to be judged are traversed row by row according to the arrangement order of the pixel block array. Combining the coordinates of the pixel blocks in screen space with the texture mapping relationship of the second buffer, the mapped coordinates are used as the index address of each pixel block in the second buffer. Secondly, through the texture sampling instructions of the graphics hardware, the integer count data stored in the second buffer is read according to the index address corresponding to each pixel block, and the read integer data is used as the cumulative reuse count value of each pixel block. Next, the rendering load ratio of the graphics hardware in the current frame and the power consumption and temperature parameters of the device core are collected. The higher the rendering load and the higher the device temperature, the smaller the value of the dynamic threshold is assigned, and the lower the rendering load and the lower the device temperature, the larger the value of the dynamic threshold is assigned. The value adapted according to the running state is used as the dynamic threshold, and the cumulative reuse count value of each pixel block is compared with the dynamic threshold. Finally, for the pixel blocks whose cumulative reuse count value is greater than the dynamic threshold in the comparison result, the corresponding redraw identifier is assigned in the region mask, and the pixel block region marked with the redraw identifier is used as the forced redraw region.

[0036] It should be noted that the dynamic threshold mentioned in this application refers to the critical value of reuse count that is adapted and adjusted according to the device's operating status. It can be set in the following way: Call the hardware performance query command of the graphics application programming interface to collect the rendering load percentage of the graphics hardware in the current frame. This value is the real-time computing power occupancy percentage of the GPU rendering task. Simultaneously, read the real-time temperature value of the graphics hardware core through the system power management interface. Using a preset basic cumulative reuse count threshold as a benchmark value, which is the default reuse count threshold for the holographic zoo rendering scene, divide the rendering load percentage into three intervals and match them with weighting coefficients: a weighting coefficient of 1.2 when the rendering load is ≤30%, a weighting coefficient of 1.0 when 30% < rendering load < 70%, and a weighting coefficient of 1.0 when the rendering load is ≥70%. The weighting coefficient is 0.6 when the core temperature is 0%. The core temperature is divided into three intervals and matched with weighting coefficients: the weighting coefficient is 1.2 when the core temperature is ≤60℃, the weighting coefficient is 1.0 when the core temperature is <60℃ and <80℃, and the weighting coefficient is 0.6 when the core temperature is ≥80℃. The total weighting coefficient is calculated using a product weighting algorithm. The total weighting coefficient = rendering load weighting coefficient × temperature weighting coefficient. 15 times is used as the basic cumulative reuse threshold. The basic cumulative reuse threshold is multiplied by the total weighting coefficient and rounded down to obtain the initial dynamic threshold. At the same time, the value range of the dynamic threshold is limited to 5 to 20 times. If it is less than 5 times, it is fixed at 5 times. If it is more than 20 times, it is fixed at 20 times. The finally calculated integer value is used as the dynamic threshold of the cumulative reuse count for this rendering.

[0037] In step S4, a complete off-screen redraw is performed on the forced redraw area, pixel data is copied from the first buffer for the directly reused area, and the remaining areas to be determined are linearly mixed with the pixel data of the previous frame and the partial redraw result of the current frame.

[0038] In this embodiment, performing a complete off-screen redraw on the forced redraw area can be achieved using the following steps: Get the set of pixel coordinates covered by the mask of the forced redraw region; Construct the bounding box of the forced redraw region in screen space based on the set of pixel block coordinates; Calculate the clipping plane offset of the view frustum based on the bounding box of the forced redraw region in screen space; Based on the clipping plane offset of the view frustum, the clipping plane of the original view frustum is asymmetrically shrunk to generate a redrawn view frustum; Rendering nodes that are completely outside the redrawn view frustum are marked as culled nodes, and rendering nodes that are partially or completely inside the redrawn view frustum are marked as nodes to be rendered. Perform a complete vertex transformation and rasterization process for all nodes to be rendered, generate pixel data for the redraw area, and write it to the first buffer.

[0039] It should be noted that, in this application, the set of pixel block coordinates represents the sum of coordinate data of all pixel blocks in screen space under the mask marking of the forced redraw region; the bounding box represents the screen space axis-aligned rectangle range that encloses all forced redraw pixel blocks; the clipping plane offset represents the displacement value of each clipping plane of the original view frustum shrinking towards the redraw region; the original view frustum represents the conventional rendering view frustum used to render the entire area to be rendered; the redrawing view frustum represents the simplified view frustum that only covers the forced redraw region after asymmetric shrinkage; the rendering node represents the independent rendering unit corresponding to the pavilion model and animal model in the holographic zoo; the culling node represents the rendering unit that does not need to participate in this redraw process; the node to be rendered represents the rendering unit that needs to participate in this complete redraw process; the vertex transformation represents the coordinate transformation operation of converting model space vertices to clip space; and the rasterization represents the processing operation of converting geometric primitives into screen pixel fragments.

[0040] In specific implementation, firstly, all pixel blocks marked as redrawn in the forced redraw region mask are traversed, and the screen space coordinates corresponding to each pixel block are extracted. All extracted coordinate data are then aggregated as the set of pixel block coordinates covered by the forced redraw region mask. Secondly, the horizontal and vertical coordinate extreme values ​​in the pixel block coordinate set are traversed, and a rectangle is constructed with the coordinate extreme values ​​as the boundary axis. This constructed rectangle is used as the bounding box of the forced redraw region in screen space. Next, the inverse transformation function of the projection matrix in the graphics application programming interface is called to convert the screen coordinates of the bounding box into corresponding coordinates in the original view frustum clipping space. This transformation process establishes the association between the bounding box and the original view frustum projection mapping. Then, the distances of the bounding box to the four clipping planes (top, bottom, left, right) of the original view frustum in clipping space are calculated. These distances are multiplied by preset offset coefficients to obtain the displacement values ​​of each clipping plane. The farther the bounding box is from the original view frustum clipping plane, the larger the corresponding offset coefficient; the closer the distance, the smaller the offset coefficient. The calculated displacement value is used as the clipping plane offset of the view frustum. Then, based on the clipping plane offset, the top, bottom, left, and right clipping planes of the original view frustum are unidirectionally and asymmetrically shrunk, so that the shrunk view frustum completely fits the bounding box of the forced redraw region. The shrunk view frustum is then used as the redrawn view frustum. Subsequently, all rendering nodes in the holographic zoo scene are traversed. By comparing the spatial position relationship between the bounding box and the redrawn view frustum, rendering nodes whose bounding boxes are completely outside the redrawn view frustum are marked as culled nodes, and rendering nodes whose bounding boxes are partially or completely inside the redrawn view frustum are marked as nodes to be rendered. Finally, a complete vertex transformation process of model-view projection transformation, normal vector transformation, triangle clipping, perspective division, and viewport mapping is executed sequentially for all nodes to be rendered. Then, a complete rasterization process of fragment generation, depth testing, and color blending is executed. The pixel color and depth data obtained after the process are used as the pixel data of the redraw region. The pixel data is written to the corresponding position in the first buffer through the graphics hardware texture writing instruction.

[0041] In practice, copying pixel data from the first buffer for the directly reused region can be achieved in the following way: First, extract the coordinate range and pixel block array information corresponding to the directly reused region in screen space, and use the coordinate range and pixel block array information as the target addressing basis for data copying; Second, call the texture batch copy instruction of the graphics application programming interface to read out the pixel data in the first buffer that completely matches the coordinates of the directly reused region, and then write the read pixel data directly into the corresponding coordinate position of the current frame rendering target to complete the unmodified pixel data reuse operation.

[0042] In this embodiment, the linear blending of the pixel data from the previous frame with the partial redrawing result of the current frame for the remaining areas to be determined can be achieved using the following steps: Obtain the list of pixel indexes covered by the remaining area to be determined; Read the pixel block data of the previous frame corresponding to the pixel block index list from the first buffer; Read the cumulative reuse count of each pixel block from the pixel block index list in the second buffer; A pixel reuse confidence field is constructed for each pixel block based on the cumulative reuse count of each pixel block and the cumulative reuse count of its neighboring pixel blocks. The pixel reuse confidence field is used to characterize the numerical distribution field of the reliability of reused data for each pixel point. The animal model corresponding to the remaining region to be determined is divided into an edge vertex set and an internal vertex set. The edge vertex set is processed by the full shader pipeline, and the internal vertex set is processed by the simplified shader pipeline to generate the partial redraw result of the current frame. Based on the pixel reuse confidence field of each pixel block, edge-preserving guided filtering fusion is performed on the pixel block data of the previous frame and the partial redrawing result of the current frame to generate mixed pixel data.

[0043] It should be noted that, in this application, the remaining region to be determined refers to the set of pixel blocks in the region to be determined that are not marked as areas to be forcibly redrawn; the pixel block index list represents the sum of coordinate indices of all pixel blocks in the remaining region to be determined in the double buffer; the previous frame pixel block data represents the historical color and depth data of the corresponding pixel block stored in the first buffer; the cumulative reuse count represents the number of times the historical frame data of the corresponding pixel block has been reused since the last redraw; the complete shader pipeline processing represents the full-process shading processing that includes vertex transformation, fragment shading, texture sampling, and color blending; the simplified shader pipeline processing represents a lightweight shading processing that retains only basic coordinate transformation and simple sampling; and the guided filter fusion represents a linear blending processing method that can preserve image edge details and achieve smooth transitions.

[0044] In specific implementation, firstly, all pixel blocks in the region to be determined that are not marked as areas to be forcibly redrawn are traversed, and the coordinate index of each pixel block in the double buffer is extracted. All extracted indices are then aggregated to form a list of pixel block indices covered by the remaining region to be determined. Secondly, the texture sampling instruction of the graphics application programming interface (API) is called to read the historical color and depth data of the corresponding pixel block from the first buffer based on the pixel block index list, and the read data is used as the pixel block data of the previous frame. Next, the same API is used to read the integer count data corresponding to each pixel block from the second buffer based on the pixel block index list, and the read integer data is used as the cumulative reuse count of each pixel block. Finally, an eight-neighborhood weighted average algorithm is used, with each pixel block's own cumulative reuse count as the cumulative reuse count. The reuse count is the core weight, with the cumulative reuse count of its eight neighboring pixel blocks as an auxiliary weight. The higher the reuse count, the greater the weight. The confidence value of each pixel block is obtained through weighted calculation. The confidence values ​​of all pixel blocks are arranged according to the screen coordinate distribution, and the resulting numerical distribution field is used as the pixel reuse confidence field of each pixel block. Then, the animal model corresponding to the remaining undetermined region is divided into an edge vertex set and an internal vertex set. The edge vertex set is processed by the full shader pipeline, and the internal vertex set is processed by the simplified shader pipeline to generate the partial redraw result of the current frame. Finally, the pixel reuse confidence field of each pixel block is used to perform edge-preserving guided filtering fusion on the pixel block data of the previous frame and the partial redraw result of the current frame to generate the mixed pixel data.

[0045] In this embodiment, the animal model corresponding to the remaining region to be determined is divided into an edge vertex set and an internal vertex set. Full shader pipeline processing is performed on the edge vertex set, and simplified shader pipeline processing is performed on the internal vertex set. The generation of the partial redraw result for the current frame can be achieved through the following steps: Obtain the vertex buffer data and triangle facets of the animal model corresponding to the remaining regions to be determined; Traverse each triangular facet of the animal model and identify the triangular facets located at the edges of the animal model's outline; The vertices contained in the triangular facets at the edge of the animal model outline are labeled as the edge vertex set, and the remaining vertices of the animal model are labeled as the interior vertex set; A full vertex shader is run for each vertex in the edge vertex set, and a simplified vertex shader is run for each vertex in the interior vertex set, wherein the simplified vertex shader only performs model-view projection transformation; A full fragment shader is run on the fragments covered by the edge vertex set, and a simplified fragment shader is run on the fragments covered by the internal vertex set. The simplified fragment shader samples color values ​​from the previous frame buffer and offsets the sampled coordinates according to the motion vector field of the current frame. The rendering results of the edge vertex set and the internal vertex set are merged to generate the partial redraw result of the current frame.

[0046] It should be noted that, in this application, the vertex buffer data refers to hardware cache data storing the vertex coordinates, normal vectors, and texture coordinates of the animal model; the triangular facets refer to the basic geometric primitives constituting the surface of the animal model; the triangular facets of the animal model's outline edge refer to the geometric facets located adjacent to the background area of ​​the animal model's outer outline; the edge vertex set refers to the set of all vertices constituting the edge facets of the animal model's outline; the internal vertex set refers to the set of all vertices constituting the facets of the non-outline area of ​​the animal model; the full vertex shader refers to the full-process vertex processing program that performs model-view projection transformation, normal vector transformation, and lighting calculation; the simplified vertex shader refers to the lightweight vertex processing program that only performs model-view projection transformation; the full fragment shader refers to the full-process fragment processing program that performs texture sampling, lighting calculation, transparency blending, and color calculation; the simplified fragment shader refers to the lightweight fragment processing program that only performs historical frame texture sampling and sampling coordinate motion offset; the motion vector field refers to the set of displacement vectors of the animal model vertices in screen space; and the current frame partial redraw result refers to the local pixel data after merging the full rendering of the edge area and the simplified rendering of the internal area.

[0047] In specific implementation, firstly, the vertex cache information and geometric patch index information of the animal model corresponding to the remaining region to be determined are read from the graphics hardware's video memory space. The read information is used as the vertex buffer data and triangle patches of the animal model. Secondly, all triangle patches of the animal model are traversed, and a patch normal vector angle comparison algorithm is used to compare the difference in the normal vector angle between adjacent triangle patches with a preset contour threshold. This contour threshold is set according to the smoothness of the animal model's surface; the greater the surface undulation, the larger the threshold value. Adjacent patches whose normal vector angle difference exceeds the contour threshold are determined as contour-related patches, and these patches are used as triangle patches of the animal model's contour edge. Next, the vertex numbers associated with all triangle patches of the animal model's contour edge are extracted. The vertices corresponding to these vertex numbers are summarized as the edge vertex set, and the remaining vertices in the animal model that are not included in the edge vertex set are summarized as the inner vertex set. The process involves first creating a vertex set for the edge vertex set; then, loading and running a full vertex shader for each vertex in the edge vertex set to perform a complete coordinate transformation from model space to clip space and lighting-related calculations; and loading and running a simplified vertex shader for each vertex in the internal vertex set to perform only the model-view projection matrix transformation to complete the basic coordinate mapping. Next, a full fragment shader is run on the screen fragments covered by the rasterized edge vertex set to complete texture mapping sampling, ambient light calculation, and final color output. A simplified fragment shader is run on the screen fragments covered by the rasterized internal vertex set to sample historical pixel color values ​​from the first buffer, while simultaneously performing displacement offset processing on the texture sampling coordinates based on the current frame's motion vector field. Finally, the fragment rendering results corresponding to the edge vertex set and the fragment rendering results corresponding to the internal vertex set are superimposed and stitched at corresponding positions in screen space, and the stitched local rendering data is used as the partial redraw result for the current frame.

[0048] In this embodiment, the edge-preserving guided filtering fusion of the previous frame pixel block data and the current frame partial redrawing result is performed based on the pixel reuse confidence field of each pixel block to generate the mixed pixel data. This can be achieved through the following steps: Obtain the pixel reuse confidence field for each pixel block, downsample the pixel reuse confidence field to generate a multi-layer confidence pyramid, and simultaneously construct a multi-layer image pyramid for the pixel block data of the previous frame and the partial redrawing result of the current frame respectively. Starting from the top layer of the multi-layer confidence pyramid, traverse layer by layer. For each pixel in the current layer, determine the local window radius of the guided filter based on the confidence value of that pixel. Extract the pixel block data of the previous frame corresponding to the current layer from the multi-layer image pyramid of the previous frame pixel block data, and extract the redraw result of the current frame corresponding to the current layer from the multi-layer image pyramid of the current frame partial redraw result. Within the radius of the local window, calculate the local covariance matrix and local mean vector of the pixel block data of the previous frame corresponding to the current layer and the partial redrawing result of the current frame corresponding to the current layer. The linear transformation parameters of the guided filter are calculated based on the local covariance matrix and the local mean vector. The linear transformation parameters are then applied to the pixel block data of the previous frame corresponding to the current layer to generate the fusion result of the current layer. The fusion result of the current layer is upsampled, and the upsampled fusion result is used as the guide image for the next layer of fusion. The above process is repeated until the bottom layer of the multi-layer confidence pyramid is reached to generate the final mixed pixel data.

[0049] It should be noted that, in this application, the multi-layer confidence pyramid represents a set of confidence data at different resolution levels obtained by downsampling the pixel reuse confidence field multiple times; the multi-layer image pyramid represents a set of image data at different resolution levels obtained by downsampling the pixel block data of the previous frame and the partial redrawing result of the current frame multiple times; downsampling represents an image processing operation that reduces image resolution while retaining core features; upsampling represents an image processing operation that increases image resolution to restore detail information; the local window radius represents the window size used to calculate local statistical features during guided filtering; the local covariance matrix represents the correlation representation of pixel data within the local window; the local mean vector represents the average value representation of pixel data within the local window; the linear transformation parameter represents the coefficient used to adjust the fusion ratio of the pixel data of the previous frame and the partial redrawing result of the current frame, including the scaling factor and the offset factor; the guiding image represents the reference image used to retain edge details during guided filtering; and the mixed pixel data represents the final pixel rendering data that combines historical pixel data and current local redrawing data, balancing rendering quality and computational power consumption.

[0050] It should also be noted that the reference Figure 4As shown in the figure, this is a schematic diagram of the guided filter fusion process provided in the embodiment of this application. The figure clearly illustrates the logical connection and execution steps of the three core stages: data preprocessing, layer-by-layer traversal and local computation, and fusion and upsampling. In the data preprocessing stage, Gaussian downsampling is first performed on the pixel reuse confidence field to construct a multi-layer confidence pyramid containing three different resolution levels. Simultaneously, Gaussian downsampling is performed on the pixel block data of the previous frame and the partial redrawing result of the current frame in the first buffer, each forming a multi-layer image pyramid. In the layer-by-layer traversal and local computation stage, the process is performed from the highest level of the confidence pyramid to the lowest level. Layer iteration dynamically adjusts the radius of the local window for the guided filter based on the confidence value of each pixel. After extracting the pixel data of the corresponding layer, the local covariance matrix and local mean vector within the local window are calculated by traversing through a sliding window to solve for the linear transformation parameters. In the fusion and upsampling stage, the linear transformation parameters are applied to the pixel block data of the previous frame to generate the fusion result of the current layer. After upsampling to the next layer resolution by bilinear interpolation, it serves as the guide image for the next layer of fusion. The local calculation and fusion process is repeated until the bottom layer of the confidence pyramid is reached, and finally, mixed pixel data that balances rendering quality and computational efficiency is output.

[0051] In specific implementation, firstly, the pixel reuse confidence field of each pixel block is read, and a Gaussian downsampling algorithm is used to perform multiple downsampling processes on the pixel reuse confidence field. The number of downsampling times is set to 3 layers based on the pixel block size and the performance of low-computing-power devices (too many layers will lose confidence details, too few layers cannot achieve smooth layer fusion), generating a multi-layer confidence pyramid containing different resolution levels. At the same time, the same Gaussian downsampling algorithm is used to perform 3-layer downsampling processes on the pixel block data of the previous frame read from the first buffer and the partial redrawing results of the current frame, respectively, to construct multi-layer image pyramids. Secondly, starting from the highest layer of the multi-layer confidence pyramid, it traverses downwards layer by layer. For each pixel point in the current layer, its confidence value can be set... The local window radius for guided filtering is set as follows: 3×3 when the confidence value is ≥0.7, 5×5 when the confidence value is 0.3 < 0.7, and 7×7 when the confidence value is ≤0.3. This window size, determined based on the confidence value, is used as the local window radius for the guided filtering of the current pixel. Next, based on the current pyramid level, pixel data of the corresponding level is extracted from the constructed multi-layer image pyramid of the previous frame pixel block data and the multi-layer image pyramid of the current frame partial redraw result, respectively, and used as the previous frame pixel block data and the current frame partial redraw result for the current level. Subsequently, within the local window radius corresponding to each pixel, a sliding window traversal algorithm is used to calculate the local window radius window by window. The pixel value distribution characteristics of the previous frame's pixel block data and the partial redraw result of the current frame are calculated to obtain the local covariance matrix and local mean vector within each window. Then, based on the local covariance matrix and local mean vector of each window, the linear transformation parameters of the guided filter are calculated using the least squares method. Specifically, the pixel reuse confidence field of the current layer is used as the guiding image to construct an error function, which is used to characterize the deviation between the linearly transformed data and the partial redraw result of the current frame. The error function is then minimized using the least squares method to obtain the scaling factor and offset coefficient of the linear transformation. The local covariance matrix is ​​used to calculate the scaling factor, and the local mean vector is used to calculate the offset coefficient. The obtained scaling factor and offset coefficient are then calculated. The offset coefficients are used as linear transformation parameters for the guided filter. These parameters are then applied pixel-by-pixel to the pixel block data of the previous frame corresponding to the current layer. The fusion value of each pixel in the current layer is obtained through weighted calculation. The fusion values ​​of all pixels are then summed as the fusion result of the current layer. Finally, the fusion result of the current layer is upsampled using a bilinear interpolation algorithm. The upsampled fusion data with the same resolution as the next layer is used as the guide image for the next layer's fusion. The process of extracting data layer by layer, calculating local features, solving for linear coefficients, generating fusion results, and upsampling is repeated until the bottom layer of the multi-layer confidence pyramid is reached. The fusion data generated at the bottom layer is used as the final mixed pixel data.

[0052] In step S5, after rendering is completed, the pixel data in the first buffer is updated, the cumulative reuse count of the redrawn pixel blocks is reset to zero, and the cumulative reuse count of the unredrawn pixel blocks is increased by one.

[0053] In this embodiment, after rendering is complete, the pixel data in the first buffer is updated, and the cumulative reuse count of the redrawn pixel blocks is reset to zero. The cumulative reuse count of the non-redrawn pixel blocks is increased by one, which can be achieved by the following steps: Submit the final pixel data of the forced redraw region and linear blend region generated in this rendering to the graphics application programming interface, and update the corresponding pixel blocks in the first buffer. For all pixels marked as forced redraw areas and participating in linear blending in this rendering, reset the cumulative reuse count in the second buffer to the initial value of zero; For all pixel blocks marked as directly reusable regions in this rendering, increase the cumulative reuse count in the second buffer by one from the original value. After the second buffer is updated, the pointers of the first and second buffers are swapped, so that the first buffer points to the rendering result of the current frame and the second buffer points to the cumulative reuse count of the current frame.

[0054] It should be noted that, in this application, the graphics application programming interface refers to an interface used to interact with graphics hardware and execute rendering instructions; the initial value of zero represents the reset base value for the number of times a pixel block is reused after redrawing; the pointer swapping represents switching the read and write permissions of the double buffer to ensure that the rendering result and reuse count of the previous frame can be correctly read when the next frame is rendered; the redrawn pixel block refers to the forced redraw area and the pixel block participating in linear blending, all of which have undergone redrawing operations to varying degrees; the unredrawn pixel block refers to the pixel block of the directly reused area, which has not undergone redrawing operations and directly reuses historical frame data.

[0055] In practice, firstly, the complete pixel data of the forced redraw region and the pixel data after blending the linear blend region generated in this rendering process are aggregated and integrated into the final pixel data. The texture batch write instruction of the graphics application programming interface (API) is called, and the final pixel data is written pixel by pixel to the corresponding coordinate position of the first buffer according to the screen coordinates of the forced redraw region and the linear blend region, overwriting the original pixel data of the previous frame in the first buffer. The updated first buffer is used as the carrier to store the rendering result of the current frame. Secondly, all pixel blocks marked as forced redraw regions and linear blend regions in the region mask are traversed, and the coordinate indices of these pixel blocks in the second buffer are extracted. The cumulative reuse count value of the corresponding index position in the second buffer is then modified using the integer data modification instruction of the API. All values ​​are initialized to zero, and this modified value is used as the cumulative reuse count for this type of redrawn pixel block. Next, all pixel blocks marked as directly reused areas in the region mask are traversed, and the coordinate indices of these pixel blocks in the second buffer are extracted. Using the integer increment instruction of the graphics application programming interface, the cumulative reuse count value at the corresponding index position in the second buffer is incremented by one each time, and the incremented value is used as the cumulative reuse count for this type of non-redrawn pixel block. Finally, after the cumulative reuse count of all pixel blocks in the second buffer is updated, the buffer pointer switching instruction of the graphics application programming interface is called to swap the read and write pointers of the first and second buffers, ensuring that the rendering result and reuse count data can be correctly read in the next frame, thus completing the cyclic update of the dual buffers.

[0056] Therefore, in this application, after rendering is completed, the pixel data in the first buffer is updated, and the cumulative reuse count of the redrawn pixel blocks is reset to zero, while the cumulative reuse count of the unredrawn pixel blocks is increased by one. Firstly, by creating a dual buffer for the off-screen rendering pipeline to store the rendering result of the previous frame and the cumulative reuse count of pixel blocks respectively, and configuring the buffer by dividing the pixel block array according to model complexity, fine-grained recording and management of frame buffer reuse status can be achieved, while avoiding image anomalies caused by indiscriminate reuse of frame buffers. Secondly, a motion saliency mask is generated based on the difference between animal skeletal animation and camera viewpoint parameters, thereby dividing the static background directly reuse area and the moving animal undetermined area. This can retain the advantages of frame buffer reuse in static areas and reduce redundant rendering computational overhead while accurately locating moving areas prone to ghosting, achieving a preliminary balance between computational saving and targeted rendering. Furthermore, by reading the cumulative reuse count of pixel blocks and combining it with a dynamic threshold to determine the forced redraw area, It can promptly trigger redrawing of long-reused pixel blocks, avoiding ghosting and motion blur issues caused by continuous reuse of frame data. Moreover, it only redraws local areas, without significantly increasing the device's computing power burden. Furthermore, it simplifies the rendering process by shrinking the view frustum for the forced redrawing area, and uses model vertex layering and inter-frame pixel linear blending for the remaining undetermined areas. This ensures the rendering accuracy and image clarity of the moving area edges, and further reduces graphics resource consumption by simplifying shader calculations, effectively avoiding the frame rate drop problem caused by conventional ghosting elimination methods. Finally, after rendering is completed, the reuse count of the redrawing area is reset, the reuse count of the reused area is accumulated, and the buffer pointers are swapped, forming a closed-loop update mechanism for the inter-frame reuse state. This allows the frame buffer reuse strategy to adapt to dynamic scene changes, continuously achieving efficient reuse of off-screen rendering frame buffers on low-computing-power devices. While steadily reducing computing power consumption, it completely eliminates image ghosting and motion blur, ensuring a smooth visual experience in the holographic zoo scene.

[0057] In summary, the technical solution adopted in this application can achieve efficient reuse of off-screen rendering frame buffers, thereby reducing computing power consumption while eliminating image ghosting.

[0058] Example 2: This application provides a holographic zoo off-screen rendering optimization device for low-computing-power devices. This holographic zoo off-screen rendering optimization device includes an off-screen rendering optimization unit, as shown in the reference... Figure 5 As shown in the figure, this is a schematic diagram of exemplary hardware and / or software of an off-screen rendering optimization unit according to some embodiments of this application. The off-screen rendering optimization unit includes: a double buffer creation module 100, a motion saliency detection module 200, a forced redraw region marking module 300, a partition rendering module 400, and a rendering data update module 500, which are described below: The double buffer creation module 100 is used to create a double buffer for the off-screen rendering pipeline of the holographic zoo, wherein the first buffer stores the complete rendering result of the previous frame, and the second buffer stores the cumulative reuse count of each pixel block in the previous frame. The motion saliency detection module 200 is used to perform motion saliency detection on the area to be rendered when rendering the current frame. The exhibition hall background area with a detection result of being static is marked as a directly reused area, and the animal area with a detection result of being moving is marked as an area to be determined. The forced redrawing region marking module 300 is used to read the cumulative reuse count of each pixel block in the second buffer for each pixel block in the region to be judged, and mark the pixel block as a forced redrawing region when the cumulative reuse count exceeds the dynamic threshold. The partition rendering module 400 is used to perform a complete off-screen redraw for the forced redraw area, copy pixel data from the first buffer for the directly reused area, and linearly blend the pixel data of the previous frame with the partial redraw result of the current frame for the remaining areas to be determined. The rendering data update module 500 is used to update the pixel data in the first buffer after rendering is completed, reset the cumulative reuse count of the redrawn pixel blocks to zero, and increase the cumulative reuse count of the unredrawn pixel blocks by one.

[0059] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0060] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0061] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. A method for optimizing off-screen rendering of a holographic zoo on low-computing-power devices, characterized in that, The off-screen rendering optimization method for the holographic zoo includes the following steps: Create a double buffer for the off-screen rendering pipeline of the holographic zoo, where the first buffer stores the complete rendering result of the previous frame and the second buffer stores the cumulative reuse count of each pixel block in the previous frame; When rendering the current frame, motion saliency detection is performed on the area to be rendered. The exhibition background area that is static is marked as a directly reused area, and the animal area that is moving is marked as an area to be determined. For each pixel block in the region to be determined, the cumulative reuse count of that pixel block in the second buffer is read. When the cumulative reuse count exceeds the dynamic threshold, the pixel block is marked as a region to be forced to redraw. Perform a full off-screen redraw for the forced redraw area, copy pixel data from the first buffer for the directly reused area, and linearly blend the pixel data from the previous frame with the partial redraw result of the current frame for the remaining areas to be determined. After rendering is complete, update the pixel data in the first buffer, reset the cumulative reuse count of the redrawn pixel blocks to zero, and increment the cumulative reuse count of the unredrawn pixel blocks by one.

2. The off-screen rendering optimization method for a holographic zoo on a low-computing-power device as described in claim 1, characterized in that, A double buffer is created for the off-screen rendering pipeline of the holographic zoo. The first buffer stores the complete rendering result of the previous frame, and the second buffer stores the cumulative reuse count of each pixel block in the previous frame. Specifically, this includes: The size of the pixel blocks is determined based on the complexity of the exhibit and animal models in the holographic zoo. The rendering target is divided into a grid-like array of pixels based on the size of the pixel blocks; A first off-screen rendering texture is allocated as a first buffer for the pixel block array. The first off-screen rendering texture is used to store the complete color and depth data of each pixel block in the previous frame. A second off-screen rendering texture is allocated as a second buffer for the pixel block array. The second off-screen rendering texture is used to store the cumulative number of times each pixel block was reused in the previous frame in the form of integer data.

3. The off-screen rendering optimization method for a holographic zoo on low-computing-power devices as described in claim 1, characterized in that, When rendering the current frame, motion saliency detection is performed on the area to be rendered. Areas where the detection result indicates a stationary exhibition hall background are marked as directly reusable areas, while areas where the detection result indicates moving animals are marked as areas to be determined. Specifically, this includes: Obtain the animal skeleton animation data and camera view transformation parameters for the current frame; The animal skeleton animation data and camera view transformation parameters of the previous frame are compared with the corresponding data of the current frame to generate the motion vector field of the animal model in screen space. The motion vector field is subjected to region growing and morphological filtering to generate a motion saliency mask; Based on the motion saliency mask, the areas to be rendered with mask values ​​below the first threshold are marked as static exhibition hall background areas, i.e., directly reused areas; The areas to be rendered with a mask value higher than the second threshold are marked as moving animal areas, i.e., areas to be determined.

4. The off-screen rendering optimization method for a holographic zoo on a low-computing-power device as described in claim 1, characterized in that, For each pixel block in the region to be determined, the cumulative reuse count of that pixel block in the second buffer is read. When the cumulative reuse count exceeds the dynamic threshold, the pixel block is marked as a region to be forced to redraw. Specifically, this includes: Traverse all pixel blocks covered by the region to be determined, and obtain the index address of each pixel block in the second buffer; Read the cumulative reuse count value of each pixel block according to the index address; The accumulated reuse count value is compared with a preset dynamic threshold, which is adjusted according to the rendering load of the current frame and the power consumption and temperature status of the device. If the cumulative reuse count is greater than the dynamic threshold, the pixel block is marked as a forced redraw region in the region mask.

5. The off-screen rendering optimization method for a holographic zoo on a low-computing-power device as described in claim 1, characterized in that, Performing a full off-screen redraw for the forced redraw area specifically includes: Get the set of pixel coordinates covered by the mask of the forced redraw region; Construct the bounding box of the forced redraw region in screen space based on the set of pixel block coordinates; Calculate the clipping plane offset of the view frustum based on the bounding box of the forced redraw region in screen space; Based on the offset of the clipping plane of the view frustum, the clipping plane of the original view frustum is asymmetrically shrunk to generate a redrawn view frustum; Rendering nodes that are completely outside the redrawn view frustum are marked as culled nodes, and rendering nodes that are partially or completely inside the redrawn view frustum are marked as nodes to be rendered. Perform a complete vertex transformation and rasterization process for all nodes to be rendered, generate pixel data for the redraw area, and write it to the first buffer.

6. The off-screen rendering optimization method for a holographic zoo on a low-computing-power device as described in claim 1, characterized in that, The remaining areas to be determined are linearly blended using pixel data from the previous frame and partial redrawing results from the current frame. Specifically, this includes: Obtain the list of pixel indexes covered by the remaining area to be determined; Read the pixel block data of the previous frame corresponding to the pixel block index list from the first buffer; Read the cumulative reuse count of each pixel block from the pixel block index list in the second buffer; A pixel reuse confidence field is constructed for each pixel block based on the cumulative reuse count of each pixel block and the cumulative reuse count of its neighboring pixel blocks. The pixel reuse confidence field is used to characterize the reliability of the reused data of each pixel point. The animal model corresponding to the remaining region to be determined is divided into an edge vertex set and an internal vertex set. The edge vertex set is processed by the full shader pipeline, and the internal vertex set is processed by the simplified shader pipeline to generate the partial redraw result of the current frame. Based on the pixel reuse confidence field of each pixel block, edge-preserving guided filtering fusion is performed on the pixel block data of the previous frame and the partial redrawing result of the current frame to generate mixed pixel data.

7. The off-screen rendering optimization method for a holographic zoo on a low-computing-power device as described in claim 1, characterized in that, After rendering is complete, update the pixel data in the first buffer, reset the cumulative reuse count of redrawn pixel blocks to zero, and increment the cumulative reuse count of non-redrawn pixel blocks by one. Specifically, this includes: Submit the final pixel data of the forced redraw region and linear blend region generated in this rendering to the graphics application programming interface, and update the corresponding pixel blocks in the first buffer. For all pixels marked as forced redraw areas and participating in linear blending in this rendering, reset the cumulative reuse count in the second buffer to the initial value of zero; For all pixel blocks marked as directly reusable regions in this rendering, increase the cumulative reuse count in the second buffer by one from the original value. After the second buffer is updated, the pointers of the first and second buffers are swapped, so that the first buffer points to the rendering result of the current frame and the second buffer points to the cumulative reuse count of the current frame.

8. The off-screen rendering optimization method for a holographic zoo on a low-computing-power device as described in claim 6, characterized in that, The remaining undetermined regions corresponding to the animal model are divided into an edge vertex set and an internal vertex set. A full shader pipeline is performed on the edge vertex set, and a simplified shader pipeline is performed on the internal vertex set. The resulting partial redraw of the current frame includes: Obtain the vertex buffer data and triangle facets of the animal model corresponding to the remaining regions to be determined; Traverse each triangular facet of the animal model and identify the triangular facets located at the edges of the animal model's outline; The vertices contained in the triangular facets at the edge of the animal model outline are labeled as the edge vertex set, and the remaining vertices of the animal model are labeled as the interior vertex set; A full vertex shader is run for each vertex in the edge vertex set, and a simplified vertex shader is run for each vertex in the interior vertex set, wherein the simplified vertex shader only performs model-view projection transformation; A full fragment shader is run on the fragments covered by the edge vertex set, and a simplified fragment shader is run on the fragments covered by the internal vertex set. The simplified fragment shader samples color values ​​from the previous frame buffer and offsets the sampled coordinates according to the motion vector field of the current frame. The rendering results of the edge vertex set and the internal vertex set are merged to generate the partial redraw result of the current frame.

9. The off-screen rendering optimization method for a holographic zoo on a low-computing-power device as described in claim 6, characterized in that, Based on the pixel reuse confidence field of each pixel block, edge-preserving guided filtering fusion is performed on the pixel block data of the previous frame and the partial redrawing result of the current frame to generate the mixed pixel data, specifically including: Obtain the pixel reuse confidence field for each pixel block, downsample the pixel reuse confidence field to generate a multi-layer confidence pyramid, and simultaneously construct a multi-layer image pyramid for the pixel block data of the previous frame and the partial redrawing result of the current frame respectively. Starting from the top layer of the multi-layer confidence pyramid, traverse layer by layer. For each pixel in the current layer, determine the local window radius of the guided filter based on the confidence value of that pixel. Extract the pixel block data of the previous frame corresponding to the current layer from the multi-layer image pyramid of the previous frame pixel block data, and extract the redraw result of the current frame corresponding to the current layer from the multi-layer image pyramid of the current frame partial redraw result. Within the radius of the local window, calculate the local covariance matrix and local mean vector of the pixel block data of the previous frame corresponding to the current layer and the partial redrawing result of the current frame corresponding to the current layer. The linear transformation parameters of the guided filter are calculated based on the local covariance matrix and the local mean vector. The linear transformation parameters are then applied to the pixel block data of the previous frame corresponding to the current layer to generate the fusion result of the current layer. The fusion result of the current layer is upsampled, and the upsampled fusion result is used as the guide image for the next layer of fusion. The above process is repeated until the bottom layer of the multi-layer confidence pyramid is reached to generate the final mixed pixel data.

10. A holographic zoo off-screen rendering optimization device for low-computing-power devices, controlled by the method described in any one of claims 1 to 9, the holographic zoo off-screen rendering optimization device comprising an off-screen rendering optimization unit, characterized in that, The off-screen rendering optimization unit includes: The dual-buffer creation module is used to create dual buffers for the off-screen rendering pipeline of the holographic zoo. The first buffer stores the complete rendering result of the previous frame, and the second buffer stores the cumulative reuse count of each pixel block in the previous frame. The motion saliency detection module is used to perform motion saliency detection on the area to be rendered when rendering the current frame. The exhibition hall background area that is detected as stationary is marked as a directly reusable area, and the animal area that is detected as moving is marked as an area to be judged. The forced redraw region marking module is used to read the cumulative reuse count of each pixel block in the second buffer for each pixel block in the region to be judged. When the cumulative reuse count exceeds the dynamic threshold, the pixel block is marked as a forced redraw region. The partition rendering module is used to perform a complete off-screen redraw for the forced redraw area, copy pixel data from the first buffer for the directly reused area, and linearly blend the pixel data of the previous frame with the partial redraw result of the current frame for the remaining areas to be determined. The rendering data update module is used to update the pixel data in the first buffer after rendering is completed, reset the cumulative reuse count of the redrawn pixel blocks to zero, and increase the cumulative reuse count of the non-redrawn pixel blocks by one.

Images