Ultra-large texture zero-copy streaming method and system based on asynchronous DMA and PBO

Abstract

The application provides a large texture zero-copy streaming method and system based on asynchronous DMA and PBO, relates to the technical field of computer graphics rendering, and is used for solving the problems of large CPU copy overhead and frequent display memory switching when large textures are loaded by patches. The application screens visible texture patches in a hierarchical bounding box according to a viewpoint parameter and a frustum parameter; determines a target resolution according to the spatial distance between a viewpoint corresponding to the viewpoint parameter and each texture patch; and encapsulates the texture patches into a continuous data stream, constructs a continuous display memory block through PBO, and writes the continuous display memory block asynchronously through DMA, so that invalid texture loading and intermediate copying can be reduced, and the continuity of large texture transmission and rendering can be improved.

Description

Technical Field

[0001] This application relates to the field of computer graphics rendering technology, and in particular to a method and system for zero-copy streaming of ultra-large textures based on asynchronous DMA and PBO. Background Technology

[0002] The ultra-large texture zero-copy streaming method is mainly used in graphics rendering scenarios that require loading a large number of high-definition textures, such as 3D scenes, video twins, and digital cities. The application prospect of this method is to reduce the repeated transfer of texture data between memory and video memory, so that large-scale 3D models can be displayed and updated more smoothly.

[0003] The prior art discloses a 3D display refresh scheme based on OpenGL ES, which saves the rendering result of the previous frame through PBO, generates updated textures for the changed display areas and then performs blending display. It also shows that PBO can use DMA for asynchronous pixel data transfer, reducing CPU involvement and thus improving the 3D display refresh efficiency.

[0004] However, these methods are mainly designed for partial screen refreshes and reuse of existing rendering results. When faced with the continuous loading of extremely large textures, it is still difficult to ensure that the data to be transmitted fully matches the actual display requirements of the current screen. If a large amount of texture data is transmitted uniformly or repeatedly waits for video memory updates, it can easily lead to excessive transmission resources and discontinuous rendering process. Therefore, existing technologies suffer from the technical problem that the transmission resources are difficult to match with the actual rendering requirements when streaming extremely large textures, resulting in insufficient rendering continuity. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for zero-copy streaming of ultra-large textures based on asynchronous DMA and PBO, so as to solve the problem in the prior art that the transmission resources are difficult to match with the actual rendering requirements when streaming ultra-large textures, resulting in insufficient rendering continuity.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a method for zero-copy streaming of ultra-large textures based on asynchronous DMA and PBO, the method comprising: Based on the viewpoint and frustum parameters of the current virtual camera in the video twin engine, the target 3D model within the visible range is detected in the pre-established hierarchical bounding box, and the identification information of all texture blocks associated with the target 3D model is determined to obtain the first block set. Based on the spatial geometric angle between the surface normal vector corresponding to each texture block on the surface of the target 3D model and the viewing direction of the current virtual camera, the identification information of the texture blocks located on the back of the target 3D model in the first block set is removed to obtain the second block set; Based on the spatial distance between the viewpoint corresponding to the viewpoint parameter and each texture block, the target resolution to be loaded for each texture block in the second block set is determined, and the target resolution of the texture blocks whose spatial geometric angle is within a preset angle range is downgraded. In the main memory, all texture slices in the second slice set are encapsulated into a continuous data stream. At the same time, in the graphics processing unit memory, the target resolution is combined, and a continuous memory block for carrying the continuous data stream is constructed using a pixel buffer object (PBO). An asynchronous transmission channel is established, through which the continuous data stream is directly transferred from the main memory to the continuous video memory block with zero copy.

[0007] Optionally, the step of combining the target resolution in the graphics processing unit's video memory and constructing a contiguous video memory block to carry the continuous data stream using a pixel buffer object (PBO) includes: The target resolution is compared with a preset resolution threshold. Large-capacity page blocks are allocated to texture slices whose target resolution is higher than the resolution threshold, and standard page blocks are allocated to the remaining texture slices. Based on the arrangement order of the continuous data stream and the page block specifications corresponding to each texture slice, storage units with connected physical addresses are allocated sequentially in the graphics processing unit's video memory, and then spliced ​​together to construct a continuous video memory block with a global starting address.

[0008] Optionally, the step of allocating physically connected storage units sequentially in the graphics processing unit's video memory according to the arrangement order of the continuous data stream and the page block specifications corresponding to each texture slice, and then splicing them together to construct a continuous video memory block with a global starting address, includes: According to the arrangement order of the continuous data stream, the byte size of all preceding segments is accumulated to obtain the initial address of the current texture segment; Based on the corresponding page block specification, the spatial distance between the initial address and the hardware alignment boundary is determined to obtain the address gap. The address gap is then filled with boundary padding blocks to complete the boundary alignment, resulting in an aligned memory segment containing the boundary padding blocks. Calculate the sum of the initial address and the size of the boundary fill block to obtain the address offset of the current texture slice; The global starting address is used as the addressing zero point, and the address is added to the corresponding address offset to calculate the target addressing position of the current texture block; Using the target addressing location as the storage starting point, and according to the byte length of the corresponding aligned storage segment, a physical storage area of ​​the corresponding size is specified and divided in the graphics processing unit's video memory as the allocated storage unit. The corresponding video memory pointer is redirected to the target address location to guide the continuous data stream to be written uninterruptedly into each of the allocated memory units, so that each memory unit is connected end to end in physical space to obtain a continuous video memory block.

[0009] Secondly, this application provides a system for zero-copy streaming of large textures based on asynchronous DMA and PBO, the system comprising: The detection module is used to detect the target 3D model within the visible range in a pre-established hierarchical bounding box based on the viewpoint parameters and frustum parameters of the current virtual camera in the video twin engine, and to determine the identification information of all texture blocks associated with the target 3D model to obtain the first block set. The culling module is used to remove the identification information of texture blocks located on the back of the target 3D model from the first block set based on the spatial geometric angle between the surface normal vector corresponding to each texture block on the surface of the target 3D model and the viewing direction of the current virtual camera, so as to obtain the second block set. The determination module is used to determine the target resolution to be loaded for each texture block in the second block set based on the spatial distance between the viewpoint corresponding to the viewpoint parameter and each texture block, and to perform downgrade processing on the target resolution of texture blocks whose spatial geometric angle is within a preset angle range. The fusion module is used to encapsulate all texture slices in the second slice set into a continuous data stream in the main memory, and combine the target resolution in the graphics processing unit video memory, and use pixel buffer objects (PBOs) to construct a continuous video memory block to carry the continuous data stream. A module is established to create an asynchronous transmission channel, through which the continuous data stream is directly transferred from the main memory to the continuous video memory block with zero copy.

[0010] The ultra-large texture zero-copy streaming method based on asynchronous DMA and PBO provided in this application has the following beneficial effects: This application can first exclude model and texture data outside the current screen, reducing the number of objects to be processed subsequently; by removing the identification information of texture blocks located on the back of the target 3D model based on the spatial geometric angle between the surface normal vector and the viewing direction, it can further remove texture blocks that are not visible from the current viewpoint, avoiding invalid loading; by determining the target resolution based on the spatial distance between the viewpoint and each texture block, and performing downgrading processing on texture blocks within a specific angle range, it can make the texture clarity match the actual viewing requirements, reducing unnecessary high-resolution data transmission; by encapsulating continuous data streams in main memory and using PBO to construct continuous video memory blocks in the graphics processing unit's video memory, it can keep the data to be transmitted and the video memory carrying space in order, reducing the management overhead caused by scattered storage; by establishing an asynchronous transmission channel and directly transmitting the continuous data stream to the continuous video memory blocks with zero copies, it can reduce the time that the central processing unit participates in data transportation, and improve the continuity of the texture loading and rendering process.

[0011] Furthermore, this application enables texture slices with different resolution requirements to obtain matching video memory space, avoiding insufficient storage of high-resolution textures or excessive space occupied by low-resolution textures, and enabling data after PBO transfer to be continuously written and read at a unified address, thereby improving the stability of video memory utilization and texture access. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a zero-copy streaming method for ultra-large textures based on asynchronous DMA and PBO disclosed in this application. Figure 2 This is a schematic diagram of the visual slicing detection and backside removal disclosed in this application; Figure 3 This is a schematic diagram of the PBO contiguous video memory block and page block allocation disclosed in this application; Figure 4 This is a schematic diagram of the structure of a zero-copy streaming system for ultra-large textures based on asynchronous DMA and PBO disclosed in this application. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] Figure 1 This is a flowchart illustrating a zero-copy streaming method for ultra-large textures based on asynchronous DMA and PBO disclosed in this application. The method includes: S101. Based on the viewpoint parameters and frustum parameters of the current virtual camera in the video twin engine, detect the target 3D model within the visible range in the pre-established hierarchical bounding box, and determine the identification information of all texture blocks associated with the target 3D model to obtain the first block set.

[0016] The viewpoint parameters include the virtual camera's position, orientation, and viewing direction in the 3D scene, indicating the user's current viewing position and direction of the video twin scene. The view frustum parameters include clipping parameters and field of view parameters. The clipping parameters limit the effective viewing depth of the virtual camera in the viewing direction, including at least the near clipping distance and the far clipping distance. The near clipping distance determines the effective viewing boundary closest to the current virtual camera, and the far clipping distance determines the effective viewing boundary farthest from the current virtual camera. When constructing the view frustum space, the clipping parameters can also be converted into near clipping planes and far clipping planes, which, together with the field of view parameters, form multiple clipping planes in the view frustum space. The field of view parameters limit the horizontal or vertical viewing range of the virtual camera. The hierarchical bounding box consists of multiple bounding boxes organized according to spatial hierarchy. Each bounding box is used to wrap a 3D model, model component, or lower-level spatial region. Its function is to make a quick judgment with a simple spatial outline first, avoiding direct inspection of the complex mesh of each model one by one. The first piece set includes the identification information of the texture pieces associated with the target 3D model, which is used to provide objects to be processed for subsequent texture filtering, resolution determination, and data transmission.

[0017] like Figure 2 As shown, the current observation range of the virtual camera 21 forms a visible range 22, which is jointly defined by viewpoint parameters, clipping parameters, and field of view parameters. The hierarchical bounding box 23 is used to enclose the 3D model in the virtual space. The visible range 22 and the hierarchical bounding box 23 are intersected to determine the target 3D model 24. Figure 2The visible slice 25 and the back slice 26 are used to show the state of the texture slice associated with the target 3D model 24 before and after the subsequent orientation determination. The visible slice 25 includes the texture slice associated with the target 3D model 24 that is located within the current virtual camera 21’s field of view and has not been determined to be the back of the target 3D model.

[0018] In this embodiment of the application, S101 specifically includes: S1011. Using the viewpoint parameters of the current virtual camera as the vertex of the view frustum, and combining the clipping parameters and field of view parameters in the view frustum parameters, a view frustum space enclosed by multiple clipping planes is constructed in the virtual space.

[0019] Here, the vertex of the view frustum refers to the starting point of the current virtual camera's observation in the 3D scene; the clipping plane is used to define the geometric boundary of the view frustum space, which may include the near clipping plane determined by the near clipping distance, the far clipping plane determined by the far clipping distance, and the left, right, upper, and lower boundary planes determined according to the field of view angle parameters and the current virtual camera's observation direction; the view frustum space is used to represent the 3D spatial range that the current virtual camera can theoretically observe in this frame.

[0020] In this embodiment, the viewpoint parameters of the current virtual camera are first read to determine the position and viewing direction of the virtual camera in the video twin scene. Then, the clipping parameters and field of view parameters in the view frustum parameters are read. Based on the near and far clipping distances in the clipping parameters, the front and rear boundaries in the viewing direction are determined, and the lateral opening boundaries of the viewing range are determined based on the field of view parameters. Finally, the front and rear boundaries and lateral boundaries are converted into a near clipping plane, a far clipping plane, and four surrounding boundary planes, which together form the view frustum space. Through this process, the originally abstract camera viewing range is transformed into a geometric range that can participate in spatial judgment.

[0021] S1012. Define the view frustum as the visible range, extract the spatial boundary of the visible range, and perform intersection detection with the spatial boundary and the pre-established hierarchical bounding box.

[0022] The visible range refers to the spatial area that the current virtual camera may see in the current frame; the spatial boundary includes the various clipping planes of the view frustum and the boundary directions defined by these clipping planes; the intersection detection is used to determine whether the hierarchical bounding box overlaps with the visible range.

[0023] In this embodiment, after obtaining the view frustum, the view frustum is used as the visible range of the current frame. Then, the spatial boundary of this visible range is extracted, and pre-established hierarchical bounding boxes are accessed layer by layer. For each hierarchical bounding box, the intersection detection technique between the bounding box and the view frustum is used for judgment. Specifically, the position range of the bounding box can be compared with each clipping plane. If the bounding box is entirely outside a clipping plane, it means that the bounding box is not within the current visible range and can be directly discarded. If the bounding box is not excluded by all clipping planes, it means that it may overlap with the visible range and needs to be retained or enter the next level of detection. This coarse-to-fine detection method can reduce the traversal of irrelevant 3D models.

[0024] S1013. Select the hierarchical bounding boxes that spatially overlap with the visible range as target bounding boxes, and determine the three-dimensional model enclosed inside the target bounding box as the target three-dimensional model.

[0025] Among them, the target bounding box refers to the hierarchical bounding box that has a spatial overlap with the visible range; the target 3D model refers to the 3D model that is wrapped inside the target bounding box and may be displayed or have its texture sampled in the current virtual camera view.

[0026] In this embodiment, after intersection detection is completed, the hierarchical bounding boxes not excluded by the clipping plane are selected as target bounding boxes. If the target bounding box is a parent bounding box that does not directly correspond to a specific 3D model, the next-level bounding box is detected until the bounding box that specifically encloses the 3D model is located. If the target bounding box already corresponds to a specific 3D model, the 3D model enclosed within it is directly determined as the target 3D model. Then, the association between the target 3D model and the texture chunks is read, the identification information of the corresponding texture chunks is extracted, and these identification information are summarized to form a first chunk set. Here, the first chunk set does not mean that all chunks need to be loaded in the end, but rather represents the initial range of texture chunks that the target 3D model within the current visible range may need to use.

[0027] As an example, in a video twin park scene, the current virtual camera is facing the area where a building is located. First, the viewpoint parameters of the current virtual camera are read, and combined with the clipping parameters and field of view parameters, a view frustum space is constructed, enabling the system to clearly define the 3D area that the current image may cover. Second, this view frustum space is used as the visible range, and its spatial boundary is intersected with the pre-established hierarchical bounding boxes in the park scene. Bounding boxes corresponding to roads, distant equipment, or other buildings located outside the view frustum space are quickly excluded. Next, hierarchical bounding boxes that spatially overlap with the visible range are filtered as target bounding boxes, and the building model enclosed within the target bounding box is determined as the target 3D model. Furthermore, the texture segmentation information corresponding to the wall, roof, and window areas on the surface of the building model is read, and the information is summarized to obtain the first segmentation set. The above example is only one example of this application. In actual applications, it can be set according to the scene scale, model organization method, and rendering requirements. This application does not limit this.

[0028] This application, through the collaborative detection of viewpoint parameters, view frustum parameters, and hierarchical bounding boxes, can narrow down the scope of the target 3D model and texture chunks before texture loading, reducing irrelevant texture chunks from entering the subsequent processing flow, thereby providing a data foundation with a clear object scope for subsequent backface culling, resolution determination, and asynchronous transmission.

[0029] S102. Based on the spatial geometric angle between the surface normal vector corresponding to each texture block on the surface of the target 3D model and the viewing direction of the current virtual camera, remove the identification information of the texture blocks located on the back of the target 3D model in the first block set to obtain the second block set.

[0030] Among them, the surface normal vector is used to represent the outward orientation of the surface of the model where the texture piece is located. It is determined by the mesh geometry information formed by the target 3D model during the modeling or preprocessing stage. Specifically, the surface normal vector can be determined by the pre-stored mesh normal data, the vertex winding order of the mesh facets, or the average result of the normals of adjacent facets, and then normalized to form a direction vector of unit length; the viewing direction is used to represent the viewing direction of the current virtual camera pointing to the location of the texture piece; the spatial geometric angle is used to determine whether the surface of the texture piece is facing the current virtual camera or facing away from the current virtual camera.

[0031] like Figure 2 As shown, surface normal 27 represents the outer orientation of the model surface where the texture slice is located, and viewing direction 28 represents the direction in which the current virtual camera 21 points to the location of the texture slice. By comparing the directional relationship between surface normal 27 and viewing direction 28, the back face slice 26 can be determined as an object to be culled, and the visible slice 25 can be regarded as an object to be retained and loaded.

[0032] In this embodiment of the application, S102 specifically includes: S1021. Obtain the mesh center coordinates of each texture block in the first block set on the surface of the target 3D model, and extract the unit normal vector facing outward at the mesh center coordinates as the surface normal vector.

[0033] Here, the mesh center coordinates represent the center position of the texture chunk in the corresponding area on the surface of the target 3D model; the unit normal vector is a direction vector whose length has been normalized. The unit normal vector pointing outward refers to the direction of the normal pointing towards the external space of the target 3D model, which is first determined based on the pre-stored mesh normal data or vertex winding order of the mesh region where the mesh center coordinates are located, and then normalized to obtain a unit length direction vector. In other words, "pointing outward" is predetermined by the mesh geometry of the target 3D model, and is not determined by the surface normal vector itself, thus avoiding a circular definition.

[0034] In this embodiment, firstly, based on the identification information of the texture block in the first block set, the model mesh region bound to the texture block is queried; secondly, the vertex coordinates, pre-stored mesh normal data, or vertex wrap order of the mesh region are read, and the outer normal direction of the surface where the texture block is located is determined by these mesh geometric information; then, the center position of the mesh region corresponding to the texture block is used as the mesh center coordinate, and the outer normal direction is normalized so that it retains only the direction information and is not affected by the length, thereby obtaining the surface normal vector.

[0035] S1022. Subtract the viewpoint parameters of the current virtual camera from the coordinates of the grid center by vector subtraction to obtain the line-of-sight vector from the current virtual camera to the coordinates of the grid center.

[0036] The viewpoint parameter includes the spatial position of the current virtual camera in the 3D scene; the gaze direction vector is used to represent the direction from the current virtual camera to the center coordinates of the mesh where the texture tile is located.

[0037] In this embodiment, after obtaining the mesh center coordinates, the mesh center coordinates can be vector-subtracted from the current virtual camera's viewpoint position to obtain the direction from the current virtual camera to the mesh center coordinates. To facilitate subsequent comparison with the surface normal vector, this direction can be normalized so that the viewing direction vector also retains only directional information. In this way, subsequent judgments will not be affected by the distance between the current virtual camera and the texture tile.

[0038] S1023. Calculate the inner product between the surface normal vector and the line-of-sight direction vector, and determine the spatial geometric angle based on the inner product.

[0039] The inner product value refers to the dot product between the surface normal vector and the viewing direction vector, which is used to reflect the relative relationship between the two directions. The inner product value only corresponds to the surface normal vector and the viewing direction vector, and does not refer to the dot product result between other vectors. Since both the surface normal vector and the viewing direction vector can be normalized, the inner product value can be used to determine whether the two tend to be in the same direction, perpendicular, or opposite, and further determine the orientation state reflected by the spatial geometric angle.

[0040] In this embodiment, a vector dot product algorithm can be used to process the surface normal vector and the viewing direction vector. Specifically, the corresponding components of the two vectors in different coordinate directions are multiplied and then summed to obtain the dot product value. Since both the surface normal vector and the viewing direction vector have been normalized, the dot product value can stably reflect the directional relationship between them: if the dot product value is greater than zero, it indicates that the surface normal vector and the viewing direction vector form an acute angle; if the dot product value is less than or equal to zero, it indicates that the texture patch surface has not been determined to be facing away from the current virtual camera. Thus, the orientation state corresponding to the spatial geometric angle can be determined based on the dot product value.

[0041] S1024. When the inner product value is greater than zero, it is determined that the surface normal vector and the viewing direction vector form an acute angle, confirming that the corresponding texture block is facing away from the current virtual camera. At the same time, it is determined that the texture block is located behind the target 3D model, and the identification information of the corresponding texture block is removed from the first block set to obtain the second block set.

[0042] Here, "facing away from the current virtual camera" means that the outer direction of the surface of the model where the texture slice is located is roughly the same as the direction in which the current virtual camera points to the texture slice, so that the surface of the texture slice is in a back position relative to the current viewing direction; the second slice set is the set of texture slice identification information obtained by deleting the back texture slices based on the first slice set.

[0043] In this embodiment, after obtaining the inner product value corresponding to each texture slice, it can be determined whether the inner product value is greater than zero one by one: when the inner product value corresponding to a certain texture slice is greater than zero, it means that the surface normal vector of the texture slice forms an acute angle with the viewing direction vector, and the texture slice is in a back view state relative to the current virtual camera. Therefore, the identification information of the texture slice is removed from the first slice set; when the inner product value corresponding to a certain texture slice is less than or equal to zero, it means that the texture slice has not been determined to be a back view slice and can continue to be retained in the set. After completing the traversal of all texture slices, the remaining texture slice identification information forms the second slice set.

[0044] As an example, in a video twin park scene, step 101 has determined that the current virtual camera is observing a building and obtained the first set of texture blocks corresponding to the building's walls, roof, and back structure. First, the grid center coordinates of each texture block on the surface of the target 3D model are read, and the unit normal vector pointing outward is obtained based on the pre-stored grid normal data of the grid region; second, the viewpoint parameters of the current virtual camera are vector subtracted from the coordinates of each grid center to obtain the viewing direction vector from the current virtual camera to each texture block. Then, a vector dot product algorithm is used to compare the directional relationship between the surface normal vector and the viewing direction vector to obtain the corresponding dot product value, and the orientation state reflected by the spatial geometric angle is determined accordingly. Finally, texture pieces with dot product values ​​greater than zero are identified as texture pieces located on the back of the target 3D model and removed from the first piece set. The remaining texture piece identification information of walls, roofs, or other texture pieces that may be sampled from the current viewpoint forms the second piece set. The above example is only one example of this application. In practical applications, it can also be set according to the model normal storage method, mesh accuracy, and rendering requirements. This application does not limit this.

[0045] In another specific implementation, if a region of the target 3D model lacks pre-stored mesh normal data, the outer direction of the facet can be determined based on the vertex order of the corresponding triangular facet in that region. Then, the directions of adjacent facets are smoothed to obtain the surface normal vector corresponding to the texture patch. This method also provides a clear directional basis for subsequent inner product calculations.

[0046] This application eliminates texture slices facing away from the current virtual camera in the first slice set by continuously judging the surface normal vector, the viewing direction vector, and the inner product value. This reduces the number of invalid texture slices entering the subsequent resolution calculation and data transmission process, and makes the second slice set match the actual rendering requirements under the current viewpoint.

[0047] S103. Based on the spatial distance between the viewpoint corresponding to the viewpoint parameter and each texture block, determine the target resolution to be loaded for each texture block in the second block set, and perform downgrading processing on the target resolution of texture blocks whose spatial geometric angle is within a preset angle range.

[0048] The target resolution represents the texture clarity level that the texture slice should be loaded in the current frame. The preset angle range represents the angle range of the texture slice when it is viewed at an angle. For example, the preset angle range can be limited by a first critical angle and a second critical angle. For example, the first critical angle can be 90 degrees and the second critical angle can be 135 degrees. In this case, the preset angle range can be an angle range greater than or equal to 90 degrees and less than or equal to 135 degrees. The downgrading process means that when the spatial geometric corner of the texture slice falls into the preset angle range, the target resolution of the texture slice is reduced from the current level to the level corresponding to the preset resolution level difference in the direction of lower clarity, using the pre-established resolution ladder as the adjustment object. The reduced resolution level is then used as the new target resolution.

[0049] In this embodiment of the application, S103 specifically includes: S1031. Calculate the spatial distance between the viewpoint of the current virtual camera and the coordinates of the center of the mesh of each texture block in the second block set on the surface of the target 3D model.

[0050] Spatial distance is used to represent the distance between the current virtual camera's viewpoint and the location of the texture tile on the model surface.

[0051] In this embodiment, the viewpoint parameters of the current virtual camera are first read to obtain the position of the current virtual camera in the three-dimensional scene; then, the mesh center coordinates corresponding to each texture block in the second block set are read one by one; then, a three-dimensional spatial distance algorithm is used to calculate the distance between the viewpoint of the current virtual camera and the mesh center coordinates to obtain the spatial distance corresponding to each texture block. The smaller the spatial distance, the closer the texture block is to the current observation position, and its display area on the screen increases as the spatial distance decreases; the larger the spatial distance, the greater the distance between the texture block and the current observation position, and its display area on the screen decreases as the spatial distance increases. The calculation process of spatial distance can be represented by formula (1): (1) in, Indicates the first Spatial distance between a texture slice and the current virtual camera's viewpoint; , , Indicates the first The center coordinates of the mesh for each texture segment; , , This represents the viewpoint coordinates of the current virtual camera.

[0052] S1032. Obtain the field of view parameters of the current virtual camera and the screen resolution of the rendering terminal, and perform perspective projection calculation in combination with the spatial distance to calculate the actual projection coverage of the corresponding texture block on the physical screen.

[0053] Among them, the field of view parameter is used to represent the spatial opening range that the current virtual camera can see; the screen resolution of the rendering terminal is used to represent the pixel scale when the image is finally output to the screen; and the projection coverage is used to represent the display area occupied by the texture chunks on the physical screen after perspective projection.

[0054] In this embodiment, after obtaining the spatial distance corresponding to each texture chunk, the field of view parameters of the current virtual camera and the screen resolution of the rendering terminal are read, and perspective projection is used to map the texture chunks in three-dimensional space onto a two-dimensional screen plane. Specifically, the spatial boundary of the texture chunk can be determined based on its extent on the surface of the target three-dimensional model. Then, based on the current virtual camera's viewing direction, field of view parameters, and spatial distance, this spatial boundary is projected onto screen coordinates to obtain the actual projection coverage area occupied by the texture chunk on the physical screen. Thus, the loading specification is no longer determined solely by the original size of the texture itself, but rather by the actual display requirements of the texture chunk in the current image.

[0055] S1033. Extract the physical pixel quantity contained in the projection coverage area, match the level value that is aligned with the size of the physical pixel quantity in the pre-established resolution ladder, and determine the level value as the target resolution for the corresponding texture slice to be loaded.

[0056] Among them, physical pixel count is used to represent the pixel scale covered by the projection surface on the rendering terminal screen; the resolution ladder includes multiple resolution levels arranged from low to high, each resolution level corresponds to a loadable texture data specification, which is used to avoid the system arbitrarily generating irregular texture sizes in each frame; size alignment means selecting a resolution level from the resolution ladder that is not lower than physical pixel count and is closest to that physical pixel count, rather than requiring the pixel count corresponding to the resolution level to be exactly equal to the physical pixel count, nor selecting a resolution level lower than physical pixel count.

[0057] In this embodiment, the number of physical pixels is first calculated by counting the screen pixel area covered by the projection coverage area; then, the number of physical pixels is compared with a pre-established resolution ladder. The resolution ladder can be set in multiple levels from low to high. It can be pre-generated in the initialization stage and set in multiple levels in a proportionally increasing manner from low to high. For example, starting from 256×256, it increases sequentially to 512×512, 1024×1024 and 2048×2048. Each level corresponds to a loadable texture pixel range and texture data specification. During matching, the level value that is not lower than the number of physical pixels and is closest to the number of physical pixels can be selected, and the level value is used as the target resolution to be loaded for the corresponding texture chunk. In this way, the target resolution can meet the current screen display requirements and avoid loading texture data that exceeds the matching level corresponding to the current display requirements. The matching rule of the resolution ladder can be expressed by formula (2): (2) in, Indicates the first The target resolution of each texture slice to be loaded; Indicates the first Each texture slice corresponds to the number of physical pixels contained within the projected coverage area; This represents a pre-established resolution ladder; Represents the first step in the resolution ladder. Each level of value; Indicates the first The upper limit of the number of pixels that a resolution level can cover.

[0058] S1034. Define the angle region between the preset first critical angle and the second critical angle as the preset angle range.

[0059] The first critical angle is used to indicate the angle boundary at which degradation processing is considered; the second critical angle is used to indicate the upper limit boundary of the tilted viewing state; the preset angle range is the angle interval formed between the first critical angle and the second critical angle, used to determine whether the texture block is in a tilted display state that requires a reduction in target resolution.

[0060] In this embodiment, the first critical angle and the second critical angle can be preset based on the degree of compression of the texture patch's projection on the screen, the texture detail of the target 3D model, and the display requirements of the rendering terminal. After the spatial geometric angle has been determined based on the surface normal vector and the viewing direction vector in the aforementioned steps, it is assumed that the spatial geometric angle is compared with the preset angle range formed by the first critical angle of 90 degrees and the second critical angle of 135 degrees. If the spatial geometric angle is greater than or equal to 90 degrees and less than or equal to 135 degrees, it indicates that the texture patch is in a tilted viewing state, and its effective display area on the screen is relatively compressed, so it can proceed to the subsequent degradation processing judgment; if the spatial geometric angle is not within this range, the degradation processing in this step is not triggered.

[0061] S1035. When the spatial geometric angle falls into the preset angle range, the target resolution level of the texture block is reduced according to the preset resolution level difference, and the reduced target resolution level is used as the new target resolution of the corresponding texture block.

[0062] The preset resolution level difference indicates the number of levels the target resolution needs to be reduced towards lower resolution in the resolution ladder, and its value is an integer greater than or equal to 1. For example, the preset resolution level difference can be 1, indicating that the current target resolution is reduced to the adjacent lower resolution level; or it can be 2, indicating that it jumps down two resolution levels; the new target resolution refers to the resolution level used for actually loading texture data after the downgrading process.

[0063] In this embodiment, it is first determined whether the spatial geometric angle falls within a preset angle range: if it does not, the target resolution determined in S1033 is maintained; if it does, it indicates that the texture block is in an tilted viewing state, and its effective display detail on the screen is lower than that in a frontal viewing state. At this time, the target resolution is adjusted to a lower level in the resolution ladder according to the preset resolution level difference. If the adjusted level is lower than the lowest level in the resolution ladder, the lowest level is used as the new target resolution, thereby avoiding invalid levels that cannot be loaded. Thus, the downgrading process does not reduce the resolution by an arbitrary proportion or an arbitrary number of pixels, but rather adjusts the resolution by a predetermined level according to the pre-established resolution ladder, thereby giving the downgraded target resolution a clear determination method.

[0064] As an example, in the video twin park scene, back face tiles have been removed from the first tile set, resulting in a second tile set. First, the viewpoint of the current virtual camera and the grid center coordinates of each texture tile in the second tile set are read. A 3D spatial distance algorithm is used to calculate the spatial distance between each texture tile and the current virtual camera. Second, the field of view parameters of the current virtual camera and the screen resolution of the rendering terminal are read. Perspective projection is used to project texture tiles such as building walls and roofs onto the screen plane, obtaining the actual projection coverage area occupied by each texture tile on the physical screen. Next, the number of physical pixels contained within the projection coverage area is counted, and the layer value that is not lower than and closest to the number of physical pixels in the resolution hierarchy is selected as the target resolution. Then, the first critical angle and the second critical angle are read, and the angle region between them is defined as a preset angle range. Finally, it is determined whether the spatial geometric angle of each texture tile falls within the preset angle range. If a wall texture tile is displayed tilted, its target resolution is reduced to the adjacent lower layer according to the preset resolution layer difference, resulting in a new target resolution. The above example is only one example of this application. In practical applications, the settings can be made according to the model complexity, screen display requirements and video memory resources of the video twin scene. This application does not limit this.

[0065] In another specific implementation, the resolution ladder can be pre-generated during the initialization phase based on the rendering terminal's screen resolution, graphics processing unit memory capacity, and the original resolution of texture chunks. Specifically, the lowest and highest resolution levels that texture chunks are allowed to load can be determined first, and then intermediate levels can be generated according to a proportional increment rule to ensure a stable progression relationship between adjacent levels; alternatively, an arithmetic increment or a custom level table can be used according to actual rendering requirements. Each resolution level records the corresponding texture width, texture height, texture data specifications, and the range of physical pixels it can cover. In this way, only matching and level adjustment need to be performed during runtime, without needing to regenerate the resolution specifications for each frame.

[0066] This application enables the target resolution to be determined according to the current observation position and screen display requirements through continuous processing of spatial distance, projection coverage, physical pixel quantity and resolution steps; at the same time, by limiting the triggering conditions and adjustment methods of degradation processing through preset angle range and preset resolution level difference, it can reduce the overloading of tilted display texture chunks and make subsequent texture transmission match the actual rendering requirements of the current screen.

[0067] S104. In the main memory, all texture slices in the second slice set are encapsulated into a continuous data stream. At the same time, in the graphics processing unit memory, the target resolution is combined, and a continuous memory block for carrying the continuous data stream is constructed using a pixel buffer object (PBO).

[0068] The main memory includes memory space directly accessible to the processor, used to temporarily organize texture chunks to be transmitted. The continuous data stream includes multiple chunk data units in the main memory arranged in a preset order, each chunk data unit including a chunk header and texture data. The chunk header describes the data boundaries and write information of the corresponding texture chunk, including at least the texture chunk's identification information, target resolution, texture format, data length, and data offset. The texture data is used to store the actual texture content that the texture chunk needs to be written to the continuous video memory block at the target resolution. The pixel buffer object (PBO) includes a buffer object provided by the graphics processing unit, used to receive data from the main memory side when the texture data is uploaded, and to enable the graphics processing unit to read the data without blocking the rendering process. The continuous video memory block includes a storage area in the graphics processing unit's video memory that is continuously addressed according to a unified starting address, used to enable the texture chunks in the second chunk set to be written sequentially according to the order of the continuous data stream.

[0069] In this embodiment of the application, when encapsulating a continuous data stream, the arrangement order of each texture block in the continuous data stream is first determined according to the rendering access order of each texture block in the second block set, the spatial adjacency relationship of the target 3D model surface, or the identification information order of the texture blocks; secondly, a block header is generated for each texture block, and the block header is written before the corresponding texture data, so that the subsequent transmission process can identify the data boundary of each texture block according to the data length and data offset in the block header; then, multiple block data units are connected end to end in the order of arrangement to form a continuous data stream.

[0070] In one alternative implementation, no additional separators are used between the data units in each slice. Instead, the end position of the current data unit is determined by the data length in the slice header, and the next data unit or corresponding video memory write position is determined by the data offset. The texture data can maintain the original texture format used when the texture slices were generated, without needing to be compressed or re-encoded when encapsulating continuous data streams. If the texture slice itself uses a compressed texture format, the corresponding texture format is recorded in the slice header, and the data is encapsulated according to the data length corresponding to that format.

[0071] In graphics processing unit (GPU) memory, combining the target resolution means that when constructing contiguous memory blocks, the GPU memory capacity required for each texture slice is determined based on the target resolution and texture format of each texture slice. Based on this, the page block specification, storage unit length, and address offset corresponding to the texture slice are determined. This process does not simply write the target resolution value into the contiguous memory block, but rather uses the target resolution to participate in the capacity allocation and storage structure construction of the contiguous memory block.

[0072] In this embodiment, after obtaining the target resolution of each texture slice in the second slice set, the texture sharpness that the texture slice needs to be loaded in the current frame is first determined based on the target resolution. Then, the amount of data that the texture slice needs to be written to the graphics processing unit's video memory is determined in conjunction with the texture format. Next, the amount of data is compared with a preset resolution threshold or page block capacity. Large-capacity page blocks are allocated to texture slices whose data amount exceeds the standard page block capacity, and standard page blocks are allocated to the remaining texture slices. Then, according to the page block specifications corresponding to each texture slice, storage units are sequentially divided in the graphics processing unit's video memory, and the storage units are spliced ​​into a contiguous video memory block by address offset.

[0073] like Figure 3 As shown, the continuous data stream in the main memory indicates that each texture slice in the second slice set has been encapsulated in the transmission order, where each texture slice may include a slice header and texture data; asynchronous writing indicates that the continuous data stream is written to the continuous video memory block bound to the pixel buffer object PBO through an asynchronous transmission channel. A pixel buffer object PBO32 is set in the video memory area 31 on the graphics processing unit side. The pixel buffer object PBO32 is used to carry the continuous data stream encapsulated by the second slice set. The continuous video memory block takes the global starting address 33 as the unified addressing starting point, and sets standard page blocks 34 and large-capacity page blocks 35 in it according to the target resolution. Boundary padding blocks 36 are used to fill the addressing gaps formed by hardware alignment boundaries. Storage units 37 are used to carry the aligned texture slice data; when the video memory pointer 38 is redirected to point to the target addressing location, the continuous data stream can be written to the corresponding storage units 37 in sequence according to the global starting address 33 and the address offset.

[0074] In this embodiment of the application, S104 specifically includes: S1041. The target resolution is compared with a preset resolution threshold. Large-capacity page blocks are allocated to texture blocks whose target resolution is higher than the resolution threshold, and standard page blocks are allocated to the remaining texture blocks.

[0075] The standard page block includes fixed-size memory units for carrying texture slices with a normal target resolution, while the large-capacity page block includes extended-size memory units for carrying texture slices with a target resolution higher than a preset resolution threshold. The page block specifications include page block capacity, alignment requirements, and available storage length. The preset resolution threshold includes resolution level boundaries for distinguishing between standard page blocks and large-capacity page blocks. These boundaries can be preset levels in a resolution ladder or texture pixel quantity boundaries corresponding to the preset level.

[0076] For example, the resolution threshold can be set to the resolution level corresponding to 1024×1024 texture pixels. When the target resolution of the texture slice is higher than this resolution level, it is determined that the texture slice needs to occupy more video memory capacity than the standard page block carrying capacity, and a large-capacity page block is allocated to it. When the target resolution of the texture slice is not higher than this resolution level, it is determined that the texture slice is within the standard video memory carrying capacity, and a standard page block is allocated to it. The above 1024×1024 is only an example. In practical applications, the resolution threshold can also be set according to the maximum amount of texture data that a standard page block can carry, the texture format, the PBO mapping granularity, and the video memory capacity of the graphics processing unit. This application does not limit this.

[0077] In this embodiment, the target resolution is first read, and the texture data size corresponding to the target resolution is determined by combining the texture format used by the texture chunk. Specifically, the target resolution is used to determine the pixel scale that the texture chunk needs to retain in the width and height directions, and the texture format is used to determine the amount of data occupied by each pixel or each compressed data block during storage. Based on this, the complete data length that the texture chunk needs to be written at the current target resolution is calculated. If the texture chunk uses a compressed texture format, the length is calculated using the compressed data block as the basic unit, and necessary line alignment padding is also included to ensure that the complete boundary is maintained when writing to the PBO. In specific comparison, the target resolution can be converted into the corresponding resolution hierarchy or texture pixel count, and then compared with a preset resolution threshold to determine whether the texture chunk should use standard page blocks or large-capacity page blocks. Therefore, the resolution threshold is not an arbitrary proportional value, but a clear resolution hierarchy boundary used for page block allocation.

[0078] Next, the target resolution is compared with a preset resolution threshold. When the target resolution exceeds the preset threshold, it indicates that the amount of texture data corresponding to the texture slice meets the allocation condition for a large-capacity page block, so a large-capacity page block is allocated to the texture slice. When the target resolution does not exceed the preset threshold, it indicates that the amount of data required by the texture slice is within the normal carrying capacity range, so a standard page block is allocated to it. Through this process, each texture slice in the second slice set can obtain a page block specification that matches its target resolution, providing a clear capacity basis for the construction of subsequent consecutive memory blocks.

[0079] S1042. Based on the arrangement order of the continuous data stream and the page block specifications corresponding to each texture slice, memory units with connected physical addresses are allocated sequentially in the graphics processing unit memory, and concatenated to construct a continuous memory block with a global starting address.

[0080] S1042 may specifically include the following steps: A1. According to the arrangement order of the continuous data stream, accumulate the byte size of all preceding blocks to obtain the initial address of the current texture block; A2. Based on the corresponding page block specification, determine the spatial distance between the initial address and the hardware alignment boundary to obtain the address gap, and use boundary padding blocks to fill the address gap to complete the boundary alignment, obtaining an aligned storage segment containing the boundary padding blocks; A3. Calculate the sum of the initial address and the size of the boundary padding blocks to obtain the address offset of the current texture block; A4. Use the global starting address as the address zero point, and... The corresponding address offsets are summed to calculate the target addressing position of the current texture slice; A5. Using the target addressing position as the storage starting point, a physical storage area of ​​the corresponding size is specified and divided in the graphics processing unit video memory according to the byte length of the corresponding aligned storage segment, as the allocated storage unit; A6. The corresponding video memory pointer is redirected to the target addressing position to guide the continuous data stream to be written uninterruptedly into each of the allocated storage units, so that each of the storage units is connected end to end in physical space to obtain a continuous video memory block.

[0081] The graphics processing unit memory includes a memory area on the graphics processing unit side used to store texture data, which in this application mainly refers to the memory space allocated and managed by the PBO; the global starting address includes the starting address of a contiguous memory block, and the write position of each subsequent texture slice is offset based on this global starting address; the hardware alignment boundary includes the memory page alignment boundary corresponding to the pixel buffer object PBO mapping granularity, which is used to limit the actual write starting point of the texture slice in the contiguous memory block. The hardware alignment boundary is mainly used to match the write address of the texture slice with the read and write granularity required by the graphics processing unit memory access and PBO mapping. For example, the hardware alignment boundary can be a 4KB memory page boundary. When the graphics processing unit or PBO mapping rules require a page alignment granularity greater than 4KB, a 64KB video memory page boundary can also be used. The specific setting can be based on the graphics processing unit's video memory management rules and PBO mapping granularity. The boundary padding block includes non-texture data filled in the address gap. Its content can consist of zero-value bytes. Zero-value bytes do not represent any texture color information and do not participate in texture sampling. The size of the boundary padding block is equal to the byte distance between the initial address of the current texture slice and the next hardware alignment boundary. It is used to fill the address gap so that the current texture slice or the next texture slice can start writing from the position that conforms to the hardware alignment boundary.

[0082] In this embodiment, a write order table is first established based on the chunk headers in the continuous data stream. This write order table records the identifier information, data length, target resolution, and corresponding page block specifications of each texture chunk according to the arrangement order of the continuous data stream, and records the occupied space in the continuous video memory block through a sequentially increasing address cursor. In this way, when processing any current texture chunk, the order of the current texture chunk in the graphics processing unit's video memory and the storage capacity that needs to be reserved can be clearly determined.

[0083] For the current texture slice, the current address cursor position is used as the initial address of the current texture slice. This initial address is obtained by sequentially accumulating the space occupied by all previous slices, thus ensuring that different texture slices are not randomly written to video memory, but are arranged sequentially according to the arrangement order of the continuous data stream. Then, according to the page block specification corresponding to the current texture slice, the PBO mapping granularity corresponding to the page block specification is read, and the video memory page boundary corresponding to the PBO mapping granularity is used as the hardware alignment boundary. If the initial address of the current texture slice does not fall on the hardware alignment boundary, the byte distance between the initial address and the next video memory page alignment boundary is calculated to obtain the address gap. Subsequently, a boundary padding block of the same size as the address gap is generated and used to fill the address gap, so that the current texture slice can start writing from the position that meets the PBO mapping granularity requirement. The boundary padding block is only used to push the actual write start point of the current texture slice to the position that meets the hardware alignment boundary and does not participate in subsequent rendering sampling as texture content.

[0084] After boundary alignment, the actual write position after boundary padding is recorded as the address offset of the current texture slice, and this address offset is associated with the corresponding identification information. Then, using the global starting address of the contiguous memory block obtained through PBO as a unified addressing starting point, the global starting address and the address offset are combined to obtain the target addressing position of the current texture slice. Since each texture slice calculates its target addressing position based on the same global starting address, subsequent rendering stages only need to query the corresponding address offset based on the texture slice's identification information to locate the storage location of the texture slice in the contiguous memory block.

[0085] After the target addressing location is determined, a physical storage region matching the length of the aligned storage segment is allocated in the graphics processing unit's video memory, starting from that location, as the storage unit allocated to the current texture slice. After the storage unit is allocated, its end position is used as the starting point of the address cursor when allocating the next texture slice, allowing subsequent texture slices to continue to be arranged. In this way, even if both standard page blocks and large-capacity page blocks exist in the second slice set, storage units of different sizes can be orderly connected within the same contiguous video memory block, avoiding address overwriting or write misalignment between adjacent texture slices.

[0086] Finally, the memory pointer is redirected to the target addressing location of the current texture slice. This redirection process can be understood as follows: on the writable memory entry obtained by PBO mapping, the write starting point is moved according to the address offset of the current texture slice, so that the corresponding texture data in the continuous data stream can be directly written to the allocated storage unit; after the current texture slice is written, the memory pointer continues to move to the starting point of the next storage unit, and continues to process the next texture slice according to the write order table. Thus, the continuous data stream can be written to each allocated storage unit in sequence, so that each storage unit is connected end to end in physical space, and finally spliced ​​to construct a continuous memory block with a global starting address. In an optional implementation, the address offset and target addressing location of the current texture slice can be determined according to formula (3) and formula (4): (3) (4) in, Indicates the first The address offset of each texture slice; Indicates the first The size of the texture data in bytes for each preceding texture slice; Indicates the first Size of the boundary fill block corresponding to each preceding texture slice; Indicates the first The size of the boundary padding block required before writing the texture slice is equal to the size of the first texture slice. The byte distance between the initial address of a texture slice and the next hardware alignment boundary, when the initial address already meets the hardware alignment boundary requirements. It is zero; Indicates the first The target address location of each texture slice; This represents the global starting address of a contiguous block of video memory. The formula above illustrates how memory cells are addressed contiguously under a unified starting address, without specifying a particular address value.

[0087] As an example, in the video twin park scene, the second set of slices includes multiple texture slices corresponding to the building's front facade, side facade, and roof area. First, the target resolution of each texture slice is read and compared with a preset resolution threshold. Texture slices in the front facade with a target resolution higher than the preset resolution threshold are assigned large-capacity slices, while texture slices in the side facade and roof with a target resolution not exceeding the preset resolution threshold are assigned standard slices. Secondly, these texture chunks are processed sequentially according to the order of the continuous data stream. First, a chunk header and texture data are written to the main memory for each texture chunk. Then, the address offset is calculated sequentially in the graphics processing unit's video memory according to the page block specification, the hardware alignment boundary corresponding to the PBO mapping granularity, and the boundary padding block. Finally, the video memory pointer is pointed to the corresponding target address location through the PBO, so that the continuous data stream can be written to the continuous video memory blocks in sequence. The above example is only one example of this application. In practical applications, it can also be set according to the scale of the 3D model, the texture format, and the rendering terminal capabilities. This application does not limit this.

[0088] This application enables texture slices with different target resolutions to be written to contiguous video memory blocks in a unified addressing manner through the coordination of continuous data streams, page block specifications, hardware alignment boundaries, and PBOs. This reduces fragmented video memory allocation and multiple intermediate copying, which helps improve the stability of streaming loading and rendering continuity of ultra-large textures in video twin scenes.

[0089] S105. Establish an asynchronous transmission channel, and transmit the continuous data stream directly from the main memory to the continuous video memory block through the asynchronous transmission channel with zero copy.

[0090] The asynchronous transmission channel includes a data transmission path formed by the main memory, memory controller, hardware bus, and pixel buffer object (PBO) bound to a continuous video memory block. It is used to complete data writing without the central processing unit (CPU) moving texture data segment by segment. Zero-copy direct transmission means that the CPU is only responsible for configuring the starting address, total data length, and transmission trigger conditions. Instead of copying the continuous data stream to an additional intermediate cache, the hardware bus directly writes the continuous data stream from the main memory to the continuous video memory block.

[0091] In this embodiment of the application, S105 specifically includes: S1051. Write the starting address and total data length of the continuous data stream in the main memory into the configuration register inside the memory controller used to control DMA transfer.

[0092] The memory controller is a hardware control unit used to control the DMA transfer process. It reads transfer parameters, initiates transfer requests, and controls the hardware bus to complete data transfer. The hardware bus is the data path used to perform DMA transfers. Under the control of the memory controller, it connects the main memory to the continuous video memory blocks bound to the pixel buffer object (PBO). The configuration register includes a hardware register area inside the memory controller used to store transfer parameters. It records the starting address of the continuous data stream, the total data length, the transfer status, and the completion flag. The completion flag includes status information written by the memory controller to the configuration register or synchronization object after the transfer is completed. It informs the graphics processing unit that the texture data in the continuous video memory blocks has been written completely. The starting address indicates the starting point of the continuous data stream in the main memory, and the total data length indicates the range of data to be transferred this time.

[0093] In this embodiment, after the continuous data stream encapsulation is completed, the storage area of ​​the continuous data stream in the main memory is first locked to prevent it from being overwritten during transmission. Then, the starting position of the storage area is read as the starting address, and the complete length from the first data block to the last data block is counted as the total data length. Then, the central processing unit writes the starting address and the total data length into the configuration register inside the memory controller. At this time, the central processing unit only completes the configuration of the transmission task and does not perform the actual copying of the continuous data stream.

[0094] S1052. Establish a physical path from the main memory to the contiguous video memory block bound by the pixel buffer object PBO, as the transmission channel.

[0095] The physical path includes the hardware access path for reading a continuous data stream from the main memory and writing it into a continuous block of video memory. This path can be determined by video memory mapping relationship, PBO binding relationship and bus access permission.

[0096] In this embodiment, it is first confirmed that the PBO has been bound to the contiguous video memory block constructed in S104, and the global starting address of the contiguous video memory block is obtained. Then, the starting address in the main memory, the global starting address of the contiguous video memory block, and the transfer permission are written into the transfer control information, so that the memory controller can identify where the data is read from and where it is written. Through this process, a defined transfer channel is formed between the main memory and the contiguous video memory block, and the subsequent hardware bus can perform continuous writing along this transfer channel without the central processing unit having to specify the write position for each texture block.

[0097] S1053. In asynchronous mode where the central processing unit is not required to participate in data transfer, the transmission channel is triggered to drive the memory controller to read the starting address and the total data length in the configuration register.

[0098] Asynchronous mode includes a working mode in which the central processing unit can continue to execute other scheduled tasks after issuing a transmission start command, and the actual data transfer is completed by the memory controller and hardware bus.

[0099] In this embodiment, after completing the configuration register writing and transmission channel establishment, the central processing unit (CPU) sends a start signal to the memory controller. Upon receiving the start signal, the memory controller reads the starting address and total data length from the configuration register and uses them as the basis for this transmission. Because this process is performed asynchronously, the CPU does not need to wait for each segment of texture data transmission to complete, nor does it need to participate in the byte-by-byte copying of continuous data streams, thus allowing it to continue processing the visual inspection, rendering instruction organization, or resource status maintenance of the next frame.

[0100] S1054. The memory controller controls the hardware bus used to perform DMA transfer based on the read starting address and the total data length, so that the hardware bus switches to bus master state.

[0101] The bus master state refers to the working state in which the hardware bus obtains active read and write access permissions. In this state, the hardware bus can read a continuous data stream from the main memory and write data to a continuous block of video memory according to the starting address and total data length recorded in the configuration register, without requiring the central processing unit to initiate read and write operations for each segment of texture data one by one.

[0102] In this embodiment, the memory controller determines the starting point of the continuous data stream based on the starting address, determines the termination range of the current transmission based on the total data length, and sends a transmission request to the hardware bus. After obtaining access authorization, the hardware bus switches to the bus master state. At this time, the hardware bus no longer waits for the central processing unit to issue individual transport instructions for each texture chunk, but instead continuously performs read and write operations according to the transmission range in the configuration register, thereby forming a batch transmission mode suitable for continuous data streams.

[0103] S1055. In the bus master control state, the hardware bus sequentially reads the continuous data stream from the main memory according to the starting address and writes it directly into the continuous video memory block, bypassing the central processing unit.

[0104] In this process, direct writing across the CPU involves continuous data streams not entering the CPU's intermediate copy cache, nor being repeatedly moved by the CPU. Instead, the data is delivered to the continuous video memory block bound to the PBO via the hardware bus. The CPU is only used to write to the configuration register, trigger the transfer channel, and read the transfer completion flag. The reading and writing of the continuous data stream are completed by the memory controller and the hardware bus, and the writing target is the continuous video memory block bound to the PBO.

[0105] In this embodiment, the hardware bus reads data from the starting address according to the sequential arrangement of the continuous data stream. Since the continuous data stream has been encapsulated sequentially in S104 according to the chunk header and texture data, the hardware bus can sequentially read the complete data segment and write it into the corresponding memory cell in the continuous video memory block. During the writing process, the PBO provides a writable entry point corresponding to the continuous video memory block, and the address offset in the continuous video memory block ensures that different texture chunks are written to their respective corresponding positions. In this way, the continuous data stream can directly enter the graphics processing unit's video memory from the main memory, avoiding the formation of an additional data copy on the central processing unit side.

[0106] S1056. During the writing process of the hardware bus, the total data length in the configuration register is synchronously reduced until the total data length is reduced to zero, at which point the hardware bus is released to complete the zero-copy direct transmission of the continuous data stream.

[0107] The decreasing total data length indicates the amount of data remaining to be transmitted; releasing the hardware bus includes ending the current bus master state and handing the hardware bus back to other hardware access tasks.

[0108] In this embodiment, after each segment of continuous data stream is written by the hardware bus, the memory controller updates the total data length in the configuration register according to the amount of data written, so that the total data length continuously represents the remaining transmission range. When the total data length decreases to zero, it indicates that the continuous data stream starting from the first address has been completely written to the contiguous video memory block. Then, the memory controller stops the current transmission task, releases the hardware bus, and writes a transmission completion flag. The graphics processing unit can use this transmission completion flag to determine whether the texture data in the contiguous video memory block is ready, thereby avoiding premature sampling before the data is completely written.

[0109] After transferring the continuous data stream directly from main memory to the contiguous video memory block via an asynchronous transfer channel using zero-copy transfer, the process also includes: B1. When the hardware pipeline of the graphics processing unit receives a rendering instruction for the target 3D model, it calls the rendering shader to perform texture sampling on the ready texture data in the contiguous video memory block according to the vertex texture coordinates of the target 3D model; B2. After the texture sampling operation is completed and the current frame is submitted, it releases the exclusive lock state of the contiguous video memory block and marks the contiguous video memory block as available; B3. It cancels the global starting address of the contiguous video memory block, releases the physical address concatenation lock of each memory unit, and disassembles the contiguous video memory block back into independent large-capacity page blocks and standard page blocks for cyclic use in the next transmission cycle.

[0110] Among them, ready texture data refers to texture data that has been completely written into the contiguous video memory block according to the starting address and total data length, and has been confirmed to be readable by the graphics processing unit through the transfer completion flag; the transfer completion flag can be written by the memory controller into the configuration register, synchronization object or resource status flag after the total data length is reduced to zero, to indicate that the texture data in the contiguous video memory block has been completely written.

[0111] Exclusive lock state refers to the resource occupancy state in which a contiguous block of video memory is marked as non-reusable and non-overwriteable during data writing or rendering sampling. It can be achieved through resource status flags, synchronization signals, mutexes, fence objects, or rendering completion callbacks. This state is not limited to a specific software lock, but is used to ensure that a contiguous block of video memory will not be prematurely overwritten in the next transfer cycle when it is written to by the hardware bus or read by the rendering shader.

[0112] The available state refers to the state where the contiguous video memory block has completed texture sampling and image submission for the current frame and is no longer occupied by the current frame rendering process, and can re-enter the page block allocation and transmission multiplexing process; the storage unit refers to the physical storage area in the video memory of the graphics processing unit that is divided according to the address offset and used to carry texture block data; large capacity page block and standard page block refer to video memory page blocks of different capacity specifications allocated according to the target resolution and the preset resolution threshold.

[0113] In this embodiment, when the graphics processing unit's hardware pipeline receives a rendering instruction for the target 3D model, it first reads the transfer completion flag corresponding to the contiguous video memory block. If the transfer completion flag indicates that the continuous data stream has been completely written to the contiguous video memory block, it confirms that the texture data in the contiguous video memory block is in a ready state. Subsequently, the rendering shader is called to read the vertex texture coordinates of the target 3D model, and combined with the address offset of each texture slice in the contiguous video memory block, texture sampling operations are performed from the corresponding storage units.

[0114] After the texture sampling operation is completed and the current frame is submitted, it indicates that the contiguous memory block is no longer being read during the current frame rendering process. At this point, the exclusive lock state of the contiguous memory block is released, and the contiguous memory block is marked as available. Subsequently, the global starting address of the contiguous memory block is deregistered, the physical address concatenation lock between each memory unit is released, and the contiguous memory block is disassembled and restored into independent large-capacity pages and standard pages according to the page block specification, so that it re-enters the page block pool for cyclic use in the next transfer cycle.

[0115] As an example, in a video twin park scene, the current frame needs to load multiple texture chunks of the building facade. First, the starting address and total data length of the continuous data stream in main memory are written to the configuration register. Second, the write relationship between main memory, PBO, and continuous video memory blocks is configured as a transmission channel. Next, asynchronous mode is initiated, and the hardware bus enters bus master mode. Then, the hardware bus sequentially reads the continuous data stream and directly writes it to the continuous video memory blocks. Finally, when the total data length decreases to zero, the hardware bus is released and a transmission completion flag is written. After confirming that the texture data is ready, the graphics processing unit performs texture sampling and releases the continuous video memory blocks after the current frame is submitted, allowing the corresponding large-capacity pages and standard pages to enter the next round of use. The above example is only one example of this application. In practical applications, settings can be made according to the hardware platform, PBO mapping method, and rendering pipeline scheduling method. This application does not limit these settings.

[0116] This application clarifies the transmission start and range through configuration registers, limits the active transport capability of the hardware bus through bus master status, and connects data transmission, texture sampling, and page block reuse through transmission completion flags, exclusive lock states, and available states. This enables continuous data streams to be stably and directly transferred to contiguous video memory blocks with zero copies, and reduces conflicts between data transport and rendering reads during the streaming loading of large textures. Therefore, zero-copy direct transfer in this application does not mean that the CPU is completely uninvolved in the transmission process, but rather that the CPU does not participate in copying the data body of the continuous data stream. The transmission process is completed through configuration registers, memory controllers, hardware buses, and PBO binding relationships, thus clearly defining the implementation boundaries of zero-copy direct transfer.

[0117] This application, through steps S101-S105, first determines the target 3D model and texture chunks to be displayed from the current perspective, then removes the invisible chunks on the back side, and determines the appropriate target resolution based on distance and viewing angle, so that the data to be loaded matches the actual screen requirements; then, the texture chunks are encapsulated into a continuous data stream, and a continuous video memory block and asynchronous transmission channel are constructed through PBO to achieve zero-copy direct transmission from main memory to video memory, thereby reducing invalid loading, scattered writing and intermediate copying, and improving the efficiency of ultra-large texture streaming transmission and rendering continuity.

[0118] Figure 4 This is a schematic diagram of the structure of a zero-copy streaming system for ultra-large textures based on asynchronous DMA and PBO disclosed in this application, with reference to... Figure 4 The system may include: The detection module 41 is used to detect the target 3D model within the visible range in a pre-established hierarchical bounding box according to the viewpoint parameters and view frustum parameters of the current virtual camera in the video twin engine, and to determine the identification information of all texture blocks associated with the target 3D model to obtain the first block set. The elimination module 42 is used to eliminate the identification information of the texture blocks located on the back of the target 3D model in the first block set based on the spatial geometric angle between the surface normal vector corresponding to each texture block on the surface of the target 3D model and the viewing direction of the current virtual camera, so as to obtain the second block set. The determination module 43 is used to determine the target resolution to be loaded for each texture block in the second block set based on the spatial distance between the viewpoint corresponding to the viewpoint parameter and each texture block, and to perform downgrade processing on the target resolution of texture blocks whose spatial geometric angle is within a preset angle range. The fusion module 44 is used to encapsulate all texture slices in the second slice set into a continuous data stream in the main memory, and combine the target resolution in the graphics processing unit video memory, and use pixel buffer objects (PBOs) to construct a continuous video memory block to carry the continuous data stream. Module 45 is used to establish an asynchronous transmission channel, through which the continuous data stream is directly transferred from the main memory to the continuous video memory block with zero copy.

[0119] The ultra-large texture zero-copy streaming system based on asynchronous DMA and PBO in this application embodiment is used to implement the aforementioned ultra-large texture zero-copy streaming method based on asynchronous DMA and PBO. Therefore, the specific implementation of the ultra-large texture zero-copy streaming system based on asynchronous DMA and PBO can be found in the embodiment section of the ultra-large texture zero-copy streaming method based on asynchronous DMA and PBO above. The specific implementation can be referred to the description of the corresponding embodiments, and will not be repeated here.

[0120] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0121] The foregoing has provided a detailed description of a method and system for zero-copy streaming of ultra-large textures based on asynchronous DMA and PBO. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for zero-copy streaming of ultra-large textures based on asynchronous DMA and PBO, characterized in that, include: Based on the viewpoint and frustum parameters of the current virtual camera in the video twin engine, the target 3D model within the visible range is detected in the pre-established hierarchical bounding box, and the identification information of all texture blocks associated with the target 3D model is determined to obtain the first block set. Based on the spatial geometric angle between the surface normal vector corresponding to each texture block on the surface of the target 3D model and the viewing direction of the current virtual camera, the identification information of the texture blocks located on the back of the target 3D model in the first block set is removed to obtain the second block set; Based on the spatial distance between the viewpoint corresponding to the viewpoint parameter and each texture block, the target resolution to be loaded for each texture block in the second block set is determined, and the target resolution of the texture blocks whose spatial geometric angle is within a preset angle range is downgraded. In the main memory, all texture slices in the second slice set are encapsulated into a continuous data stream. At the same time, in the graphics processing unit memory, the target resolution is combined, and a continuous memory block for carrying the continuous data stream is constructed using a pixel buffer object (PBO). An asynchronous transmission channel is established, through which the continuous data stream is directly transferred from the main memory to the continuous video memory block with zero copy. The zero copy transfer means that the central processing unit is only responsible for configuring the starting address, total data length and transmission trigger conditions, and no longer copies the continuous data stream to an additional intermediate cache. Instead, the hardware bus directly writes the continuous data stream from the main memory to the continuous video memory block.

2. The method according to claim 1, characterized in that, The step of combining the target resolution in the graphics processing unit's video memory and constructing a contiguous video memory block to carry the continuous data stream using pixel buffer objects (PBOs) includes: The target resolution is compared with a preset resolution threshold. Large-capacity page blocks are allocated to texture slices whose target resolution is higher than the resolution threshold, and standard page blocks are allocated to the remaining texture slices. Based on the arrangement order of the continuous data stream and the page block specifications corresponding to each texture slice, storage units with connected physical addresses are allocated sequentially in the graphics processing unit's video memory, and then spliced ​​together to construct a continuous video memory block with a global starting address.

3. The method according to claim 2, characterized in that, Based on the arrangement order of the continuous data stream and the page block specifications corresponding to each texture slice, the process of sequentially allocating physically connected storage units in the graphics processing unit's video memory and piecing them together to construct a contiguous video memory block with a global starting address includes: According to the arrangement order of the continuous data stream, the byte size of all preceding segments is accumulated to obtain the initial address of the current texture segment; Based on the corresponding page block specification, the spatial distance between the initial address and the hardware alignment boundary is determined to obtain the address gap. The address gap is then filled with boundary padding blocks to complete the boundary alignment, resulting in an aligned memory segment containing the boundary padding blocks. Calculate the sum of the initial address and the size of the boundary fill block to obtain the address offset of the current texture slice; The global starting address is used as the addressing zero point, and the address is added to the corresponding address offset to calculate the target addressing position of the current texture block; Using the target addressing location as the storage starting point, and according to the byte length of the corresponding aligned storage segment, a physical storage area of ​​the corresponding size is specified and divided in the graphics processing unit's video memory as the allocated storage unit. The corresponding video memory pointer is redirected to the target address location to guide the continuous data stream to be written uninterruptedly into each of the allocated memory units, so that each memory unit is connected end to end in physical space to obtain a continuous video memory block.

4. The method according to claim 1, characterized in that, The step of determining the target resolution to be loaded for each texture slice in the second slice set based on the spatial distance between the viewpoint corresponding to the viewpoint parameter and each texture slice includes: Calculate the spatial distance between the viewpoint of the current virtual camera and the mesh center coordinates of each texture slice in the second slice set on the surface of the target 3D model; The field of view parameters of the current virtual camera and the screen resolution of the rendering terminal are obtained. The perspective projection operation is performed in combination with the spatial distance to calculate the actual projection coverage of the corresponding texture block on the physical screen. Extract the physical pixel count contained within the projection coverage area, match the level value aligned with the physical pixel count size in a pre-established resolution ladder, and determine the level value as the target resolution for loading the corresponding texture slice.

5. The method according to claim 1, characterized in that, The step of downgrading the target resolution of texture segments whose spatial geometric angles are within a preset angle range includes: The angle region between the preset first critical angle and the preset second critical angle is defined as the preset angle range; When the spatial geometric angle falls within the preset angle range, the target resolution level of the texture block is reduced according to the preset resolution level difference, and the reduced target resolution level is used as the new target resolution of the corresponding texture block.

6. The method according to claim 1, characterized in that, The establishment of an asynchronous transmission channel, through which the continuous data stream is directly transferred from the main memory to the continuous video memory block via zero copy, includes: The starting address and total data length of the continuous data stream in the main memory are written into the configuration register inside the memory controller used to control DMA transfer. A physical path is established from the main memory to the contiguous video memory block bound by the pixel buffer object PBO, serving as the transmission channel; In asynchronous mode where no central processing unit is required to participate in data transfer, the transmission channel is triggered, causing the memory controller to read the starting address and the total data length from the configuration register; The memory controller controls the hardware bus used to perform DMA transfer based on the read starting address and the total data length, so that the hardware bus switches to the bus master state; In the bus master control state, the hardware bus sequentially reads the continuous data stream from the main memory according to the starting address, and writes it directly into the continuous video memory block, bypassing the central processing unit; During the writing process of the hardware bus, the total data length in the configuration register is synchronously decremented until the total data length is reduced to zero, at which point the hardware bus is released to complete the zero-copy direct transmission of the continuous data stream.

7. The method according to claim 1, characterized in that, The second set of texture blocks is obtained by removing the identification information of texture blocks located on the back side of the target 3D model from the first set of texture blocks based on the spatial geometric angle between the surface normal vector corresponding to each texture block on the surface of the target 3D model and the viewing direction of the current virtual camera. This second set includes: Obtain the mesh center coordinates of each texture block in the first block set on the surface of the target 3D model, and extract the unit normal vector facing outward at the mesh center coordinates as the surface normal vector; By subtracting the viewpoint parameters of the current virtual camera from the coordinates of the grid center, a line-of-sight vector from the current virtual camera to the coordinates of the grid center is obtained. Calculate the inner product between the surface normal vector and the line-of-sight direction vector, and determine the spatial geometric angle based on the inner product value; When the inner product value is greater than zero, it is determined that the surface normal vector and the viewing direction vector form an acute angle, confirming that the corresponding texture block is facing away from the current virtual camera. At the same time, it is determined that the texture block is located behind the target 3D model, and the identification information of the corresponding texture block is removed from the first block set to obtain the second block set.

8. The method according to claim 1, characterized in that, The step of detecting the target 3D model within the visible range in a pre-established hierarchical bounding box based on the viewpoint parameters and frustum parameters of the current virtual camera in the video twin engine includes: Using the viewpoint parameters of the current virtual camera as the vertex of the view frustum, and combining the clipping parameters and field of view parameters in the view frustum parameters, a view frustum space enclosed by multiple clipping planes is constructed in the virtual space. The view frustum is defined as the visible range, the spatial boundary of the visible range is extracted, and the spatial boundary is used to perform intersection detection with the pre-established hierarchical bounding box. Hierarchical bounding boxes that spatially overlap with the visible range are selected as target bounding boxes, and the 3D model enclosed within the target bounding box is determined as the target 3D model.

9. The method according to claim 2, characterized in that, After the continuous data stream is directly transferred from the main memory to the continuous video memory block via the asynchronous transmission channel using zero copy, the method further includes: When the hardware pipeline of the graphics processing unit receives a rendering instruction for the target 3D model, it calls the rendering shader to perform texture sampling operation on the ready texture data in the contiguous video memory block according to the vertex texture coordinates of the target 3D model. After the texture sampling operation is completed and the current frame is submitted, the exclusive lock state of the contiguous video memory block is released and the contiguous video memory block is marked as available. Cancel the global starting address of the contiguous video memory block, release the physical address concatenation lock of each memory unit, and disassemble the contiguous video memory block back into independent large-capacity page blocks and standard page blocks for cyclic use in the next transmission cycle.

10. A zero-copy streaming system for ultra-large textures based on asynchronous DMA and PBO, characterized in that, include: The detection module is used to detect the target 3D model within the visible range in a pre-established hierarchical bounding box based on the viewpoint parameters and frustum parameters of the current virtual camera in the video twin engine, and to determine the identification information of all texture blocks associated with the target 3D model to obtain the first block set. The culling module is used to remove the identification information of texture blocks located on the back of the target 3D model from the first block set based on the spatial geometric angle between the surface normal vector corresponding to each texture block on the surface of the target 3D model and the viewing direction of the current virtual camera, so as to obtain the second block set. The determination module is used to determine the target resolution to be loaded for each texture block in the second block set based on the spatial distance between the viewpoint corresponding to the viewpoint parameter and each texture block, and to perform downgrade processing on the target resolution of texture blocks whose spatial geometric angle is within a preset angle range. The fusion module is used to encapsulate all texture slices in the second slice set into a continuous data stream in the main memory, and combine the target resolution in the graphics processing unit video memory, and use pixel buffer objects (PBOs) to construct a continuous video memory block to carry the continuous data stream. A module is established to create an asynchronous transmission channel, through which the continuous data stream is directly transferred from the main memory to the continuous video memory block with zero copy.

Images