A method and apparatus for rendering non-transparent point cloud and mesh based on pixel sampling

Abstract

The application discloses a non-transparent point cloud and grid rendering method and device based on pixel sampling, and relates to the technical field of rendering. The method comprises the following steps: obtaining non-transparent point cloud data or grid data to be rendered; converting vertex coordinates corresponding to the point cloud data or the grid data into screen coordinates, and discretizing the screen coordinates into a plurality of pixel grids with an actual screen size, so that the vertex falls into the corresponding pixel grid; wherein one pixel grid corresponds to one pixel point of the screen; determining the color of each pixel grid according to the relationship between the vertex corresponding to the point cloud data or the grid data and each pixel grid and the distance relationship between the point cloud or the grid and a viewpoint, and then determining the color of the corresponding pixel point, so as to finally obtain a corresponding color texture map; and transmitting the obtained color texture map to a rendering device to complete the display of the texture by the rendering device. The application utilizes the pixel grid and depth comparison to eliminate data that is not finally displayed, reduces the rendering amount, and improves the rendering efficiency.

Description

Technical Field

[0001] This invention relates to the field of rendering technology, and specifically to a rendering method, apparatus, rendering device, computer device, and computer-readable storage medium based on pixel sampling of non-transparent point clouds and meshes. Background Technology

[0002] The main workflow of current mainstream rendering pipelines (such as OpenGL) is: vertex shader - primitive assembly - tessellation shader - geometry shader - rasterization - fragment shader - blending and testing, such as... Figure 1 As shown. Rasterization is the process of converting geometric data into pixels through a series of transformations, which are then displayed on the display device. Fragments are candidate pixels; only those that pass subsequent clipping, stencil, and depth tests will become the final displayed pixels, while fragments that fail the tests will be discarded.

[0003] As shown above, the traditional fixed-function rendering pipeline produces fragment shaders after rasterization. Only after passing subsequent testing will a fragment become the final displayed pixels. Many fragments are discarded during the testing phase, wasting the computational resources allocated to these discarded fragments. Furthermore, the traditional rendering pipeline linearly traverses all data to be rendered, meaning that rendering time increases with the total number of pixels to be rendered. The limit of screen display is the actual pixel size of the screen. Summary of the Invention

[0004] The purpose of this invention is to provide a rendering method and apparatus for non-transparent point clouds and meshes based on pixel sampling, so as to at least solve the problem that the screen display range is limited, while the traditional rendering pipeline linearly traverses all the data to be rendered, resulting in long rendering time.

[0005] The first aspect of the present invention provides a rendering method for non-transparent point clouds and meshes based on pixel sampling, the method comprising: Obtain the non-transparent point cloud data or mesh data to be rendered; The vertex coordinates corresponding to point cloud data or mesh data are converted into screen coordinates, and the screen coordinates are discretized into several pixel grids of the actual screen size, so that the vertex falls into the corresponding pixel grid; where one pixel grid corresponds to one pixel on the screen; Based on the relationship between the vertices and each pixel grid corresponding to the point cloud data or mesh data, as well as the distance relationship between the point cloud or mesh and the viewpoint, the color of each pixel grid is determined, and then the color of the corresponding pixel is determined, finally obtaining the corresponding color texture map; The obtained color texture map is transmitted to the rendering device, which then displays the texture.

[0006] In some embodiments, the color of each pixel is determined based on the relationship between the vertices corresponding to the point cloud data and each pixel grid, as well as the distance relationship between the point cloud and the viewpoint, including: If multiple vertices fall within a single pixel, only the vertex closest to the viewpoint is retained, and the color of that vertex is determined, thereby determining the color of that pixel.

[0007] In some embodiments, the color of each pixel is determined based on the relationship between the vertices corresponding to the grid data and each pixel grid, as well as the distance relationship between the grid and the viewpoint, including: The grid is rasterized to determine the pixel grids covered by the grid and the color of the grid, thereby determining the color of the corresponding pixel grids and ultimately the color of the corresponding pixel. If a pixel is covered by multiple grids, the grid closest to the viewpoint is identified, and its color is used as the color of the corresponding pixel.

[0008] In some of these embodiments, the color of the vertices or the color of the mesh are determined based on a lighting model.

[0009] In some embodiments, acquiring non-transparent point cloud data or mesh data to be rendered includes: The non-transparent point cloud data or mesh data to be rendered in the physical space is stored in a tree structure. Each leaf node of the tree structure represents a cube space of a fixed size at a different location. Determine the bounding box of the current viewport based on the viewpoint; Determine the 3D coordinates of the eight vertices of the cube space of each leaf node and compare them with the bounding box of the current viewport to remove leaf nodes that are outside the viewport and do not intersect with the viewport, while retaining leaf nodes that are within the viewport and leaf nodes that intersect with the viewport.

[0010] In some embodiments, acquiring the non-transparent point cloud data or mesh data to be rendered further includes: The point cloud data or mesh data of the leaf nodes that intersect with the viewport are compared with the bounding box of the current viewport to remove point cloud data or mesh data outside the viewport, retain the point cloud data or mesh data within the viewport, and together with the point cloud data or mesh data of the leaf nodes within the viewport, form the point cloud data or mesh data to be rendered.

[0011] In some of these embodiments, all cube spaces are the same size.

[0012] In some embodiments, when storing point cloud data or mesh data to be rendered in physical space into a tree structure, data compression is performed on the point cloud data or mesh data to be rendered using a custom data structure; wherein, the custom data structure includes changing the data storage type of the normal vector.

[0013] In some embodiments, the vertex coordinates corresponding to point cloud data or mesh data are converted into screen coordinates based on the MVP matrix.

[0014] In some of these embodiments, the number of pixel grids is determined based on the screen resolution.

[0015] In some embodiments, the method further includes: The screen coordinates are discretized into a custom number of pixel grids, the number of pixel grids in both the length and width directions being in a first ratio to the screen resolution, and the first ratio being less than 1; After obtaining the corresponding color texture map, the color texture map is interpolated to restore it to the screen size.

[0016] In some embodiments, interpolation includes linear interpolation.

[0017] According to a second aspect of the present invention, a rendering apparatus based on pixel sampling of non-transparent point clouds and meshes is provided, the apparatus comprising: an acquisition unit, a transformation unit, a texture unit, and a rendering unit; The acquisition unit is used to acquire non-transparent point cloud data or mesh data to be rendered; The conversion unit is used to convert the vertex coordinates corresponding to point cloud data or mesh data into screen coordinates, and discretize the screen coordinates into several pixel grids of the actual screen size, so that the vertex falls into the corresponding pixel grid; wherein, one pixel grid corresponds to one pixel point on the screen; Texture units are used to determine the color of each pixel cell based on the relationship between the vertices and each pixel cell in the point cloud data or mesh data, as well as the distance relationship between the point cloud or mesh and the viewpoint, thereby determining the color of the corresponding pixel and finally obtaining the corresponding color texture map. The rendering unit is used to transmit the obtained color texture map to the rendering device, which then displays the texture.

[0018] According to a third aspect of the present invention, a rendering apparatus is provided, comprising the rendering apparatus for non-transparent point clouds and meshes based on pixel sampling as described in the second aspect.

[0019] According to a fourth aspect of the present invention, a computer device is provided, comprising: a processor and a memory, the memory storing a program or instructions executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the rendering method for non-transparent point clouds and meshes based on pixel sampling as described in any one of the first aspects.

[0020] According to a fifth aspect of the invention, a readable storage medium is provided having a program or instructions stored thereon, which, when executed by a processor, implement the steps of the rendering method for non-transparent point clouds and meshes based on pixel sampling as described in any one of the first aspects.

[0021] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: This invention samples data based on screen pixels. By converting the vertex coordinates corresponding to non-transparent point cloud data or mesh data into screen coordinates and discretizing them into pixel grids the size of the actual screen, the non-transparent point cloud data or mesh data can be selected using these pixel grids, significantly reducing the total amount of rendering and improving rendering efficiency. In point cloud mode, by converting vertex coordinates to screen coordinates and discretizing them into pixel grids, the nearest vertex is selected using depth judgment, greatly reducing the total amount of rendering and processing only the vertex data to be displayed, thereby improving rendering efficiency. In mesh mode, after a mesh vertex falls into a pixel grid, its color is determined through rasterization and depth judgment, effectively reducing unnecessary data processing and also improving rendering efficiency.

[0022] Furthermore, this invention further improves rendering efficiency through a two-stage culling process. First, a coarse culling process is performed, storing the data to be rendered in a tree structure. Based on the relationship between the vertices of the leaf node cube space and the viewport bounding box, leaf nodes that do not intersect with the viewport are culled, significantly reducing subsequent computational load. Next, a fine culling process is performed, comparing the data within the selected leaf nodes again with the viewport bounding box to further exclude data outside the viewport, further reducing the amount of data involved in rendering calculations and optimizing the entire rendering process.

[0023] Furthermore, the screen coordinates can be discretized into a custom number of pixel grids. The number of pixel grids in both the length and width directions is proportional to the screen resolution, and this proportion is less than 1. This allows sampling at a size smaller than the screen resolution to obtain the corresponding color texture map. The color texture map is then interpolated to restore the screen size. This method of sampling at a size smaller than the screen resolution and stretching it for display effectively eliminates the strong correlation between rendering efficiency and screen resolution. By discretizing the screen coordinates into a smaller, custom grid for sampling, the computational resource consumption for generating textures from point clouds / meshes is reduced, decreasing the amount of data processed. Finally, linear interpolation restores the texture map to the screen size, making rendering efficiency no longer constrained by screen resolution. Even on high-resolution screens, rendering speed can be improved while maintaining display quality. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the main process of the mainstream rendering pipeline in the background technology of this application; Figure 2 A flowchart illustrating a rendering method for non-transparent point clouds and meshes based on pixel sampling, provided in an embodiment of this application; Figure 3 A schematic diagram of a tree structure provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the depth determination process in a point cloud pattern, as provided in an embodiment of this application. Figure 5 This is a schematic diagram illustrating the removal of occluded data outside the field of view and data with depth in the background, as provided in an embodiment of this application. Figure 6 This is a schematic diagram of a rasterization process provided in an embodiment of this application; Figure 7 This is a schematic diagram of texture map interpolation provided in an embodiment of this application; Figure 8 A framework diagram of a rendering apparatus for non-transparent point clouds and meshes based on pixel sampling, provided for embodiments of this application; Figure 9 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this invention.

[0026] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0027] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0028] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0029] This application provides a rendering method and apparatus for non-transparent point clouds and meshes based on pixel sampling. From the perspective of screen pixels, it reduces the amount of rendering computation and improves rendering efficiency from the following four aspects: (1) Construct a tree structure based on physical space. Each leaf node represents a cube space of fixed size at different locations. Perform coarse filtering based on the relationship between the eight points of the cube and the bounding box of the view, and eliminate the linear relationship between efficiency and total data volume.

[0030] (2) Based on screen pixels, sampling is performed. For point cloud data, only the vertex data that falls on the pixel grid and is closest to the screen is retained. This is equivalent to selecting only the closest vertex for each grid, which greatly reduces the total amount of rendering and improves the rendering efficiency. For mesh data, only the mesh closest to the screen in the corresponding pixel grid is retained, and a large amount of data that will not be displayed in the end is removed.

[0031] (3) Custom sampling resolution: You can select a size smaller than the actual screen resolution for sampling, stretch the rendered image and display it on the screen, thus eliminating the strong correlation between efficiency and screen resolution.

[0032] (4) Custom data structure makes data storage more flexible and allows for compression of variables that do not require precision, reducing the use of video memory.

[0033] Specifically, this application reduces the amount of data to be processed by pre-filtering using a tree structure. Pre-filtering removes a large amount of data that will not be ultimately displayed, thus reducing the number of vertices to be traversed and rendered. Furthermore, this application performs uniform sampling based on the screen's pixel grid, retaining only the data that needs to be rendered, further reducing the amount of data and thus improving rendering efficiency.

[0034] First, this application provides a rendering method for non-transparent point clouds and meshes based on pixel sampling, which is applicable to various rendering devices. Figure 2 A flowchart illustrating a rendering method for non-transparent point clouds and meshes based on pixel sampling, provided in an embodiment of this application, is shown below. Figure 2 As shown, the method includes the following steps: S101. Obtain the non-transparent point cloud data or mesh data to be rendered; S102. Convert the vertex coordinates corresponding to the point cloud data or mesh data into screen coordinates, and discretize the screen coordinates into several pixel grids of the actual screen size, so that the vertices fall into the corresponding pixel grids; wherein, one pixel grid corresponds to one pixel point on the screen; S103. Based on the relationship between the vertices and each pixel grid corresponding to the point cloud data or mesh data, and the distance relationship between the point cloud or mesh and the viewpoint, determine the color of each pixel grid, and then determine the color of the corresponding pixel point, and finally obtain the corresponding color texture map. S104. The obtained color texture map is transmitted to the rendering device, which then displays the texture.

[0035] Through the above steps, this application converts the vertex coordinates corresponding to non-transparent point cloud data or mesh data into screen coordinates and discretizes them into pixel grids of actual screen size, so that one pixel grid corresponds to one pixel on the screen. This pixel grid is then used to select non-transparent point cloud data or mesh data, eliminating a large amount of data that will not be ultimately displayed, significantly reducing the total rendering load and improving rendering efficiency. In point cloud mode, by converting vertex coordinates to screen coordinates and discretizing them into pixel grids, and using depth judgment to select the nearest vertex, the total rendering load is greatly reduced, processing only the vertex data to be displayed, thereby improving rendering efficiency. In mesh mode, after mesh vertices fall into pixel grids, they undergo rasterization processing and depth judgment to determine their color, effectively reducing unnecessary data processing and also improving rendering efficiency.

[0036] In some embodiments, step S101 involves acquiring the non-transparent point cloud data or mesh data to be rendered, including: The non-transparent point cloud data or mesh data to be rendered in the physical space is stored in a tree structure. Each leaf node of the tree structure represents a cube space of a fixed size at a different location. Determine the bounding box of the current viewport based on the viewpoint; Determine the 3D coordinates of the eight vertices of the cube space of each leaf node and compare them with the bounding box of the current viewport to remove leaf nodes that are outside the viewport and do not intersect with the viewport, while retaining leaf nodes that are within the viewport and leaf nodes that intersect with the viewport.

[0037] This step stores the point cloud / mesh data to be rendered in the physical space into a tree structure. Subsequent steps involve using the leaf nodes of the tree structure as units for the initial coarse culling. The main principle of constructing a tree structure based on physical space is to divide the entire physical space into multiple cubic spaces, such as... Figure 3 As shown, each cube space serves as a leaf node in the tree structure, and the data contained within the cube space is stored in the corresponding leaf node. The tree structure can have multiple levels, meaning the leaf nodes can be further divided; the specific number of levels in the tree structure can be determined based on the actual situation.

[0038] When storing point cloud or mesh data to be rendered in physical space into a tree structure, a custom data structure can be used to compress the data, reducing memory usage. For example, storing the coordinates and normal data of a point in 3D space typically looks like this: struct Point { float xyz[3]; / / 12 bytes float nor[3]; / / 12 bytes } To reduce the storage space required for normal data, the data storage type for normal vectors can be changed, as shown below: struct Point { float xyz[3]; / / 12 bytes unsigned char nor[3]; / / 3 bytes } This can reduce video memory usage.

[0039] In the first coarse culling step, a simple numerical comparison is made between the three-dimensional coordinates of the eight vertices of the leaf (i.e., the corresponding cubic space) and the bounding box of the current viewport. Leaves that do not intersect with the viewport are culled, which can greatly reduce the amount of computation in rendering traversal and eliminate the linear relationship between the time spent on data judgment and the total amount of data.

[0040] To further eliminate useless data and reduce computation, the 3D coordinates of the selected leaf nodes can be compared with the bounding box of the current viewport for fine-grained elimination. Point cloud data or mesh data outside the field of view is removed, while point cloud data or mesh data within the field of view is retained. This retained point cloud data or mesh data is then combined with the point cloud data or mesh data of the leaf nodes within the field of view to form the point cloud data or mesh data to be rendered, further reducing the total amount of subsequent computation.

[0041] In some embodiments, in step S102, the vertex coordinates corresponding to the point cloud data or mesh data are converted into screen coordinates according to the MVP matrix, and the screen coordinates are discretized into a number of pixel grids of the actual screen size, thereby mapping all vertices to pixel grids. The number of pixel grids can be determined according to the screen resolution. For example, if the screen resolution is 1920*1080, then the number of pixel grids is 1920*1080, and one pixel grid corresponds to one pixel on the screen.

[0042] The MVP matrix is ​​the most fundamental matrix transformation in computer graphics. It consists of three matrices: model, view, and projection, and is used to represent objects in three-dimensional space on a two-dimensional screen. Thus, all candidate vertices fall into their corresponding pixel grids.

[0043] In some embodiments, in step S103, it is necessary to determine the color of each pixel cell based on the relationship between the vertices and pixel cells corresponding to the point cloud data or mesh data, as well as the distance relationship between the point cloud or mesh and the viewpoint, thereby determining the color of the corresponding pixel. Different modes are required due to the differences between point cloud data and mesh data.

[0044] Point cloud mode: First, the vertex coordinates are converted to screen coordinates based on the MVP matrix, with the coordinate range being (-1, 1). The screen coordinates are then discretized into a pixel grid of the actual screen size (e.g., 1920*1080), so all candidate vertices will fall into their corresponding pixel grids. However, there may be cases where multiple candidate vertices fall into the same pixel grid. Therefore, depth judgment is introduced, such as... Figure 4 As shown, based on the distance between each candidate vertex and the viewpoint, the candidate vertex closest to the viewpoint is selected for each pixel. After calculating the color using the lighting model, the color of that pixel is stored, thus obtaining the corresponding texture map.

[0045] Mesh Mode: Vertex coordinates are converted to screen coordinates based on the MVP matrix, with the coordinate range being (-1, 1). The screen coordinates are then discretized into a pixel grid of the actual screen size (e.g., 1920*1080), ensuring that the three vertices of the mesh fall into their corresponding pixel grids. Rasterization is then performed, incorporating depth detection, such as... Figure 5 and Figure 6 As shown, the color of each triangular facet is calculated using the lighting model, and the nearest mesh color is stored for each pixel to obtain the corresponding texture map.

[0046] Finally, the obtained color texture map is transmitted to the rendering device, which then displays the texture.

[0047] Thus, this application completes the rendering of non-transparent point cloud data and mesh data based on pixel sampling. Specifically, it utilizes a tree structure to store the data to be rendered, performs coarse culling, reduces the computational load of traversing the entire rendering process, and minimizes the impact of the total data volume on the rendering process. Simultaneously, a custom data structure can be defined to compress data that does not require precision, consuming less video memory resources than traditional fixed rendering pipelines. Finally, pixel grid and depth comparisons are used to culle data that will not be ultimately displayed, reducing the rendering load and improving rendering efficiency.

[0048] This application embodiment also provides another rendering method for non-transparent point clouds and meshes based on pixel sampling. The principle and idea of ​​this embodiment are basically the same as those of the previous embodiment. The difference is that in this embodiment, the screen coordinates are discretized into a custom number of pixel grids. The number of pixel grids in the length and width directions is in a first ratio with the screen resolution, and the first ratio is less than 1.

[0049] For example, when encountering high-resolution screens such as 4K, to eliminate the correlation between rendering efficiency and screen resolution, the screen coordinates (-1, 1) can be discretized into a smaller, custom grid (960*540), not based on the actual screen size. Finally, the resulting texture map is linearly interpolated, such as... Figure 7As shown, the original screen size is restored. This way, the computational resources and time spent generating textures from point clouds / meshes are no longer affected by screen resolution, significantly improving rendering efficiency and allowing for combination with pixel sampling methods.

[0050] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0051] This application also provides a rendering apparatus for non-transparent point clouds and meshes based on pixel sampling. These apparatuses are used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0052] Figure 8 This is a structural block diagram of a rendering apparatus for non-transparent point clouds and meshes based on pixel sampling, according to an embodiment of this application. Figure 8 As shown, the device includes an acquisition unit 201, a conversion unit 202, a color unit 203, and a rendering unit 204; Acquisition unit 201 is used to acquire non-transparent point cloud data or mesh data to be rendered; The conversion unit 202 is used to convert the vertex coordinates corresponding to point cloud data or mesh data into screen coordinates, and discretize the screen coordinates into a number of pixel grids of the actual screen size, so that the vertex falls into the corresponding pixel grid; wherein, one pixel grid corresponds to one pixel point on the screen; Texture unit 203 is used to determine the color of each pixel cell based on the relationship between the vertices and each pixel cell corresponding to the point cloud data or mesh data and the distance relationship between the point cloud or mesh and the viewpoint, thereby determining the color of the corresponding pixel and finally obtaining the corresponding color texture map. The rendering unit 204 is used to transmit the obtained color texture map to the rendering device, which then displays the texture.

[0053] It should be noted that the aforementioned units can be either functional units or program units, and can be implemented in either software or hardware. For units implemented in hardware, the aforementioned units can reside in the same processor; or the aforementioned units can be located in different processors in any combination.

[0054] In addition, combined Figure 2 The pixel-sampling-based rendering method for non-transparent point clouds and meshes described in this application can be implemented by a computer device. Figure 9 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application. Figure 9 As shown, the device may include a processor 301 and a memory 302 storing computer program instructions.

[0055] Specifically, the processor 301 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0056] Memory 302 may include a large-capacity memory for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to a data processing device. In a particular embodiment, memory 302 is non-volatile memory. In a particular embodiment, memory 302 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0057] The memory 302 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 301.

[0058] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any of the pixel-sampling-based rendering methods for non-transparent point clouds and meshes in the above embodiments.

[0059] In some embodiments, the point cloud generation device may further include a communication interface 303 and a bus 300. Wherein, as... Figure 9 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 300 and complete communication with each other.

[0060] The communication interface 303 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of this application. The communication interface 303 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0061] Bus 300 includes hardware, software, or both, that couples the components of the point cloud generation device together. Bus 300 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 300 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 300 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnection.

[0062] The computer device can execute the pixel-sampling-based rendering method for non-transparent point clouds and meshes in this application embodiment based on the rendering device, thereby achieving a combination of... Figure 2 The description describes a rendering method for non-transparent point clouds and meshes based on pixel sampling.

[0063] Furthermore, in conjunction with the pixel-sampling-based rendering method for opaque point clouds and meshes described in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the pixel-sampling-based rendering methods for opaque point clouds and meshes described in the above embodiments.

[0064] It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. In addition, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0065] It will be readily understood by those skilled in the art that the above-described embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A rendering method for non-transparent point clouds and meshes based on pixel sampling, characterized in that, The method includes: Obtain the non-transparent point cloud data or mesh data to be rendered; The vertex coordinates corresponding to point cloud data or mesh data are converted into screen coordinates, and the screen coordinates are discretized into several pixel grids of the actual screen size, so that the vertex falls into the corresponding pixel grid; where one pixel grid corresponds to one pixel on the screen; Based on the relationship between the vertices and each pixel grid corresponding to the point cloud data or mesh data, as well as the distance relationship between the point cloud or mesh and the viewpoint, the color of each pixel grid is determined, and then the color of the corresponding pixel is determined, finally obtaining the corresponding color texture map; The obtained color texture map is transmitted to the rendering device, which then displays the texture.

2. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 1, characterized in that, Based on the relationship between vertices and individual pixels in the point cloud data, and the distance between the point cloud and the viewpoint, the color of each pixel is determined, including: If multiple vertices fall within a single pixel, only the vertex closest to the viewpoint is retained, and the color of that vertex is determined, thereby determining the color of that pixel.

3. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 1, characterized in that, Based on the relationship between the vertices and individual pixels in the grid data, and the distance between the grid and the viewpoint, the color of each pixel is determined, including: The grid is rasterized to determine the pixel grids covered by the grid and the color of the grid, thereby determining the color of the corresponding pixel grids and ultimately the color of the corresponding pixel. If a pixel is covered by multiple grids, the grid closest to the viewpoint is identified, and its color is used as the color of the corresponding pixel.

4. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 2 or 3, characterized in that, The color of the vertex or the color of the mesh are determined based on the lighting model.

5. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 1, characterized in that, Obtain the non-transparent point cloud data or mesh data to be rendered, including: The non-transparent point cloud data or mesh data to be rendered in the physical space is stored in a tree structure. Each leaf node of the tree structure represents a cube space of a fixed size at a different location. Determine the bounding box of the current viewport based on the viewpoint; Determine the 3D coordinates of the eight vertices of the cube space of each leaf node and compare them with the bounding box of the current viewport to remove leaf nodes that are outside the viewport and do not intersect with the viewport, while retaining leaf nodes that are within the viewport and leaf nodes that intersect with the viewport.

6. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 5, characterized in that, Obtaining non-transparent point cloud data or mesh data to be rendered also includes: The point cloud data or mesh data of the leaf nodes that intersect with the viewport are compared with the bounding box of the current viewport to remove point cloud data or mesh data outside the viewport, retain the point cloud data or mesh data within the viewport, and together with the point cloud data or mesh data of the leaf nodes within the viewport, form the point cloud data or mesh data to be rendered.

7. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 5 or 6, characterized in that, All cube spaces are the same size.

8. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 5 or 6, characterized in that, When storing point cloud data or mesh data to be rendered in physical space into a tree structure, data compression is performed on the point cloud data or mesh data to be rendered using a custom data structure; the custom data structure includes changing the data storage type of the normal vector.

9. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 1, characterized in that, Convert the vertex coordinates corresponding to point cloud data or mesh data into screen coordinates based on the MVP matrix.

10. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 1, characterized in that, The number of pixels is determined based on the screen resolution.

11. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 1, characterized in that, The method also includes: The screen coordinates are discretized into a custom number of pixel grids, the number of pixel grids in both the length and width directions being in a first ratio to the screen resolution, and the first ratio being less than 1; After obtaining the corresponding color texture map, the color texture map is interpolated to restore it to the screen size.

12. The rendering method for non-transparent point clouds and meshes based on pixel sampling according to claim 11, characterized in that, Interpolation includes linear interpolation.

13. A rendering apparatus for non-transparent point clouds and meshes based on pixel sampling, characterized in that, The device includes an acquisition unit, a conversion unit, a color unit, and a rendering unit; The acquisition unit is used to acquire non-transparent point cloud data or mesh data to be rendered; The conversion unit is used to convert the vertex coordinates corresponding to point cloud data or mesh data into screen coordinates, and discretize the screen coordinates into several pixel grids of the actual screen size, so that the vertex falls into the corresponding pixel grid; wherein, one pixel grid corresponds to one pixel point on the screen; Texture units are used to determine the color of each pixel cell based on the relationship between the vertices and each pixel cell in the point cloud data or mesh data, as well as the distance relationship between the point cloud or mesh and the viewpoint, thereby determining the color of the corresponding pixel and finally obtaining the corresponding color texture map. The rendering unit is used to transmit the obtained color texture map to the rendering device, which then displays the texture.

14. A rendering device, characterized in that, The rendering device includes the rendering apparatus for non-transparent point clouds and meshes based on pixel sampling as described in claim 13.

15. A computer device, characterized in that, include: A processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, implement the steps of the rendering method for non-transparent point clouds and meshes based on pixel sampling as described in any one of claims 1 to 12.

16. A readable storage medium, characterized in that, It stores a program or instructions that, when executed by a processor, implement the steps of the rendering method for non-transparent point clouds and meshes based on pixel sampling as described in any one of claims 1 to 12.

Images