Point cloud rendering method, apparatus and device

Abstract

The embodiments of the present disclosure disclose a point cloud rendering method, device and equipment, wherein the method comprises: obtaining a first display area of a point cloud model in a target frame image, a second display area of a contour structure model in the target frame image and a third display area of a separation light band in the target frame image; based on the first display area and the second display area, rendering the point cloud model and the contour structure model to obtain a first image, and storing the first image in a first off-screen buffer; based on the third display area, rendering the contour structure model and the separation light band to obtain a second image, and storing the second image in a second off-screen buffer; superimposing and rendering the first image and the second image to obtain the target frame image. The embodiments of the present disclosure can realize the switching display of the point cloud model and the contour structure model in the target frame image, and can not only clearly and accurately display the contour structure of the three-dimensional space.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, and more particularly to a point cloud rendering method, apparatus, and device. Background Technology

[0002] Laser point cloud refers to a set of spatial points collected by a 3D lidar device, which can be used for 3D spatial reconstruction, positioning, etc.

[0003] In related technologies, when rendering 3D space based on point cloud data, the point cloud data often contains noise, missing or erroneous data, which can easily lead to inaccurate rendering results and make it impossible to clearly and accurately represent the structure of 3D space, resulting in poor structured rendering effect when rendering 3D models. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this disclosure provide a point cloud rendering method, apparatus, and device.

[0005] According to one aspect of the embodiments of this disclosure, a point cloud rendering method is provided, comprising:

[0006] The first display area of ​​the point cloud model in the target frame image, the second display area of ​​the contour structure model in the target frame image, and the third display area of ​​the separating light strip in the target frame image are obtained. The separating light strip is used to separate the point cloud model and the contour structure model in the target frame image.

[0007] Based on the first display area and the second display area, the point cloud model and the contour structure model are rendered to obtain a first image, and the first image is stored in the first off-screen buffer.

[0008] Based on the third display area, the contour structure model and the dividing light band are rendered to obtain a second image, and the second image is stored in the second off-screen buffer.

[0009] The first image and the second image are overlaid and rendered to obtain the target frame image.

[0010] In some optional implementations, the steps for constructing the contour structure model include:

[0011] The top view outer contour points of the point cloud to be processed are sorted according to clock sequence to obtain a set of ordered contour points;

[0012] Based on the height value of the bounding box corresponding to the point cloud to be processed in the vertical direction and the set of ordered contour points, an initial structural model is generated, which is a single-layer structured model.

[0013] Texture mapping is performed on the initial structural model to obtain the contour structural model.

[0014] In some alternative implementations, in response to applying a gradient material map to the initial structural model, the process of applying a texture map to the initial structural model to obtain the contour structural model includes:

[0015] Based on the position and orientation information of the virtual camera, the angle between the line of sight and the plane normal vector of each plane of the initial structural model is determined;

[0016] Based on the included angle, the texture surface of each plane of the initial structural model is determined, and the texture surface is at least one of the following: the inner surface of the model and the outer surface of the model;

[0017] In response to the fact that the texture surface is the outer surface of the model, the gradient material texture is mapped onto the texture surface of the plane of the initial structural model;

[0018] In response to the fact that the texture surface is the inner surface of the model, the texture horizontal coordinate of the gradient material texture is mirrored and flipped and then mapped onto the texture surface of the plane of the initial structure model.

[0019] In some alternative implementations, the step of rendering the point cloud model and the contour structure model to obtain the first image based on the first display area and the second display area includes:

[0020] The first shader renders each pixel of the point cloud model within the coverage area of ​​the first display area to obtain the third image; and the second shader renders each pixel of the contour structure model within the coverage area of ​​the second display area to obtain the fourth image.

[0021] The third image and the fourth image are superimposed to obtain the first image.

[0022] In some alternative implementations, the step of rendering the contour structure model and the segmented light bands to obtain a second image based on the third display area includes:

[0023] In the third shader, each pixel within the area covered by the dividing light band on the contour structure model is rendered with a set color to obtain the fifth image;

[0024] Gaussian blurring is applied to each pixel within the coverage area of ​​the dividing light band in the fifth image to obtain the second image.

[0025] In some alternative implementations, the step of performing Gaussian blur processing on each pixel within the coverage area of ​​the separated light band in the fifth image to obtain the second image includes:

[0026] A one-dimensional Gaussian blur is applied to each pixel within the coverage area of ​​the dividing light band in the fifth image in the horizontal direction to obtain the sixth image;

[0027] Perform one-dimensional Gaussian blurring on each pixel within the coverage area of ​​the dividing light band in the fifth image in the vertical direction to obtain the seventh image;

[0028] The sixth image and the seventh image are superimposed to obtain the second image.

[0029] According to another aspect of the embodiments of this disclosure, a point cloud rendering apparatus is provided, comprising:

[0030] The acquisition module is used to acquire the first display area of ​​the point cloud model in the target frame image, the second display area of ​​the contour structure model in the target frame image, and the third display area of ​​the separating light strip in the target frame image, wherein the separating light strip is used to separate the point cloud model and the contour structure model in the target frame image.

[0031] The first rendering module is used to render the point cloud model and the contour structure model based on the first display area and the second display area to obtain a first image, and store the first image in a first off-screen buffer.

[0032] The second rendering module is used to render the contour structure model and the dividing light band based on the third display area to obtain a second image, and to store the second image in the second off-screen buffer.

[0033] The overlay rendering module is used to overlay the first image and the second image to obtain the target frame image.

[0034] In some alternative implementations, the apparatus also includes:

[0035] The point sorting module is used to sort the outer contour points of the top view of the point cloud to be processed according to the clock sequence, so as to obtain a set of ordered contour points.

[0036] The model generation module is used to generate an initial structural model based on the height value of the bounding box corresponding to the point cloud to be processed in the vertical direction and the set of ordered contour points. The initial structural model is a single-layer structured model.

[0037] The texture mapping module is used to apply texture mapping to the initial structural model to obtain the contour structural model.

[0038] In some alternative implementations, the texture mapping module includes:

[0039] The first determining submodule is used to determine the angle between the line of sight and the plane normal vector of each plane of the initial structure model in response to applying a gradient material texture to the initial structure model, based on the position and orientation information of the virtual camera.

[0040] The second determining submodule is used to determine the texture surface of each plane of the initial structural model based on the included angle, wherein the texture surface is at least one of the following: the inner surface of the model and the outer surface of the model;

[0041] The first mapping submodule is used to map the gradient material texture onto the texture surface of the plane of the initial structural model in response to the texture surface being the outer surface of the model.

[0042] The second mapping submodule is used to mirror and flip the texture horizontal coordinate of the gradient material texture and then map it onto the texture surface of the plane of the initial structure model in response to the texture surface being the inner surface of the model.

[0043] In some alternative implementations, the first rendering module includes:

[0044] The first rendering submodule is used to render each pixel of the point cloud model within the coverage area of ​​the first display area in the first shader to obtain the third image; and to render each pixel of the contour structure model within the coverage area of ​​the second display area in the second shader to obtain the fourth image.

[0045] The overlay submodule is used to overlay the third image and the fourth image to obtain the first image.

[0046] In some alternative implementations, the second rendering module includes:

[0047] The second rendering submodule is used in the third shader to render each pixel within the coverage area of ​​the dividing light band on the contour structure model as a set color to obtain the fifth image;

[0048] The blurring submodule is used to perform Gaussian blurring on each pixel within the coverage area of ​​the dividing light band in the fifth image to obtain the second image.

[0049] In some alternative implementations, the fuzzing submodule includes:

[0050] The first blurring unit is used to perform one-dimensional Gaussian blurring on each pixel within the coverage area of ​​the dividing light band in the fifth image in the horizontal direction to obtain the sixth image;

[0051] The second blurring unit is used to perform one-dimensional Gaussian blurring on each pixel within the coverage area of ​​the dividing light band in the fifth image in the vertical direction to obtain the seventh image;

[0052] The overlay unit is used to overlay the sixth image and the seventh image to obtain the second image.

[0053] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing computer program instructions, which, when executed, implement the above-described point cloud rendering method.

[0054] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0055] Memory, used to store computer program products;

[0056] A processor is configured to execute a computer program product stored in the memory, and when the computer program product is executed, to implement the above-described point cloud rendering method.

[0057] According to another aspect of the present disclosure, a computer program product is provided, including computer program instructions that, when executed by a processor, implement the above-described point cloud rendering method.

[0058] Based on the above embodiments of this disclosure, when rendering a scene, if the user triggers a switching rendering operation, a first display area of ​​the point cloud model in the target frame image, a second display area of ​​the contour structure model in the target frame image, and a third display area of ​​the separating light strip in the target frame image can be obtained. Based on the first and second display areas, the point cloud model and the contour structure model are rendered to obtain a first image, which is then stored in a first off-screen buffer. Based on the third display area, the contour structure model and the separating light strip are rendered to obtain a second image, which is then stored in a second off-screen buffer. The first and second images are then superimposed to obtain the target frame image. Thus, the target frame image rendered by the technical solution of this disclosure can realize the switching display of the point cloud model and the contour structure model, which can not only clearly and accurately display the contour structure in three-dimensional space, but also achieve a good visual switching effect between the point cloud model and the contour structure model by using a separating light strip to separate them.

[0059] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0060] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0061] This disclosure will become clearer with reference to the accompanying drawings and the following detailed description, wherein:

[0062] Figure 1 A flowchart illustrating one embodiment of the point cloud rendering method disclosed herein;

[0063] Figure 2 This is a schematic diagram illustrating the effect of the point cloud rendering method disclosed herein;

[0064] Figure 3 This is a flowchart illustrating the point cloud rendering method disclosed herein for generating a contour structure model.

[0065] Figure 4 This is a flowchart of the point cloud rendering method disclosed herein, which describes the application of gradient mapping to a contour structure model.

[0066] Figure 5 This is a flowchart illustrating the rendering of the first image in the point cloud rendering method of this disclosure;

[0067] Figure 6 This is a flowchart illustrating the rendering of the second image in the point cloud rendering method of this disclosure;

[0068] Figure 7 This is a schematic diagram of the outer contour points of the point cloud disclosed herein;

[0069] Figure 8 This is a schematic diagram of the structure of one embodiment of the point cloud rendering apparatus disclosed herein;

[0070] Figure 9 This is a schematic diagram of the structure of yet another embodiment of the point cloud rendering apparatus disclosed herein;

[0071] Figure 10 This is a structural diagram of an electronic device provided as an illustrative embodiment of the present disclosure. Detailed Implementation

[0072] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0073] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0074] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary functional module order among them.

[0075] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0076] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0077] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0078] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0079] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0080] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0081] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0082] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0083] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0084] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, functional modules, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are performed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0085] This disclosure outlines

[0086] In the process of realizing this disclosure, the inventors discovered through research that point cloud data often contains noise, missing or erroneous data, which can easily lead to inaccurate rendering results and make it impossible to clearly and accurately represent the structure of three-dimensional space, resulting in poor structured rendering effect when rendering three-dimensional models.

[0087] The technical solution disclosed herein obtains point cloud models and contour structure models based on point cloud data, and renders the point cloud models and contour structure models simultaneously in the target frame image during rendering. This not only clearly and accurately displays the contour structure in three-dimensional space, but also achieves a good visual switching effect between the point cloud models and contour structure models by separating the point cloud models and contour structure models with a separating light strip.

[0088] Exemplary methods

[0089] Figure 1 This is a flowchart illustrating an embodiment of the point cloud rendering method disclosed herein; this point cloud rendering method can be applied to electronic devices (such as computer systems, servers, etc.). Figure 1 As shown, the point cloud rendering method includes the following steps 101 to 104. Each step is explained below.

[0090] In step 101, the first display area of ​​the point cloud model in the target frame image, the second display area of ​​the contour structure model in the target frame image, and the third display area of ​​the separating light strip in the target frame image are obtained. The separating light strip is used to separate the point cloud model and the contour structure model in the target frame image.

[0091] The point cloud model and the contour structure model are both derived from the same point cloud data. The point cloud model is a set of 3D points with color information, obtained by storing these points in a common 3D point cloud format. The contour structure model is a model formed by connecting a set of contour points in the point cloud to define the contour. The implementation method for constructing the contour structure model using this set of contour points can be found in [link to relevant documentation]. Figure 3 The illustrated embodiment will not be described in detail here. The separating light band is a region of a certain height extracted from the contour structure model, and a light emission effect is implemented for this region. It is used to separate the point cloud model and the contour structure model when rendering the target frame image.

[0092] In this embodiment, the point cloud data is the color point cloud data obtained by registering and fusing the point cloud data collected by the LiDAR on the target space and the image data collected by the camera on the target space.

[0093] In some alternative implementations, preprocessing operations, including point cloud thinning, homogenization, and noise reduction, can be performed on the point cloud acquired by the lidar. Preprocessing can reduce the number of point clouds and improve processing efficiency.

[0094] In this embodiment, when the electronic device receives a user-triggered display switching operation, it will simultaneously display the point cloud model and the contour structure model in the rendered target frame image, such as... Figure 2 As shown, the display shows a point cloud model in the first display area (the area below the white light band) of the rendered target frame image, a contour structure model in the second display area (the area above the white light band), and a separating light band in the third display area (the area indicated by the white light band). As the separating light band moves from top to bottom, the area of ​​the contour structure model displayed on the screen becomes larger and larger, while the area of ​​the point cloud model becomes smaller and smaller, achieving a transitional display effect from the point cloud model to the contour structure model. It is understood that the point cloud model can also be displayed in the upper half of the target frame image, and the contour structure model in the lower half, and the separating light band can also move from bottom to top; this disclosure does not limit this.

[0095] It is understandable that when rendering point cloud data frame by frame, the sizes of the first and second display areas in each frame of the image change, while the size of the third display area remains constant, but its position in the image changes frame by frame.

[0096] In step 102, a first image is obtained by rendering a point cloud model and a contour structure model based on the first display area and the second display area, and the first image is stored in the first off-screen buffer.

[0097] The first off-screen buffer is a temporary storage space used to store data that does not need to be directly displayed on the screen during graphics rendering. In this disclosure, the first image needs to undergo further processing in subsequent operations rather than being directly displayed on the screen, therefore it needs to be stored in the first off-screen buffer.

[0098] In this embodiment, the point cloud model and the contour structure model can be rendered in different shaders. Since the point cloud model is only displayed in the first display area and the contour structure model is displayed in the second display area in the target frame image, that is, the point cloud model and the contour structure model only render the pixels of a part of the area, while the pixels of other areas are hidden using the discard function and not rendered.

[0099] After rendering the point cloud model and the contour structure model in different shaders, they can be overlaid to obtain the first image.

[0100] In step 103, based on the third display area, a contour structure model and a segmented light band are rendered to obtain a second image, which is then stored in a second off-screen buffer.

[0101] The second off-screen buffer has the same function as the first off-screen buffer, used to store data that does not need to be directly displayed on the screen during graphics rendering. In this embodiment, the second image needs to be overlaid with the first image obtained in step 102 in subsequent operations, rather than being directly displayed on the screen, therefore it needs to be stored in the second off-screen buffer.

[0102] In this embodiment, a strip-shaped region of a height corresponding to the third display area can be selected on the contour structure model as a separating light strip. The strip-shaped region of that height on the contour structure model is rendered with a set color in the shader, and the pixels of other areas on the contour structure model are hidden using the discard function and not rendered, thus obtaining the second image.

[0103] In other implementations, a strip-shaped region of a height corresponding to the third display area is selected on the contour structure model as a separating light strip. After rendering the strip-shaped region of that height on the contour structure model with a set color in the shader, a glowing effect can also be achieved on the separating light strip. For example, Gaussian blur or radial blur processing is performed on the pixels in the region, and the resulting second image has a glowing effect.

[0104] The color of the dividing light strip can be white, as white has high brightness and produces a better glowing effect after Gaussian blurring. The dividing light strip can also be a color different from the texture color of the contour structure model. For example, if the texture color of the contour structure model is green, then bright yellow, red, or other colors can be chosen as the color of the dividing light strip to achieve the glowing effect.

[0105] This disclosure does not limit the execution order of steps 102 and 103. Step 102 may be executed first, followed by step 103; or step 103 may be executed first, followed by step 102; or steps 102 and 103 may be executed simultaneously.

[0106] In step 104, the first image and the second image are overlaid and rendered to obtain the target frame image.

[0107] The method provided in the above embodiments of this disclosure, when performing scene rendering, if the user triggers a switching rendering operation, can acquire a first display area of ​​the point cloud model in the target frame image, a second display area of ​​the contour structure model in the target frame image, and a third display area of ​​the separating light strip in the target frame image; based on the first and second display areas, the point cloud model and the contour structure model are rendered to obtain a first image, and the first image is stored in a first off-screen buffer; based on the third display area, the contour structure model and the separating light strip are rendered to obtain a second image, and the second image is stored in a second off-screen buffer; the first image and the second image are superimposed and rendered to obtain the target frame image. Therefore, the target frame image rendered by the technical solution of this disclosure can realize the switching display of the point cloud model and the contour structure model, which can not only clearly and accurately display the contour structure in three-dimensional space, but also achieve a good visual switching effect between the point cloud model and the contour structure model by separating the point cloud model and the contour structure model with a separating light strip.

[0108] Figure 3 This is a flowchart illustrating the point cloud rendering method of this disclosure for generating a contour structure model; this embodiment uses the generation of the contour structure model as an example for illustrative explanation. Figure 3 As shown, the process includes steps 301 to 303. Each step is explained below.

[0109] In step 301, the top view outer contour points of the point cloud to be processed are sorted according to clock sequence to obtain a set of ordered contour points.

[0110] In this context, the top-view outline points of the point cloud to be processed are a set of points connected end-to-end after the outline of the point cloud to be processed is extracted. This set of points is either the outline points of the bottom surface or the outline points of the top surface. Each point in this set represents the starting point of one wall and the ending point of another wall. Figure 7As shown, wall AB and wall CD share a common contour point B(C). When the outer contour points of the top view of the point cloud to be processed are sorted counterclockwise according to the clock sequence, the starting point of wall AB is A and the ending point is B, while the starting point of wall CD is C and the ending point is D. Thus, the outer contour points of the top view form a set of ordered contour points.

[0111] In this embodiment, the outer contour points of the top view can be sorted in either counterclockwise or clockwise order, and there is no specific limitation on this.

[0112] In step 302, an initial structural model is generated based on the height value of the bounding box corresponding to the point cloud to be processed in the vertical direction and a set of ordered contour points. The initial structural model is a single-layer structured model.

[0113] The initial structural model is used to indicate a model in which the walls and surfaces of a three-dimensional space have been constructed, but without textures.

[0114] In this embodiment, based on the above set of ordered contour points, a rectangular wall is constructed between every two points. The width of the wall is the distance between the two points, and the height is the vertical height value obtained by calculating the bounding box of the point cloud. Thus, the normal vectors of all walls in the initial structural model are aligned in the same direction, either facing outwards or inwards.

[0115] Specifically, if the above set of ordered contour points is a set of points arranged counterclockwise, then based on the right-hand rule of plane normal vectors, it can be determined that the normal vectors of all walls face outwards; if the above set of ordered contour points is a set of points arranged clockwise, then based on the right-hand rule of plane normal vectors, it can be determined that the normal vectors of all walls face inwards.

[0116] In this implementation, the constructed model is a single-layer structured model, that is, the wall is a single-layer plane rather than a three-dimensional wall, which ensures that the constructed outline structure model is lightweight and occupies little space.

[0117] In this process, after sorting the contour points according to clock sequence, when constructing the initial structural model, the horizontal U value in the texture map coordinates of the starting point of each wall is 0, and the horizontal U value in the texture map coordinates of the ending point is 1. For example... Figure 7 In the diagram, the horizontal U value of point A on the bottom contour of wall AB is 0, and the horizontal U value of point B is 1.

[0118] In step 303, texture mapping is performed on the initial structural model to obtain the contour structural model.

[0119] In this embodiment, in order to enhance the visual effect of the rendering, textures can be applied to the initial structural model to obtain a more vivid and full outline structural model.

[0120] There are various ways to implement textures, such as color textures, transparency textures, ambient light absorption textures, and self-illuminating textures.

[0121] Through steps 301 to 303 above, the implementation method of generating contour structure model is disclosed. By generating a single-layer structured model through contour points, the model's space occupation can be reduced while displaying the model's structural effect. Furthermore, by applying textures to the model, rich visual effects can be added to the model.

[0122] In some implementations, when color mapping the initial structural model, basic color mapping can be applied to each wall of the model. To enhance the visual appeal of the model, a gradient effect with lighting transition can be implemented for each wall of the model. Figure 4 This is a flowchart illustrating the gradient mapping process for a contour structure model in the point cloud rendering method of this disclosure; this embodiment uses the example of how to perform gradient mapping on a contour structure model for illustrative purposes. In the above... Figure 3 Based on the illustrated embodiment, step 303 may include steps 331 to 334. Each step is described below.

[0123] In step 331, based on the position and orientation information of the virtual camera, the angle between the line of sight and the plane normal vectors of each plane of the initial structural model is determined.

[0124] The virtual camera is used to indicate a virtual component that maps a 3D scene onto the screen, simulating the imaging process of the real world. The position and orientation information of the virtual camera can be set using 3D modeling software. Based on the position and orientation information of the virtual camera, the line-of-sight information can be obtained. The starting point of the line of sight is the position of the virtual camera, and the direction of the line of sight is determined based on the orientation information of the virtual camera.

[0125] In this embodiment, by inputting the position and orientation information of the virtual camera into the shader, the angle between the structural model and the normal vector of each wall can be automatically determined.

[0126] In step 332, based on the included angle, the texture surface of each plane of the initial structural model is determined, and the texture surface is at least one of the following: the inner surface of the model and the outer surface of the model.

[0127] The texture surface is used to indicate the surface to which colors, textures, etc. are mapped. If the viewpoint is a planar inner surface of the model, the texture surface is the inner surface of the model; if the viewpoint is a planar outer surface of the model, the texture surface is the outer surface of the model.

[0128] For example, if the normal vector of the wall f of the initial structural model faces the outer surface of the model, and the line of sight is also directed towards the outer surface of the wall f, then the angle between the line of sight and the plane normal vector of the wall f is less than 90 degrees. Therefore, for the wall f, the inner surface that the line of sight can see is the inner surface of the model. If the normal vector of the wall e of the initial structural model faces the outer surface of the model, and the line of sight is also directed towards the inner surface of the wall e, then the angle between the line of sight and the plane normal vector of the wall e is less than 90 degrees. Therefore, for the wall e, the outer surface that the line of sight can see is the outer surface of the model.

[0129] Furthermore, if the texture surface is the outer surface of the model, then proceed to step 333; if the texture surface is the inner surface of the model, then proceed to step 334.

[0130] In step 333, in response to the texture surface being the outer surface of the model, the gradient material texture is mapped onto the texture surface of the plane of the initial structural model.

[0131] In this embodiment, if the texture surface is the outer surface of the model, the gradient material texture mapping can be directly performed on the outer surface of the model according to the UV coordinates of each point.

[0132] In step 334, in response to the fact that the texture surface is the inner surface of the model, the texture horizontal coordinate of the gradient material texture is mirrored and flipped and then mapped onto the texture surface of the plane of the initial structural model.

[0133] In this embodiment, if the texture surface is the inner surface of the model, the texture can be directly mirrored and reversed according to the texture horizontal coordinate (the horizontal coordinate U coordinate of the texture map coordinate) of each point, and then the gradient material texture mapping can be performed on the inner surface of the model.

[0134] In this disclosure, the mirror reversal process is used to instruct the operation of 1-u on the U coordinate. After this operation, the point with U coordinate 0 becomes 1, the point with U coordinate 1 becomes 0, and the point with U coordinate 0.3 becomes 0.7.

[0135] By using steps 333 and 334, the gradient effect can be guaranteed to be the same regardless of whether the surface in sight is the inner or outer surface. For example, the gradient effect can be from the upper left to the lower right or from the upper right to the lower left.

[0136] Steps 331 to 334 above disclose a method for applying gradient material mapping to the initial structural model. Based on the angle between the viewpoint and the normal vector of each wall, it is determined whether the viewpoint is on the inner or outer surface of the wall, and different mapping operations are applied to the inner and outer surfaces. Therefore, this embodiment of the invention can achieve the same gradient effect on all walls of the structural model, further improving the point cloud rendering effect.

[0137] Figure 5 This is a flowchart illustrating the rendering of the first image in the point cloud rendering method of this disclosure; this embodiment uses the rendering of the first image as an example for illustrative explanation. In the above... Figure 1 Based on the illustrated embodiment, step 102 may include the following steps 121 to 122. Each step is described below.

[0138] In step 121, each pixel of the point cloud model within the coverage area of ​​the first display area is rendered in the first shader to obtain the third image; and each pixel of the contour structure model within the coverage area of ​​the second display area is rendered in the second shader to obtain the fourth image.

[0139] The first shader is the pixel shader for rendering the point cloud model. The first display area is the area in the target frame image where the point cloud model is displayed. For example, if the first display area in the target frame image is the lower half of the image, then when rendering the point cloud model in the first shader, the lower half of the point cloud model is rendered, while the pixels of the upper half of the point cloud model are discarded without rendering.

[0140] The second shader is a pixel shader that renders the contour structure model. The second display area is the area in the target frame image where the contour structure model is displayed. For example, if the second display area in the target frame image is the upper half of the image, then when rendering the contour structure model in the second shader, the upper half of the point cloud model is rendered, while the pixels of the lower half of the contour structure model are discarded without rendering.

[0141] In step 122, the third and fourth images are superimposed to obtain the first image.

[0142] There is a third display area between the first display area and the second display area for displaying the dividing light band. Therefore, in the first image, the point cloud model is displayed in the first display area, the contour structure model is displayed in the second display area, and no information is displayed in the middle third display area.

[0143] Steps 121 and 122 above disclose a rendering method for the first image. This method allows for the simultaneous display of point cloud models and contour structure models on the first image, as well as the switching effect between solid line point cloud models and road surface expansion structure models.

[0144] Figure 6 This is a flowchart illustrating the rendering of the second image in the point cloud rendering method of this disclosure; this embodiment uses the rendering of the first image as an example for illustrative explanation. In the above... Figure 1Based on the illustrated embodiment, step 103 may include the following steps 131 to 132. Each step is described below.

[0145] In step 131, the pixels within the coverage area of ​​the dividing light band on the contour structure model are rendered with the set color in the third shader to obtain the fifth image.

[0146] The third shader is a shader that renders the contour structure model. When rendering the contour structure model in the third shader, a strip-shaped area of ​​a height corresponding to the third display area can be selected on the contour structure model as a dividing light strip. The strip-shaped area of ​​that height on the contour structure model is rendered with a set color in the shader, and the pixels of other areas on the contour structure model are hidden using the discard function and not rendered, thus obtaining the fifth image.

[0147] The set color can be a bright color, such as pure white or golden yellow, and this set color needs to be distinguishable from the pixel color of the point cloud model in order to highlight the light strip effect.

[0148] In step 132, Gaussian blurring is applied to each pixel within the coverage area of ​​the dividing light band in the fifth image to obtain the second image.

[0149] In this embodiment, when performing Gaussian blur processing, one-dimensional Gaussian blur processing can be first performed on each pixel within the coverage area of ​​the dividing light band in the fifth image in the horizontal direction to obtain the sixth image; one-dimensional Gaussian blur processing can then be performed on each pixel within the coverage area of ​​the dividing light band in the fifth image in the vertical direction to obtain the seventh image; the sixth and seventh images are then superimposed to obtain the second image. In this implementation, performing one-dimensional Gaussian blur on each pixel within the coverage area of ​​the dividing light band in the fifth image in both the horizontal and vertical directions helps to improve the efficiency of Gaussian blur.

[0150] Steps 131 and 132 above disclose a method for rendering the segmented light band. This method renders the segmented light band on the contour structure model, avoiding blurring of some pixels in the point cloud model when achieving the glow effect of the segmented light band. This ensures that only the segmented light band has a glow effect in the superimposed target frame image, and also helps improve the clarity of the point cloud model in the image. Furthermore, using one-dimensional Gaussian blur processing reduces the computational cost of Gaussian blur and improves its efficiency.

[0151] Corresponding to the aforementioned embodiments of the point cloud rendering method, this disclosure also provides embodiments of the point cloud rendering apparatus.

[0152] Exemplary device

[0153] Figure 8 This is a schematic diagram of one embodiment of the point cloud rendering apparatus disclosed herein, which is applied to electronic devices (such as computer systems, servers), such as... Figure 8 As shown, the device includes:

[0154] The acquisition module 81 is used to acquire the first display area of ​​the point cloud model in the target frame image, the second display area of ​​the contour structure model in the target frame image, and the third display area of ​​the separating light strip in the target frame image. The separating light strip is used to separate the point cloud model and the contour structure model in the target frame image.

[0155] The first rendering module 82 is used to render a point cloud model and a contour structure model based on a first display area and a second display area to obtain a first image, and to store the first image in a first off-screen buffer.

[0156] The second rendering module 83 is used to render the outline structure model and the segmented light band based on the third display area to obtain the second image, and to store the second image in the second off-screen buffer.

[0157] The overlay rendering module 84 is used to overlay the first image and the second image to obtain the target frame image.

[0158] Figure 9 This is a schematic diagram of another embodiment of the point cloud rendering apparatus disclosed herein, as shown below. Figure 9 As shown, in Figure 8 Based on the illustrated embodiments, in some optional implementations, the apparatus further includes:

[0159] The point sorting module 85 is used to sort the outer contour points of the top view of the point cloud to be processed according to the clock sequence to obtain a set of ordered contour points.

[0160] The model generation module 86 is used to generate an initial structural model based on the height value of the bounding box corresponding to the point cloud to be processed in the vertical direction and a set of ordered contour points. The initial structural model is a single-layer structured model.

[0161] The texture mapping module 87 is used to apply texture mapping to the initial structural model to obtain the contour structural model.

[0162] In some alternative implementations, the texture mapping module 87 includes:

[0163] The first determining submodule 871 is used to respond to applying gradient material mapping to the initial structural model, and to determine the angle between the line of sight and the plane normal vectors of each plane of the initial structural model based on the position and orientation information of the virtual camera.

[0164] The second determining submodule 872 is used to determine the texture surface of each plane of the initial structural model based on the included angle. The texture surface is at least one of the following: the inner surface of the model and the outer surface of the model.

[0165] The first mapping submodule 873 is used to map the gradient material texture onto the texture surface of the plane of the initial structural model in response to the texture surface being the outer surface of the model;

[0166] The second mapping submodule 874 is used to mirror and flip the texture horizontal coordinates of the gradient material texture and map them onto the texture surface of the plane of the initial structural model in response to the texture surface being the inner surface of the model.

[0167] In some alternative implementations, the first rendering module 82 includes:

[0168] The first rendering submodule 821 is used to render each pixel of the point cloud model within the coverage area of ​​the first display area in the first shader to obtain the third image; and to render each pixel of the contour structure model within the coverage area of ​​the second display area in the second shader to obtain the fourth image.

[0169] The overlay submodule 822 is used to overlay the third image and the fourth image to obtain the first image.

[0170] In some alternative implementations, the second rendering module 83 includes:

[0171] The second rendering submodule 831 is used in the third shader to render each pixel within the coverage area of ​​the dividing light band on the contour structure model as a set color to obtain the fifth image.

[0172] The blurring submodule 832 is used to perform Gaussian blurring on each pixel within the coverage area of ​​the dividing light band in the fifth image to obtain the second image.

[0173] In some alternative implementations, the fuzzing submodule 832 includes:

[0174] The first blurring unit 8321 is used to perform one-dimensional Gaussian blurring on each pixel within the coverage area of ​​the dividing light band in the fifth image in the horizontal direction to obtain the sixth image.

[0175] The second blurring unit 8322 is used to perform one-dimensional Gaussian blurring on each pixel within the coverage area of ​​the dividing light band in the fifth image in the vertical direction to obtain the seventh image.

[0176] The overlay unit 8323 is used to overlay the sixth image and the seventh image to obtain the second image.

[0177] The modules and units in this disclosed device can be further divided into finer-grained units according to actual needs, and the specific configuration can be set according to actual needs.

[0178] The apparatus of this disclosure embodiment can be used to implement the methods of the above embodiments of this disclosure. The two correspond to each other in specific implementation, and the specific implementation of related parts can be referred to each other, which will not be repeated here.

[0179] Exemplary electronic devices, computer program products, and computer-readable storage media

[0180] This disclosure also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, wherein when the computer program is executed, it implements the point cloud rendering method described in any of the above embodiments of this disclosure.

[0181] Below, for reference Figure 10 This describes an electronic device according to embodiments of the present disclosure, wherein apparatus for implementing methods according to embodiments of the present disclosure may be integrated. Figure 10 This is a structural diagram of an electronic device provided in an illustrative embodiment of the present disclosure, such as... Figure 10 As shown, the electronic device includes one or more processors 101, one or more memory 102s of computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 102 is executed, the point cloud rendering method described above can be implemented.

[0182] Specifically, in practical applications, the electronic device may also include components such as an input device 103 and an output device 104, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). Those skilled in the art will understand that... Figure 10 The structure of the electronic device shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or certain components, or different component arrangements.

[0183] in:

[0184] The processor 101 may be a central processing unit (CPU) or other form of processing unit with point cloud rendering capability and / or instruction execution capability. It performs various functions and processes data by running or executing software programs and / or modules stored in memory 102 and calling data stored in memory 102, thereby performing overall monitoring of the electronic device.

[0185] The memory 102 can store one or more computer program products. The memory can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program products can be stored on the computer-readable storage medium, and the processor 101 can run the computer program products to implement the point cloud rendering methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components can also be stored in the computer-readable storage medium.

[0186] The input device 103 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0187] The output device 104 can output various information to the outside, including determined distance information, direction information, etc. The output device 104 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0188] The electronic device may also include a power supply for powering various components, which can be connected to the processor 101 functional module through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and any other components.

[0189] Of course, for the sake of simplicity, Figure 10 Only some of the components of the electronic device relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0190] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the point cloud rendering methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0191] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0192] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the point cloud rendering methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0193] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0194] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0195] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0196] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0197] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0198] The description in this disclosure is provided for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the disclosure to its forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of this disclosure and to enable those skilled in the art to understand this disclosure and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A point cloud rendering method, characterized in that, include: The first display area of ​​the point cloud model in the target frame image, the second display area of ​​the contour structure model in the target frame image, and the third display area of ​​the separating light strip in the target frame image are obtained. The separating light strip is used to separate the point cloud model and the contour structure model in the target frame image. Based on the first display area and the second display area, the point cloud model and the contour structure model are rendered to obtain a first image, and the first image is stored in the first off-screen buffer. Based on the third display area, the contour structure model and the dividing light band are rendered to obtain a second image, and the second image is stored in the second off-screen buffer. The first image and the second image are overlaid and rendered to obtain the target frame image; The steps for constructing the contour structure model include: The top view outer contour points of the point cloud to be processed are sorted according to clock sequence to obtain a set of ordered contour points; Based on the height value of the bounding box corresponding to the point cloud to be processed in the vertical direction and the set of ordered contour points, an initial structural model is generated, which is a single-layer structured model. Texture mapping is performed on the initial structural model to obtain the contour structural model.

2. The method according to claim 1, characterized in that, In response to applying a gradient material map to the initial structural model, the step of applying a texture map to the initial structural model to obtain the contour structural model includes: Based on the position and orientation information of the virtual camera, the angle between the line of sight and the plane normal vector of each plane of the initial structural model is determined; Based on the included angle, the texture surface of each plane of the initial structural model is determined, and the texture surface is at least one of the following: the inner surface of the model and the outer surface of the model; In response to the fact that the texture surface is the outer surface of the model, the gradient material texture is mapped onto the texture surface of the plane of the initial structural model; In response to the fact that the texture surface is the inner surface of the model, the texture horizontal coordinate of the gradient material texture is mirrored and flipped and then mapped onto the texture surface of the plane of the initial structure model.

3. The method according to any one of claims 1-2, characterized in that, The step of rendering the point cloud model and the contour structure model to obtain a first image based on the first display area and the second display area includes: The first shader renders each pixel of the point cloud model within the coverage area of ​​the first display area to obtain the third image; and the second shader renders each pixel of the contour structure model within the coverage area of ​​the second display area to obtain the fourth image. The third image and the fourth image are superimposed to obtain the first image.

4. The method according to any one of claims 1-2, characterized in that, The step of rendering the contour structure model and the segmented light bands based on the third display area to obtain the second image includes: In the third shader, each pixel within the area covered by the dividing light band on the contour structure model is rendered with a set color to obtain the fifth image; Gaussian blurring is applied to each pixel within the coverage area of ​​the dividing light band in the fifth image to obtain the second image.

5. The method according to claim 4, characterized in that, The step of performing Gaussian blur processing on each pixel within the coverage area of ​​the separated light band in the fifth image to obtain the second image includes: A one-dimensional Gaussian blur is applied to each pixel within the coverage area of ​​the dividing light band in the fifth image in the horizontal direction to obtain the sixth image; Perform one-dimensional Gaussian blurring on each pixel within the coverage area of ​​the dividing light band in the fifth image in the vertical direction to obtain the seventh image; The sixth image and the seventh image are superimposed to obtain the second image.

6. A point cloud rendering device, characterized in that, include: The acquisition module is used to acquire the first display area of ​​the point cloud model in the target frame image, the second display area of ​​the contour structure model in the target frame image, and the third display area of ​​the separating light strip in the target frame image, wherein the separating light strip is used to separate the point cloud model and the contour structure model in the target frame image. The first rendering module is used to render the point cloud model and the contour structure model based on the first display area and the second display area to obtain a first image, and store the first image in a first off-screen buffer. The second rendering module is used to render the contour structure model and the dividing light band based on the third display area to obtain a second image, and to store the second image in the second off-screen buffer. An overlay rendering module is used to overlay and render the first image and the second image to obtain the target frame image; The device further includes: The point sorting module is used to sort the outer contour points of the top view of the point cloud to be processed according to the clock sequence, so as to obtain a set of ordered contour points. The model generation module is used to generate an initial structural model based on the height value of the bounding box corresponding to the point cloud to be processed in the vertical direction and the set of ordered contour points. The initial structural model is a single-layer structured model. The texture mapping module is used to apply texture mapping to the initial structural model to obtain the contour structural model.

7. The apparatus according to claim 6, characterized in that, The texture mapping module includes: The first determining submodule is used to determine the angle between the line of sight and the plane normal vector of each plane of the initial structure model in response to applying a gradient material texture to the initial structure model, based on the position and orientation information of the virtual camera. The second determining submodule is used to determine the texture surface of each plane of the initial structural model based on the included angle, wherein the texture surface is at least one of the following: the inner surface of the model and the outer surface of the model; The first mapping submodule is used to map the gradient material texture onto the texture surface of the plane of the initial structural model in response to the texture surface being the outer surface of the model. The second mapping submodule is used to mirror and flip the texture horizontal coordinate of the gradient material texture and then map it onto the texture surface of the plane of the initial structure model in response to the texture surface being the inner surface of the model.

8. An electronic device, characterized in that, The electronic device includes: Memory, used to store computer program products; A processor is configured to execute a computer program product stored in the memory, wherein, when the computer program product is executed, it implements the method described in any one of claims 1-5.

Images