Point cloud rendering method, apparatus, device, and storage medium

Abstract

Embodiments of the present disclosure disclose a point cloud rendering method, device, equipment and storage medium, wherein the method comprises: determining a thickness threshold of a to-be-processed point cloud, the thickness threshold being used to indicate a depth range of points participating in transparency blending when rendering a target scene using the to-be-processed point cloud; performing a depth rendering operation of the target scene to obtain an original depth value of the target scene; obtaining a target depth value of the target scene based on the original depth value of the target scene and the corresponding thickness threshold, and writing the target depth value of the target scene into a depth buffer; performing a depth test based on the target depth value of the target scene to obtain a target rendering point set that passes the depth test; and rendering the target rendering point set in a transparency blending manner. The embodiments of the present disclosure can ensure that the rendered object surface contour is clear and stable by performing transparency blending rendering on target rendering points in the thickness threshold range of the to-be-processed point cloud.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of data processing, and in particular, to a point cloud rendering method, device, equipment and storage medium. BACKGROUND

[0002] The laser point cloud refers to a data set of space points collected by a three-dimensional laser radar device, which can be used for model reconstruction, positioning, etc.

[0003] In the related art, after collecting point cloud data, the point cloud can be processed through point cloud thinning, uniformization, etc. However, there may still be occlusion relationships between points in the processed point cloud. When there is an occlusion relationship between points and the distance between the points is very close, a z-fighting phenomenon may occur. The z-fighting phenomenon refers to the fact that the depth values of two points are very close, so that the occlusion relationship between the two points cannot be distinguished, resulting in flickering and jittering phenomenon during rendering, and poor rendering effect. SUMMARY

[0004] To solve the above technical problems, embodiments of the present disclosure provide a point cloud rendering method, device, equipment and storage medium.

[0005] According to an aspect of an embodiment of the present disclosure, a point cloud rendering method is provided, comprising:

[0006] determining a thickness threshold of a to-be-processed point cloud, the thickness threshold being used to indicate a depth range of points participating in transparency blending when rendering a target scene using the to-be-processed point cloud;

[0007] performing a depth rendering operation of the target scene to obtain an original depth value of the target scene, the depth rendering operation comprising: setting a state of depth testing to an open state, setting a depth write state of the to-be-processed point cloud to an open state, and closing transparency blending;

[0008] obtaining a target depth value of the target scene based on the original depth value of the target scene and the corresponding thickness threshold, and writing the target depth value of the target scene into a depth buffer;

[0009] performing depth testing based on the target depth value of the target scene to obtain a target rendering point set passing the depth testing, the target rendering point set including at least one target rendering point in the to-be-processed point cloud;

[0010] rendering the target rendering point set in a transparency blending manner.

[0011] In some optional embodiments, the determining the thickness threshold of the to-be-processed point cloud comprises:

[0012] plane fitting is performed on the points in the point cloud to be processed to obtain a corresponding plane;

[0013] Based on the thickness value of the plane, the planar rate of the plane is calculated, which is used to indicate the proportion of the number of points in the point cloud to be processed that are less than the thickness value from the plane;

[0014] The thickness value of the plane is iteratively updated, and the calculation of the planar rate of the plane is realized until a preset condition is met, thereby obtaining the thickness threshold of the point cloud to be processed.

[0015] In still some optional embodiments, the depth test is performed based on the target depth value of the target scene, comprising:

[0016] The depth write state of the target scene is set to the off state, and the state of the depth test is set to the on state;

[0017] The target depth value of the target scene is obtained from the depth buffer;

[0018] Based on the obtained target depth value of the target scene, the depth test is performed on the target scene.

[0019] In still some optional embodiments, before the target rendering point set is rendered in the transparency blending mode, further comprising:

[0020] The fragment blending weight of each target rendering point in the at least one target rendering point is calculated, and the fragment blending weight indicates the blending weight of each fragment corresponding to each target rendering point;

[0021] The blending weight of each fragment corresponding to each target rendering point is stored in the transparency channel.

[0022] In still some optional embodiments, the target rendering point set is rendered in the transparency blending mode, comprising:

[0023] According to the three-dimensional coordinates of each fragment in each target rendering point, it is determined whether each fragment has other fragments with occlusion relationship;

[0024] In response to one fragment having other fragments with occlusion relationship, each fragment with occlusion relationship is rendered in the transparency blending mode based on the order-independent translucent rendering algorithm;

[0025] In response to one fragment not having other fragments with occlusion relationship, the one fragment and the background fragment are rendered in the transparency blending mode.

[0026] In some optional embodiments, the calculating the fragment blending weight of each target rendering point in the at least one target rendering point comprises:

[0027] calculating a distance between each fragment and a center fragment of a corresponding target rendering point;

[0028] normalizing the distance corresponding to each fragment to obtain a fragment blending weight of each fragment.

[0029] In some optional embodiments, the point cloud to be processed is obtained by:

[0030] performing point cloud acquisition on the target scene to obtain an original point cloud;

[0031] performing preprocessing on the original point cloud to obtain a preprocessed point cloud;

[0032] segmenting the preprocessed point cloud into at least one point cloud to be processed.

[0033] According to still another aspect of the embodiments of the present disclosure, a point cloud rendering device is provided, comprising:

[0034] a first determining module configured to determine a thickness threshold of a point cloud to be processed, the thickness threshold being used to indicate a depth range of points participating in transparency blending when the target scene is rendered using the point cloud to be processed;

[0035] a first rendering module configured to perform a depth rendering operation of the target scene to obtain an original depth value of the target scene, the depth rendering operation comprising: setting a state of depth test to an open state, setting a depth write state of the point cloud to be processed to an open state, and closing transparency blending;

[0036] a first calculating module configured to obtain a target depth value of the target scene based on the original depth value of the target scene and a corresponding thickness threshold, and write the target depth value of the target scene into a depth buffer;

[0037] a depth test module configured to perform depth test based on the target depth value of the target scene to obtain a target rendering point set passing the depth test, the target rendering point set comprising at least one target rendering point in the point cloud to be processed;

[0038] a second rendering module configured to render the target rendering point set in a transparency blending manner.

[0039] In some optional embodiments, the first determining module comprises:

[0040] a fitting sub-module configured to perform plane fitting on points in the point cloud to be processed to obtain a corresponding plane;

[0041] a calculation sub-module, configured to calculate a planar rate of the plane based on the thickness value of the plane, the planar rate being used to indicate a proportion of points in the point cloud to be processed that are less than the thickness value from the plane;

[0042] an iteration sub-module, configured to iteratively update the thickness value of the plane, and implement calculation of the planar rate of the plane until a preset condition is met, to obtain a thickness threshold of the point cloud to be processed.

[0043] In some other optional embodiments, the depth test module comprises:

[0044] an operation execution sub-module, configured to set a depth write state of a target scene as a close operation, and set a state of depth test as an open state;

[0045] an acquisition sub-module, configured to acquire a target depth value of the target scene from the depth buffer;

[0046] a test sub-module, configured to perform depth test on the target scene based on the acquired target depth value of the target scene.

[0047] In some other optional embodiments, the apparatus further comprises:

[0048] a second calculation module, configured to calculate a fragment blending weight of each target rendering point in the at least one target rendering point, the fragment blending weight indicating a blending weight of each fragment corresponding to each target rendering point;

[0049] a storage module, configured to store the blending weight of each fragment corresponding to each target rendering point in a transparency channel.

[0050] In some other optional embodiments, the second rendering module comprises:

[0051] a determination sub-module, configured to determine whether each fragment has other fragments having an occlusion relationship according to three-dimensional coordinates of each fragment in each target rendering point;

[0052] a first rendering sub-module, configured to, in response to one fragment having other fragments having an occlusion relationship, perform rendering on the fragments having an occlusion relationship in a transparency blending manner based on a sequential-independent translucent rendering algorithm;

[0053] a second rendering sub-module, configured to, in response to one fragment not having other fragments having an occlusion relationship, perform transparency blending rendering on the one fragment and a background fragment.

[0054] In some other optional embodiments, the second calculation module comprises:

[0055] a distance calculation sub-module, configured to calculate a distance between each patch and a center patch of a corresponding target rendering point;

[0056] a normalization sub-module, configured to normalize the distance corresponding to each patch to obtain a patch blending weight of each patch.

[0057] In some optional embodiments, the apparatus further includes:

[0058] a point cloud acquisition module, configured to acquire a point cloud of the target scene to obtain an original point cloud;

[0059] a processing module, configured to pre-process the original point cloud to obtain a pre-processed point cloud;

[0060] a segmentation module, configured to segment the pre-processed point cloud into at least one to-be-processed point cloud.

[0061] According to still another aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores computer program instructions, and the computer program instructions, when executed, implement the point cloud rendering method.

[0062] According to still another aspect of the embodiments of the present disclosure, an electronic device is provided, which includes:

[0063] a memory, configured to store a computer program product;

[0064] a processor, configured to execute the computer program product stored in the memory, and the computer program product, when executed, implements the point cloud rendering method.

[0065] According to still another aspect of the embodiments of the present disclosure, a computer program product is provided, which includes computer program instructions, and the computer program instructions, when executed by a processor, implement the point cloud rendering method.

[0066] Based on the point cloud rendering method, apparatus and storage medium provided by the embodiments of the present disclosure, when rendering a scene, the thickness threshold of each to-be-processed point cloud can be determined first, then a depth rendering operation of a target scene is performed to obtain an original depth value of the target scene, the target depth value of the target scene that can be rendered is determined according to the original depth value and the thickness threshold, and the target depth value is written into a depth buffer, the target rendering point is obtained by performing a depth test using the target depth value in the depth buffer, and the target rendering point is rendered in a transparency blending manner. Thus, the technical solution of the present disclosure can avoid the flickering and jitter phenomenon of the surface of an object during rendering due to depth conflict by rendering the target rendering point in the thickness threshold range of the to-be-processed point cloud in a transparency blending manner, and ensure that the rendered surface of the object is clear and stable.

[0067] The technical solutions of the present disclosure will be described in further detail below with reference to the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0068] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

[0069] The present disclosure can be understood moreappreciably with reference to the following detailed description when taken in conjunction with the accompanying drawings, in which:

[0070] Figure 1 a flow chart of an embodiment of the point cloud rendering method of the present disclosure;

[0071] Figure 2 a flow chart of obtaining the thickness threshold of the point cloud to be processed in the point cloud rendering method of the present disclosure;

[0072] Figure 3 a flow chart of determining the blending weight of each patch in the point cloud rendering method of the present disclosure;

[0073] Figure 4 a flow chart of performing transparency blending rendering in the point cloud rendering method of the present disclosure;

[0074] Figure 5 a comparison chart of the rendering effect achieved in the point cloud rendering method of the present disclosure according to the transparency blending mode;

[0075] Figure 6 a structural schematic diagram of an embodiment of the point cloud rendering device of the present disclosure;

[0076] Figure 7 a structural schematic diagram of another embodiment of the point cloud rendering device of the present disclosure;

[0077] Figure 8 a structural diagram of an electronic device provided by an illustrative embodiment of the present disclosure. DETAILED DESCRIPTION

[0078] The example embodiments according to the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited by the example embodiments described herein.

[0079] It should be noted that the relative arrangement of components and steps, numerical expressions, and numerical values set forth in these embodiments are not limiting to the scope of the present disclosure unless specifically stated otherwise.

[0080] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor represent the inherent functional module order between them.

[0081] It should also be understood that in the embodiments of the present disclosure, "multiple" can mean two or more, and "at least one" can mean one, two or more.

[0082] It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, it can be understood as one or more in general, without explicit limitation or in the context of the opposite indication.

[0083] In addition, the term "and / or" in the present disclosure is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the front and rear associated objects.

[0084] It should also be understood that the description of the embodiments of the present disclosure emphasizes the differences between the embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0085] At the same time, it should be understood that in order to facilitate the description, the size of each part shown in the drawings is not drawn according to the actual proportional relationship.

[0086] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application or uses.

[0087] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered part of the specification where appropriate.

[0088] It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be discussed further in subsequent drawings.

[0089] The embodiments of the present disclosure can be applied to terminal devices, computer systems, servers and other electronic devices, which can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with terminal devices, computer systems, servers and other electronic devices include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above systems, and the like.

[0090] Terminal devices, computer systems, servers and other electronic devices can be described in the general context of computer system executable instructions, such as program modules, executed by the computer system. Generally, program modules can include routines, programs, objects, components, functional modules, data structures, and the like, which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, in which tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media, including storage devices.

[0091] Summary of the disclosure

[0092] In the process of implementing the present disclosure, the inventors found that since the laser point cloud has a certain range of depth offset, that is, the point cloud has a certain thickness, when rendering the target scene using the point cloud, as the camera view changes, the rendered scene may have occlusion disturbance, and the rendering effect is poor.

[0093] wherein the thickness of the point cloud is the surface roughness of the point cloud, due to the influence of factors such as laser radar ranging error, when collecting point cloud on the surface of an object, the distribution of laser radar point cloud on the surface of the object may have a certain shock, which forms the thickness of the point cloud.

[0094] Exemplary method

[0095] Figure 1 A flowchart of one embodiment of the point cloud rendering method of the present disclosure; the point cloud rendering method can be applied to electronic devices (such as computer systems, servers), as shown in Figure 1 The point cloud rendering method includes the following steps 101-105. Each step is described below.

[0096] In step 101, the thickness threshold of the point cloud to be processed is determined.

[0097] The thickness threshold is used to indicate a depth range of points participating in transparency mixing when rendering the target scene using the to-be-processed point cloud. Points in the to-be-processed point cloud within the depth range of the thickness threshold can be rendered by means of transparency mixing. The thickness threshold of point clouds of different object surfaces can be different. For example, a smaller thickness threshold can be set for a large-area plane such as a ground surface or a wall surface. An empirical value can be set as the thickness threshold in a case where there are many indoor objects. For details, refer to Figure 2 The embodiment disclosed in the foregoing manner determines the thickness threshold of the to-be-processed point cloud, which will not be described in detail herein.

[0098] In some optional embodiments, the to-be-processed point cloud is a point cloud collected by using a laser radar on the target scene. After an original point cloud is collected by using a laser radar on the target scene, the original point cloud can be preprocessed to obtain a preprocessed point cloud, and then the preprocessed point cloud is subjected to point cloud segmentation to obtain at least one to-be-processed point cloud.

[0099] In some optional embodiments, the preprocessing performed on the original point cloud can include point cloud thinning, uniformization, and de-noising, to obtain a preprocessed point cloud. Through preprocessing, the number of point clouds can be reduced, and the processing efficiency can be improved.

[0100] In some optional embodiments, point cloud segmentation refers to dividing the preprocessed point cloud according to spatial, geometric, and texture features. After point cloud segmentation, at least one to-be-processed point cloud can be obtained, and points in each to-be-processed point cloud have similar features. In a specific implementation, a region-based segmentation manner can be used to implement point cloud segmentation. The region-based segmentation manner refers to using neighborhood information to classify nearby points with similar attributes to obtain a to-be-processed point cloud corresponding to each segmentation region.

[0101] In step 102, a depth rendering operation of the target scene is performed to obtain an original depth value of the target scene. The depth rendering operation includes: setting a state of depth testing to an enabled state, setting a depth write state of the to-be-processed point cloud to an enabled state, and disabling transparency mixing.

[0102] The depth testing is a technology for judging the front and back occlusion relationship between objects in the target scene to determine which fragments can be displayed on the screen. In rendering, a fragment that passes the depth testing can be rendered, and a fragment that does not pass the depth testing is discarded. The depth write is used to indicate whether the depth buffer needs to be overwritten on the basis of the depth testing, that is, whether the depth value of the depth testing needs to be reset. When the depth testing is performed, if the depth write of the to-be-processed point cloud is also started, it is indicated that the value of the depth testing needs to be set. The transparency mixing is a technology for implementing a semi-transparent effect. The transparency of a current fragment can be used as a mixing factor to mix with a color value already stored in a color buffer to obtain a new color.

[0103] In the execution of the depth rendering operation, by setting the state of the depth test as an open state, setting the depth write state of the to-be-processed point cloud as an open state, and closing the transparency blending, the original depth value of the target scene can be directly output in the vertex shader.

[0104] In some optional embodiments, the original depth value of the target scene can be understood as an original depth map of the target scene, and each pixel point is used to indicate the depth information of each point in the scene.

[0105] In step 103, based on the original depth value of the target scene and the corresponding thickness threshold, the target depth value of the target scene is obtained, and the target depth value of the target scene is written into the depth buffer.

[0106] In some optional embodiments, for different to-be-processed point clouds, different thickness thresholds d are used, and when calculating the target depth value of the target scene, the original depth value of the target scene and the corresponding thickness threshold can be used to obtain the target depth value of the target scene, which is equivalent to increasing the depth of each point in the target scene by the corresponding thickness threshold d.

[0107] In specific implementation, when calculating the target depth value of the target scene, the original depth value of each point can be increased by the thickness threshold of the point cloud to which the point belongs, for example, point 1 is in to-be-processed point cloud 1, and the thickness threshold of to-be-processed point cloud 1 is 2 mm, then the depth value of point 1 is increased by 2 mm to obtain the target depth value.

[0108] In some optional embodiments, the depth buffer is a buffer used in graphics rendering, which is a two-dimensional texture used to store the depth values of each point in the scene.

[0109] In step 104, based on the target depth value of the target scene, a depth test is performed to obtain a target rendering point set passing the depth test.

[0110] Among the target rendering point set, at least one target rendering point in the to-be-processed point cloud is included.

[0111] In some optional embodiments, when performing the depth test, the depth write state of the target scene can be set as a closed operation, and the state of the depth test can be set as an open state, so that by closing the depth write, the original depth value of the target scene is no longer written, and thus the target depth value in the depth buffer can be used for the depth test.

[0112] In some optional embodiments, the target depth value is obtained by increasing the original depth value of each point by the thickness threshold of the point cloud to which the point belongs, and therefore the target rendering point passing the depth test is all points in the to-be-processed point cloud within the range between the original depth value and the target depth value obtained by moving the thickness threshold d away.

[0113] In step 105, the target rendering point set is rendered in a transparency blending manner.

[0114] In some optional embodiments, the specific implementation of rendering the points in the target rendering point set in a transparency blending manner can refer to the specific implementation of rendering the points in the target rendering point set in a transparency blending manner in the embodiment shown in Figure 4 which will not be described in detail here.

[0115] The method provided by the above embodiments of the present disclosure can determine the thickness threshold of each point cloud to be processed when rendering a scene, then perform a depth rendering operation on the target scene to obtain an original depth value of the target scene, determine a target depth value of the target scene according to the original depth value and the thickness threshold, and write the target depth value into a depth buffer, perform a depth test using the target depth value in the depth buffer to obtain a target rendering point, and render the target rendering point in a transparency blending manner. Thus, the technical solution of the present disclosure can render the target rendering point in the thickness threshold range in the point cloud to be processed in a transparency blending manner, so as to avoid flickering and jitter on the surface of an object during rendering due to depth conflicts, and ensure that the rendered surface of the object is clear and stable.

[0116] In some optional examples, when performing a depth test based on the target depth value of the target scene, the operation of closing depth writing and starting the depth test is performed, so as to ensure that the target depth value of the target scene is obtained from the depth buffer, and the target scene is subjected to a depth test according to the obtained target depth value of the target scene. In this implementation, since the target depth value of the target scene is obtained by adding the thickness threshold of the point cloud to the original depth value of each point cloud of the target scene, when performing the depth test, the points subjected to the depth test are all points in the range between the original depth value and the target depth value obtained by moving the thickness threshold d away from the target scene.

[0117] Figure 2 The flowchart of the method for obtaining the thickness threshold of the point cloud to be processed is provided for the point cloud rendering method of the present disclosure; the present embodiment takes how to obtain the thickness threshold of the point cloud to be processed as an example for illustrative description, as shown in Figure 2 the steps 101 can include the following steps 111-113. Each step will be described below.

[0118] In step 111, the points in the point cloud to be processed are subjected to plane fitting to obtain a corresponding plane.

[0119] In some optional embodiments, the points in the point cloud to be processed can be fitted to a plane by a plane fitting algorithm, which is based on the principle of least squares and aims to minimize the sum of squares of distances from all data points to the plane to estimate a plane and obtain a plane equation.

[0120] In step 112, a planeness rate of the plane is calculated based on the thickness value of the plane, and the planeness rate is used to indicate a proportion of points in the point cloud to be processed that are less than the thickness value from the plane.

[0121] In some optional embodiments, the thickness value of the plane is used to indicate the thickness of the plane, which is a value that can be iteratively updated. The initial value of the thickness value of the plane can be a fixed small value, for example, 1 mm.

[0122] In some optional embodiments, by calculating the coordinates of each point in the point cloud to be processed and the plane equation of the fitted plane, the distance of each point from the plane can be calculated, and then the number of points less than the thickness value of the plane can be obtained. The number of points and the total number of points in the point cloud to be processed are calculated, and the proportion of points less than the thickness value of the plane is obtained, that is, the planeness rate corresponding to the thickness value of the plane is obtained.

[0123] In step 113, the thickness value of the plane is iteratively updated to calculate the planeness rate of the plane until a preset condition is met to obtain the thickness threshold of the point cloud to be processed.

[0124] In some optional embodiments, the preset condition can be that the planeness rate reaches a set planeness rate threshold. In order to render as many points in the point cloud to be processed as possible, the planeness rate threshold can be a large value, such as 80%. When the planeness rate reaches the planeness rate threshold, the iteration can be stopped, and the current updated thickness value of the plane is determined as the thickness threshold of the point cloud to be processed.

[0125] In some optional embodiments, the preset condition can also be that the iteratively updated thickness value reaches a set threshold. In order to ensure the rendering effect, the thickness value of the plane cannot exceed the set threshold. The set threshold can be a large thickness value, such as 10 mm. If the planeness rate calculated after multiple iterations of updating the thickness value of the plane still cannot meet the condition, the set threshold can be determined as the thickness threshold of the point cloud to be processed.

[0126] In some optional embodiments, the thickness thresholds of the point clouds of different object surfaces can not be the same, and the thickness thresholds calculated according to different point clouds to be processed can not be the same.

[0127] Through steps 111-113, the manner of determining the thickness threshold of the transparency blending rendering is disclosed, and different manners can be used to determine the thickness threshold for different point clouds, such as determining the thickness threshold according to the plane rate or determining the thickness threshold according to the set threshold set according to experience, thereby providing a feasible implementation manner for the transparency blending rendering of the point cloud to be processed.

[0128] In some optional examples, in order to achieve proper transparency blending rendering, the fragment blending weight of each target rendering point in at least one target rendering point can be calculated in advance, and the fragment blending weight of each target rendering point indicates the blending weight of each fragment corresponding to each target rendering point; and the blending weight of each fragment corresponding to each target rendering point is stored in the transparency channel. In specific implementation, each point in the point cloud can be rendered as a group of fragments according to a set radius, and each fragment can implement transparency blending with other fragments having an occlusion relationship according to the fragment blending weight. The manner of calculating the fragment blending weight of each fragment is described in the embodiment shown in Figure 3

[0129] Figure 3 The flowchart of the point cloud rendering method of the present disclosure for determining the fragment blending weight of each fragment; the present embodiment takes how to determine the fragment blending weight of each fragment as an example for illustrative description, as shown in Figure 3

[0130] In step 301, the distance between each fragment and the center fragment corresponding to the target rendering point is calculated.

[0131] In some optional embodiments, a target rendering point can be drawn as a group of fragments according to a set radius, and the fragment located at the center position of the group of fragments is the center fragment of the target rendering point. When each point is drawn as a group of fragments, the two-dimensional coordinates (x, y) and the depth z of each fragment can be recorded at the same time.

[0132] In a group of fragments corresponding to a target rendering point, the distance between each fragment and the center fragment can be calculated according to the two-dimensional coordinates (x, y) of each fragment.

[0133] In step 302, the distance corresponding to each fragment is normalized to obtain the fragment blending weight of each fragment.

[0134] In some optional embodiments, normalization is a data processing manner, which can limit the data within a certain fixed range after processing.

[0135] In the present implementation, the distance between each fragment and the center fragment can be normalized to a decimal number between 0 and 1 to obtain the fragment blending weight of each fragment.

[0136] ​​The distance is inversely proportional to the fragment blending weight. The greater the distance, the smaller the fragment blending weight. The smaller the distance, the greater the fragment blending weight. For example, the fragment blending weight of a fragment at an edge position of a target rendering point is 0, and the fragment blending weight of a central fragment is 1.

[0137] In some optional embodiments, after the fragment blending weight of each fragment is calculated, the fragment blending weight can be written into the alpha channel of gl_FragColor for subsequent transparency blending.

[0138] The steps 301-302 disclose a method for determining the fragment blending weight of each fragment for subsequent transparency blending.

[0139] Figure 4 A flowchart for performing transparency blending rendering of the point cloud rendering method of the present disclosure is shown in FIG. 5. The present embodiment takes how to perform transparency blending rendering as an example for illustrative description. Figure 4 As shown in FIG. 5, the step 105 can include the following steps 151-153. Each step is described below.

[0140] In step 151, whether each fragment has other fragments with occlusion relationship is determined according to the three-dimensional coordinates of each fragment in each target rendering point.

[0141] In some optional embodiments, the three-dimensional coordinates of each fragment include two-dimensional coordinates (x, y) and depth z. If the two-dimensional coordinates (x, y) of two fragments are the same and the depth z is different, it can be determined that there is an occlusion relationship between the two fragments.

[0142] Further, if a fragment has other fragments with occlusion relationship, step 152 can be performed; if a fragment does not have other fragments with occlusion relationship, step 153 can be performed.

[0143] In step 152, if a fragment has other fragments with occlusion relationship, each fragment with occlusion relationship is rendered in a transparency blending manner based on an order-independent transparency rendering algorithm.

[0144] In some optional embodiments, the order-independent transparency rendering algorithm (OIT for short) refers to a transparency rendering algorithm that does not require sorting of transparent objects in the rasterization process. Since sorting is not required, the performance consumption problem caused by rendering sorting can be avoided.

[0145] In a specific implementation, each fragment that has an occlusion relationship can be rendered according to the formula Color = (sourceColor * sourceAlpha + dstColor * 1) / 2.

[0146] For example, when the two-dimensional coordinates of the fragment 1 and the fragment 2 are the same, the color of the fragment 1 can be taken as the sourceColor, the fragment blending weight of the fragment 1 can be taken as the sourceAlpha, and the color of the fragment 2 can be taken as the dstColor. The mixed color can be calculated according to the above formula. After the color blending of the fragment 1 and the fragment 2 is completed, the obtained mixed color is further blended with the background fragment. The transparency of the background fragment is a set parameter 0, and the color of the background is also a set value 0.

[0147] When the two-dimensional coordinates of the fragment 1, the fragment 2, the fragment 3, the fragment 4, and the fragment n are the same, the fragment 1, the fragment 2, the fragment 3, and the fragment n can be iteratively blended. For example, the color of the fragment 1 and the fragment 2 is first blended, and then the blended color is blended with the fragment 3, and the iteration is sequentially performed, and finally the fragment 4 and the fragment n are blended with the background fragment to obtain the final color at the coordinate position.

[0148] In step 153, if a fragment does not have other fragments that have an occlusion relationship, the fragment is rendered by transparency blending with a background fragment.

[0149] In some optional embodiments, if a fragment does not have other fragments that have an occlusion relationship, the fragment can be directly rendered by transparency blending with a background fragment.

[0150] By rendering each fragment of the target rendering point by transparency blending, the clear and stable effect of the object contour can be achieved. Referring to FIG. 6, the left drawing is the rendering effect after the collected point cloud is rendered according to the depth rejection, and the contour has many burrs and is not clear. The right drawing is the rendering effect of the scheme of the embodiment of the present disclosure, and the object contour is clear. Figure 5

[0151] The steps 151 to 153 disclose the implementation manner of rendering the fragments by transparency blending. By using the manner, the sequential independent transparency blending rendering can be achieved without sorting the fragments according to the distance, and the implementation is more convenient.

[0152] Corresponding to the foregoing embodiments of the point cloud rendering method, the present disclosure also provides corresponding embodiments of a point cloud rendering device.

[0153] Exemplary apparatus

[0154] Figure 6 ​A structural schematic diagram of one embodiment of a point cloud rendering device of the present disclosure, which is applied on an electronic device (such as a computer system, a server), as shown in Figure 6 The device comprises:

[0155] A first determining module 61 is configured to determine a thickness threshold of a to-be-processed point cloud, the thickness threshold being used to indicate a depth range of points participating in transparency blending when rendering a target scene using the to-be-processed point cloud;

[0156] A first rendering module 62 is configured to perform a depth rendering operation of the target scene to obtain an original depth value of the target scene, the depth rendering operation comprising: setting a state of depth testing to an open state, setting a depth write state of the to-be-processed point cloud to an open state, and closing transparency blending;

[0157] A first calculating module 63 is configured to obtain a target depth value of the target scene based on the original depth value of the target scene and the corresponding thickness threshold, and write the target depth value of the target scene into a depth buffer;

[0158] A depth testing module 64 is configured to perform depth testing based on the target depth value of the target scene to obtain a target rendering point set passing the depth testing, the target rendering point set comprising at least one target rendering point in the to-be-processed point cloud;

[0159] A second rendering module 65 is configured to render the target rendering point set in a transparency blending manner.

[0160] Figure 7 A structural schematic diagram of another embodiment of a point cloud rendering device of the present disclosure, as shown in Figure 7 Based on the embodiment shown in Figure 6 In some optional implementations based on the embodiment shown in

[0161] A fitting sub-module 611 is configured to perform plane fitting on points in the to-be-processed point cloud to obtain a corresponding plane;

[0162] A calculating sub-module 612 is configured to calculate a plane rate of the plane based on a thickness value of the plane, the plane rate being used to indicate a proportion of points in the to-be-processed point cloud that are less than the thickness value in distance from the plane;

[0163] An iterating sub-module 613 is configured to iteratively update the thickness value of the plane to realize calculation of the plane rate of the plane until a preset condition is met, thereby obtaining the thickness threshold of the to-be-processed point cloud.

[0164] In still some optional implementations, the depth testing module 64 comprises:

[0165] The operation execution submodule 641 is configured to set the depth write state of the target scene as a closed operation, and set the depth test state as an open state.

[0166] The acquisition submodule 642 is configured to acquire a target depth value of the target scene from the depth buffer.

[0167] The test submodule 643 is configured to perform depth test on the target scene based on the acquired target depth value of the target scene.

[0168] In some optional embodiments, the device further includes:

[0169] The second calculation module 66 is configured to calculate a fragment blending weight of each target rendering point in the at least one target rendering point, the fragment blending weight indicating a blending weight of each fragment corresponding to each target rendering point.

[0170] The storage module 67 is configured to store the blending weight of each fragment corresponding to each target rendering point in the transparency channel.

[0171] In some optional embodiments, the second rendering module 65 includes:

[0172] The determination submodule 651 is configured to determine whether each fragment has other fragments having an occlusion relationship according to the three-dimensional coordinates of each fragment in each target rendering point.

[0173] The first rendering submodule 652 is configured to, in response to one fragment having other fragments having an occlusion relationship, perform rendering on the fragments having an occlusion relationship in a transparency blending manner based on a sequential-independent translucent rendering algorithm.

[0174] The second rendering submodule 653 is configured to, in response to one fragment not having other fragments having an occlusion relationship, perform transparency blending rendering on one fragment and a background fragment.

[0175] In some optional embodiments, the second calculation module 66 includes:

[0176] The distance calculation submodule 661 is configured to calculate a distance between each fragment and a center fragment of a corresponding target rendering point.

[0177] The normalization submodule 662 is configured to perform normalization processing on the distance corresponding to each fragment to obtain a fragment blending weight of each fragment.

[0178] In some optional embodiments, the device further includes:

[0179] The point cloud acquisition module 68 is configured to perform point cloud acquisition on the target scene to obtain an original point cloud.

[0180] The processing module 69 is configured to pre-process the original point cloud to obtain a pre-processed point cloud.

[0181] The segmentation module 70 is configured to segment the pre-processed point cloud into at least one to-be-processed point cloud.

[0182] The modules and units in the device of the present disclosure can also be divided into more granularities according to actual needs, and can be specifically set according to actual needs.

[0183] The device of the embodiment of the present disclosure can be used to implement the method of each embodiment of the present disclosure, and the two are corresponding to each other in specific implementation, and the specific implementation of the related parts is mutually referred to, which will not be described here.

[0184] Exemplary electronic device, computer program product, and computer readable storage medium

[0185] The embodiment of the present disclosure also provides an electronic device, which comprises a memory for storing a computer program, and a processor for executing the computer program stored in the memory, and when the computer program is executed, the point cloud rendering method of any one of the embodiments of the present disclosure is implemented.

[0186] Next, the electronic device according to the embodiment of the present disclosure will be described with reference to Figure 8 The electronic device according to the embodiment of the present disclosure can implement the device of the method of the embodiment of the present disclosure. Figure 8 The structure diagram of the electronic device provided by the illustrative embodiment of the present disclosure is shown in Figure 8 The electronic device comprises one or more processors 81, a memory 82 of one or more computer readable storage media, and a computer program stored on the memory and executable on the processor. When the program of the memory 82 is executed, the point cloud rendering method described above can be implemented.

[0187] Specifically, in actual application, the electronic device can further comprise an input device 83, an output device 84 and the like. These components are interconnected through a bus system and / or other forms of connection mechanism (not shown). Those skilled in the art can understand that Figure 8 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device, and can comprise more or fewer components than the figure, or some components, or different arrangement of components. Among them:

[0188] The processor 81 can be a central processing unit (CPU) or other forms of processing units with point cloud rendering capability and / or instruction execution capability. By running or executing software programs and / or modules stored in the memory 82 and calling data stored in the memory 82, various functions and data processing are performed, so as to monitor the whole electronic device.

[0189] The memory 82 can store one or more computer program products. The memory can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can include, for example, random access memory (RAM), cache memory, and / or the like. The non-volatile memory can include, for example, read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program products can be stored on the computer-readable storage media, and the processor 81 can execute the computer program products to implement the point cloud rendering method of various embodiments of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, and the like can also be stored in the computer-readable storage media.

[0190] The input device 83 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0191] The output device 84 can output various information to the outside, including determined distance information, direction information, and the like. The output device 84 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0192] The electronic device can also include a power supply to supply power to each component, and the power supply management system can be connected to the functional modules of the processor 81, so as to realize the functions of managing charging, discharging, and power consumption management through the power supply management system. The power supply can also include one or more direct current or alternating current power supplies, a recharging system, a power supply failure detection circuit, a power supply converter or inverter, a power supply status indicator, and the like.

[0193] Of course, in order to simplify, Figure 8 Only some of the components related to the present disclosure in the electronic device are shown in the figure, and components such as buses, input / output interfaces, and the like are omitted. In addition, the electronic device can also include any other appropriate components according to specific application cases.

[0194] In addition to the above methods and devices, embodiments of the present disclosure can also be computer program products including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the point cloud rendering method according to various embodiments of the present disclosure described in the above “Exemplary Methods” section of the specification.

[0195] The computer program product can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server.

[0196] Furthermore, embodiments of the present disclosure can also be a computer readable storage medium, having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps described in the above "Exemplary Method" section of the present specification for the point cloud rendering method according to various embodiments of the present disclosure.

[0197] The computer readable storage medium can be any combination of one or more computer readable medium(s). The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0198] The above describes the basic principles of the present disclosure in combination with specific embodiments, but it should be noted that the advantages, benefits, effects and the like mentioned in the present disclosure are only examples and are not limiting, and these advantages, benefits, effects and the like cannot be considered as necessary for each embodiment of the present disclosure. In addition, the above specific details are only for the purpose of example and understanding, and the above details do not limit the present disclosure to the above specific details.

[0199] Each embodiment in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between each embodiment can be understood by mutual reference. For system embodiments, since they are basically corresponding to method embodiments, the description is relatively simple, and the relevant parts can be understood by referring to the part of the method embodiment.

[0200] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes ROM, RAM, magnetic disc or optical disc and various storage medium capable of storing program codes.

[0201] The methods and apparatus of this disclosure can be implemented in numerous ways, such as with software, hardware, firmware, or any combination of software, hardware, and firmware. The order of any steps described above is merely exemplary and the steps of the methods of this disclosure need not be performed in the order described unless otherwise specified. Additionally, in some embodiments, the methods of this disclosure can also be implemented as program instructions embodied on a recording medium. Thus, the disclosure also covers a recording medium storing program instructions to perform the methods of this disclosure.

[0202] The description of the disclosure is given by way of example and description, and is not exhaustive or limiting to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are chosen and described in order to best explain the principles of the disclosure and its practical application, and to enable others skilled in the art to understand the disclosure in order to design various embodiments with various modifications for particular uses.

Claims

1. A method of point cloud rendering, the method comprising: The method comprises the following steps: determining a thickness threshold of a to-be-processed point cloud, the thickness threshold being used to indicate a depth range of points participating in transparency blending when rendering a target scene using the to-be-processed point cloud; performing a depth rendering operation of the target scene to obtain an original depth value of the target scene, the depth rendering operation comprising: setting a state of depth testing to an open state, setting a depth write state of the to-be-processed point cloud to an open state, and closing transparency blending; based on the original depth value of the target scene and the corresponding thickness threshold, obtaining a target depth value of the target scene, and writing the target depth value of the target scene into a depth buffer, the target depth value being obtained by summing the original depth value of the target scene and the corresponding thickness threshold; based on the target depth value of the target scene, performing depth testing to obtain a target rendering point set passing the depth testing, the target rendering point set comprising at least one target rendering point in the to-be-processed point cloud; rendering the target rendering point set in a transparency blending manner.

2. The method of claim 1, wherein, The method comprises the following steps: plane fitting is performed on points in the to-be-processed point cloud to obtain a corresponding plane; based on a thickness value of the plane, a plane rate of the plane is calculated, the plane rate being used to indicate a number ratio of points in the to-be-processed point cloud that are at a distance less than the thickness value from the plane; iteratively updating the thickness value of the plane to realize calculation of the plane rate of the plane until a preset condition is met, thereby obtaining the thickness threshold of the to-be-processed point cloud.

3. The method according to any of claims 1-2, characterized by, The method comprises the following steps: setting a depth write state of the target scene to a closed operation, and setting a state of depth testing to an open state; obtaining the target depth value of the target scene from the depth buffer; based on the obtained target depth value of the target scene, performing depth testing on the target scene.

4. The method according to any of claims 1-2, characterized by, The method further comprises the following steps before the rendering of the target rendering point set in a transparency blending manner: calculating a fragment blending weight of each target rendering point in the at least one target rendering point, the fragment blending weight indicating a blending weight of each fragment corresponding to each target rendering point; storing the blending weight of each fragment corresponding to each target rendering point in a transparency channel.

5. The method of claim 4, wherein, The method comprises the following steps: determining whether each fragment has other fragments having an occlusion relationship according to three-dimensional coordinates of each fragment in each target rendering point; in response to one fragment having other fragments having an occlusion relationship, rendering each fragment having an occlusion relationship in a transparency blending manner based on a sequential-independent translucent rendering algorithm; in response to one fragment not having other fragments having an occlusion relationship, performing transparency blending rendering on the one fragment and a background fragment.

6. The method of claim 4, wherein, The method comprises the following steps: calculating a distance between each fragment and a center fragment corresponding to a target rendering point; normalizing the distance corresponding to each fragment to obtain a fragment blending weight of each fragment.

7. The method of any of claims 1-2, wherein, The to-be-processed point cloud is obtained by the following manner: Point cloud collection is performed on the target scene to obtain an original point cloud; The original point cloud is preprocessed to obtain a preprocessed point cloud; The preprocessed point cloud is segmented into at least one to-be-processed point cloud.

8. A point cloud rendering apparatus, characterized by comprising: Comprise: A first determination module is configured to determine a thickness threshold of a to-be-processed point cloud, the thickness threshold being used to indicate a depth range of points participating in transparency blending when the to-be-processed point cloud is used to render the target scene; A first rendering module is configured to perform a depth rendering operation of the target scene to obtain an original depth value of the target scene, the depth rendering operation comprising: setting a state of depth test to an open state, setting a depth write state of the to-be-processed point cloud to an open state, and closing transparency blending; A first calculation module is configured to obtain a target depth value of the target scene based on the original depth value of the target scene and the corresponding thickness threshold, and write the target depth value of the target scene into a depth buffer, the target depth value being obtained by summing the original depth value of the target scene and the corresponding thickness threshold; A depth test module is configured to perform depth test based on the target depth value of the target scene to obtain a target rendering point set passing the depth test, the target rendering point set comprising at least one target rendering point in the to-be-processed point cloud; A second rendering module is configured to render the target rendering point set in a transparency blending manner.

9. A computer-readable storage medium, characterized in that, The storage medium stores computer program instructions, and the computer program instructions are executed to implement the method in any one of claims 1-7.

10. An electronic device, comprising: The electronic device comprises: A memory is configured to store a computer program product; A processor is configured to execute the computer program product stored in the memory, and the computer program product is executed to implement the method in any one of claims 1-7.

11. A computer program product comprising computer program instructions, characterised in that, The computer program instructions are executed by the processor to implement the method in any one of claims 1-7. The computer program instructions are executed by the processor to implement the method in any one of claims 1-7.

Images