Point cloud data display method and device based on digital twinning, equipment and medium

Abstract

The invention provides a point cloud data display method and device based on digital twinning, equipment and a medium. The method comprises the steps of obtaining to-be-displayed point cloud data; wherein the to-be-displayed point cloud data is obtained by performing de-noising processing and thinning processing on original point cloud data; performing interpolation processing on the to-be-displayed point cloud data to obtain interpolated point cloud data; and displaying the interpolated point cloud data in the digital twinning scene in real time. According to the method and the device, the to-be-displayed point cloud data is obtained by performing denoising processing and thinning processing on the original point cloud data, so that the transmission data volume of the to-be-displayed point cloud data is reduced, and the bandwidth occupation is reduced. According to the method, after the to-be-displayed point cloud data is obtained, interpolation processing is firstly performed on the to-be-displayed point cloud data to obtain the interpolated point cloud data, and then the interpolated point cloud data is displayed in real time in the digital twinning scene, so that details of the finally displayed point cloud data are enriched through the processing, a smoother playing frame rate is kept, and a better impression is provided.

Description

Technical Field

[0001] The present application relates to the field of digital twin technology, and in particular to a point cloud data display method, device, equipment, and medium based on digital twin. Background Art

[0002] With the rapid development of science and technology, the performance of lidar products is becoming more and more powerful. The laser beam of mainstream products has gradually transitioned from 32 lines to 64 lines or even 128 lines. The resolution has also increased accordingly, and the number of data points in each frame of the point cloud has also become larger. If the point cloud data scanned by the vehicle is to be displayed in real time, the communication link will take up a lot of traffic.

[0003] The current solution is to reduce the frame rate of transmission to reduce bandwidth usage. However, since the frame rate of LiDAR is not high to begin with, if the frame rate is further reduced, the point cloud data displayed on the digital twin end will be unnatural and affect the viewing experience. Summary of the invention

[0004] In order to solve one of the above-mentioned technical defects, the present application provides a point cloud data display method, device, equipment, and medium based on digital twins.

[0005] In a first aspect, the present application provides a point cloud data display method based on digital twins, the method comprising: Obtaining point cloud data to be displayed; wherein the point cloud data to be displayed is obtained by performing denoising and thinning processing on the original point cloud data; Perform interpolation processing on the point cloud data to be displayed to obtain interpolated point cloud data; Display interpolated point cloud data in real time in the digital twin scene.

[0006] Optionally, denoising processing includes: Determine the neighborhood range of each point in the original point cloud data; among them, any point in the original point cloud data The radius of the neighborhood is , is the average distance of the original point cloud data, The sum of the distances between all points in the original point cloud data and their nearest neighbor points / the total number of points in the original point cloud data, is the first multiple, ; Determine the denoised value of each point in the original point cloud data according to the neighborhood range; The denoised value of , For any point Point identifiers within the neighborhood of For point The value of For point The weight of .

[0007] Optionally, ; in, For point To any point The contribution weight of , For any point The value of For any point The standard deviation of , For any point The total number of points in the neighborhood of For any point The mean of .

[0008] Optionally, the thinning process includes: Determine the voxel grid edge length as ,in, is the average distance of the point cloud data after denoising, The sum of the distances between all points in the denoised point cloud data and their nearest neighbor points / the total number of points in the denoised point cloud data, is the second multiple, Determined according to the density of the point cloud data after denoising; Determine the voxel index of each point in the denoised point cloud data; where the voxel index of any point in the denoised point cloud data is , is the voxel index of any point on the horizontal axis in the point cloud data after denoising, is the rounded value of the quotient of the horizontal axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid. is the voxel index of any point on the vertical axis in the point cloud data after denoising, is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid. is the voxel index of any point in the point cloud data on the vertical axis after denoising, It is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid; In the denoised point cloud data, voxels are formed with voxel indices as keys; wherein the voxels with any voxel index as a key include all points with any voxel index; The points in each voxel are thinned to one point.

[0009] Optionally, the points in each voxel are thinned to one point, including: Thin out any point in a voxel into a single point ; in, is the key of any voxel, is the total number of points in any voxel, is the point identifier in any voxel, The midpoint of any voxel The value of .

[0010] Optionally, interpolation processing is performed on the point cloud data to be displayed to obtain interpolated point cloud data, including: Determine the neighborhood range of each point in the point cloud data to be displayed; among them, any point in the point cloud data to be displayed The radius of the neighborhood is , is the average distance of the point cloud data to be displayed, The sum of the distances between all points in the point cloud data to be displayed and their nearest neighbor points / the total number of points in the point cloud data to be displayed, is the first multiple, ; Insert the predicted value at each insertion point in the neighborhood of each point in the point cloud data to be displayed; The predicted value of , Any point in the point cloud data to be displayed The identifier of the point in the neighborhood of For point The value of For point The normalized weight of .

[0011] Optionally, ; in, For point The weight of , To insert point to point The distance is the third parameter, .

[0012] In a second aspect of the present application, a point cloud data display device based on digital twins is provided, the device comprising: An acquisition module is used to acquire point cloud data to be displayed; wherein the point cloud data to be displayed is obtained by performing denoising and thinning processing on the original point cloud data; An interpolation module is used to perform interpolation processing on the point cloud data to be displayed acquired by the acquisition module to obtain interpolated point cloud data; The display module is used to display the interpolated point cloud data obtained by the interpolation module in real time in the digital twin scene.

[0013] In a third aspect of the present application, an electronic device is provided, including: Memory; Processor; and Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect above.

[0014] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored; the computer program is executed by a processor to implement the method described in the first aspect above.

[0015] The present application provides a method, device, equipment, and medium for displaying point cloud data based on digital twins. The method includes: obtaining point cloud data to be displayed; wherein the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data; interpolating the point cloud data to be displayed to obtain interpolated point cloud data; and displaying the interpolated point cloud data in real time in the digital twin scene. In the method provided in the present application, the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data. This processing reduces the amount of transmission data of the point cloud data to be displayed and reduces bandwidth occupancy. After obtaining the point cloud data to be displayed, the point cloud data to be displayed is first interpolated to obtain interpolated point cloud data, and then the interpolated point cloud data is displayed in real time in the digital twin scene. This processing enriches the details of the final displayed point cloud data, thereby maintaining a smoother playback frame rate, thereby providing a better viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of a process flow of a point cloud data display method based on digital twins provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a point cloud data display device based on digital twins provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0018] In the process of implementing this application, the inventor found that with the rapid development of science and technology, the performance of LiDAR products is becoming more and more powerful. The laser beam of mainstream products has gradually transitioned from 32 lines to 64 lines or even 128 lines. The resolution has also increased accordingly, and the number of data points in each frame of the point cloud has also become larger. If the point cloud data scanned by the vehicle is to be displayed in real time, the communication link will take up a lot of traffic. The current solution is to reduce the frame rate of transmission to reduce bandwidth usage. However, since the frame rate of LiDAR is not high, if the frame rate is reduced further, the point cloud data displayed on the digital twin end will be unnatural and affect the viewing experience.

[0019] In response to the above problems, a method, device, equipment, and medium for displaying point cloud data based on digital twins are provided in an embodiment of the present application. The method includes: obtaining point cloud data to be displayed; wherein the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data; interpolating the point cloud data to be displayed to obtain interpolated point cloud data; and displaying the interpolated point cloud data in real time in the digital twin scene. In the method provided in the present application, the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data. This processing reduces the amount of transmission data of the point cloud data to be displayed and reduces bandwidth occupancy. After obtaining the point cloud data to be displayed, the point cloud data to be displayed is first interpolated to obtain interpolated point cloud data, and then the interpolated point cloud data is displayed in real time in the digital twin scene. This processing enriches the details of the final displayed point cloud data, thereby maintaining a smoother playback frame rate, thereby providing a better viewing experience.

[0020] See also Figure 1 , this embodiment provides a point cloud data display method based on digital twins, and the implementation process of the method is as follows: 101, obtaining point cloud data to be displayed.

[0021] The point cloud data to be displayed is obtained by denoising and thinning the original point cloud data.

[0022] In this step, the point cloud data to be displayed can be obtained from the data server. That is, after the point cloud data acquisition device obtains the original point cloud data, it will store it in the data server. Whenever the point cloud data needs to be displayed, the point cloud data display method device based on digital twins will execute Figure 1The method shown obtains point cloud data to be displayed from a data server, and displays the point cloud data in real time in a digital twin scene.

[0023] After the point cloud data acquisition device acquires the original point cloud data, it does not directly store the original point cloud data in the data server, but first performs denoising and thinning processing on the original point cloud data, and then stores the denoised and thinned point cloud data in the data server. The point cloud data to be displayed obtained in step 101 is the point cloud data obtained by denoising and thinning the original point cloud data.

[0024] The denoising process is as follows: A1, determine the neighborhood range of each point in the original point cloud data.

[0025] Among them, any point in the original point cloud data The neighborhood range can be a spherical range, and any point in the original point cloud data The radius of the neighborhood range can be .

[0026] is the average distance of the original point cloud data. It can be obtained by the nearest distance of each point in the original point cloud data, for example, The sum of the distances between all points in the original point cloud data and their nearest neighbor points / the total number of points in the original point cloud data.

[0027] For example, use k-dtree to index point cloud data, obtain the distance between all points in the original point cloud data and their nearest neighbor points through k-dtree, and then calculate the average distance. Assume that the original point cloud data has points, and the distance between each point and its nearest neighbor is , , , ,but .

[0028] The distance may be a Euclidean distance.

[0029] in addition, is the first multiple, .

[0030] A2, determine the denoised value of each point in the original point cloud data according to the neighborhood range.

[0031] Among them, any point The denoised value of , For any point Point identifiers within the neighborhood of For point The value of For point The weight of .

[0032] .

[0033] in, For point To any point The contribution weight of , For any point The value of For any point The standard deviation of , For any point The total number of points in the neighborhood of For any point The mean of .

[0034] therefore, is the normalized value of the contribution weight.

[0035] In specific implementation, the above denoising process is: 1.1 Select a point from the original point cloud data that was not selected by the denoising process (such as point ); 1.2 Determine the point The neighborhood range of , where the radius of the neighborhood range is ; 1.3 Determine the point Each point-to-point in the neighborhood The contribution weight of Any point in the neighborhood Point Contribution weight ; 1.4 Determine the normalized value of the contribution weight and use it as the point The weight of, for example, point The normalized value of the contribution weight ; 1.5 Determine the point The denoised value of ; 1.6 Re-execute step 1.1 and subsequent steps until all points in the original point cloud data are selected in the denoising process. At this time, the denoising process is completed and the denoised point cloud data is obtained.

[0036] In addition, the thinning process can be implemented by using the voxel grid downsampling method, which has high computational efficiency, is suitable for processing a large amount of point cloud data, and can retain the global features of the point cloud. The specific process is as follows: B1, determine the side length of the voxel grid .

[0037] in, is the average distance of the points in the denoised point cloud data. It can be obtained by the nearest distance of each point in the denoised point cloud data, for example The sum of the distances between all points in the denoised point cloud data and their nearest neighbor points / the total number of points in the denoised point cloud data.

[0038] For example, use k-dtree to index point cloud data, obtain the distance between all points in the denoised point cloud data and their nearest neighbor points through k-dtree, and then calculate the average distance. points, and the distance between each point and its nearest neighbor is , , , ,but .

[0039] The distance may be a Euclidean distance.

[0040] in addition, is the second multiple, Determined by the density of the point cloud data after denoising. .

[0041] B2, determine the voxel index of each point in the point cloud data after denoising.

[0042] Among them, the voxel index of any point in the denoised point cloud data is , is the voxel index of any point on the horizontal axis in the point cloud data after denoising, is the rounded value of the quotient of the horizontal axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid. is the voxel index of any point on the vertical axis in the point cloud data after denoising, is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid. is the voxel index of any point in the point cloud data on the vertical axis after denoising, It is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid.

[0043] For example, if the length of the edge of the voxel grid is , the voxel index of each point in the denoised point cloud data is , if the coordinate value of any point in the point cloud data after denoising is ,in, is the horizontal axis of any point in the denoised point cloud data (i.e. axis) coordinate value, is the vertical axis of any point in the denoised point cloud data (i.e. axis) coordinate value, is the vertical axis of any point in the denoised point cloud data (i.e. axis) coordinate value, then , , In this way, if the point cloud data after denoising is The maximum coordinate value of the axis is , in the point cloud data after denoising, the vertical axis (i.e. The maximum coordinate value of the axis is , in the point cloud data after denoising, the vertical axis (i.e. The maximum coordinate value of the axis is , then the horizontal axis (i.e. The number of grids in the axis direction is , the vertical axis (i.e. The number of grids in the axis direction is , the vertical axis (i.e. The number of grids in the axis direction is .

[0044] B3, in the denoised point cloud data, voxels are formed using voxel indices as keys.

[0045] Among them, the voxels with any voxel index as the key include all points with any voxel index.

[0046] For example, using voxel indexing ( ) as the key to save the point data in the grid .

[0047] B4, thin out the points in each voxel into one point.

[0048] For example, thinning the points in any voxel to a single point .

[0049] in, is the key of any voxel, is the total number of points in any voxel, is the point identifier in any voxel, The midpoint of any voxel The value of .

[0050] In step B4, the representative points within the voxel are determined. In each voxel, a representative point is retained as the downsampled point, such as using the centroid of the point set. As a representative point.

[0051] The quality of the data after thinning determines the size of the voxel grid, such as the density of the point cloud data after denoising. , the amount of data after thinning is about 1 / 4 of the amount of point cloud data after denoising.

[0052] Since the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data, the amount of point cloud data after denoising and thinning will be greatly reduced compared with the original point cloud data. This will reduce the amount of data transmitted from the point cloud data acquisition device to the data server after denoising and thinning, and reduce bandwidth occupancy. At the same time, it will also reduce the amount of data transmitted from the data server to the point cloud data display device based on digital twins, and reduce bandwidth occupancy.

[0053] 102 , interpolation processing is performed on the point cloud data to be displayed to obtain interpolated point cloud data.

[0054] Step 102 can be implemented by using the inverse distance weighted interpolation method, which has good local detail restoration and high computational efficiency, and is suitable for fast computation scenarios. The implementation process is as follows: 102-1, determine the neighborhood range of each point in the point cloud data to be displayed.

[0055] Among them, any point in the point cloud data to be displayed The radius of the neighborhood is .

[0056] is the average distance of the point cloud data to be displayed, The sum of the distances between all points in the point cloud data to be displayed and their nearest neighboring points / the total number of points in the point cloud data to be displayed.

[0057] For example, use k-dtree to index point cloud data, obtain the distance between all points in the point cloud data to be displayed and their nearest neighbor points through k-dtree, and then calculate the average distance. points, and the distance between each point and its nearest neighbor is , , , ,but .

[0058] The distance may be a Euclidean distance.

[0059] in addition, is the first multiple, .

[0060] 102-2, inserting predicted values ​​at each insertion point in the neighborhood of each point in the point cloud data to be displayed.

[0061] Among them, any point The predicted value of , Any point in the point cloud data to be displayed The identifier of the point in the neighborhood of For point The value of For point The normalized weight of .

[0062] in, .

[0063] in, For point The weight of , To insert point to point The distance is the third parameter, , The sensitivity used to adjust the distance weight can be determined based on the characteristics of the data and experience, such as .

[0064] In specific implementation, the above interpolation process is as follows: 2.1 Select a point that is not selected in the interpolation process from the point cloud data to be displayed (such as point );2.2 Determine the point The neighborhood range of , where the radius of the neighborhood range is ;2.3 Determine the point The weight of each point in the neighborhood, for example, Any point in the neighborhood Weight ;2.4 Determine the normalized value of the weight ,like ; 2.5 Determine the point The predicted value of ;2.6 Insert the predicted value at the insertion point ; 2.7 Re-execute step 2.1 and subsequent steps until all points in the point cloud data to be displayed are selected in the interpolation process, at which time the interpolation process is completed and the interpolated point cloud data is obtained.

[0065] 103, real-time display of interpolated point cloud data in the digital twin scene.

[0066] The point cloud data display method based on digital twins provided in this embodiment reduces the intermediate transmission data appropriately and reduces the bandwidth occupancy by thinning the data, but keeps the overall frame rate unchanged. During playback, interpolation processing is performed to enrich the details of the data, thereby maintaining a smoother playback frame rate and providing a better viewing experience.

[0067] The present embodiment provides a point cloud data display method based on digital twins, which obtains point cloud data to be displayed; wherein the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data; the point cloud data to be displayed is interpolated to obtain interpolated point cloud data; and the interpolated point cloud data is displayed in real time in the digital twin scene. In the method provided in the present embodiment, the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data. This processing reduces the amount of transmission data of the point cloud data to be displayed and reduces bandwidth occupancy. After obtaining the point cloud data to be displayed, the point cloud data to be displayed is first interpolated to obtain interpolated point cloud data, and then the interpolated point cloud data is displayed in real time in the digital twin scene. This processing enriches the details of the final displayed point cloud data, thereby maintaining a smoother playback frame rate, thereby providing a better viewing experience.

[0068] Based on the same inventive concept as the point cloud data display method based on digital twins, this embodiment provides a point cloud data display device based on digital twins, see Figure 2 , the device comprises: The acquisition module 201 is used to acquire the point cloud data to be displayed, wherein the point cloud data to be displayed is obtained by performing denoising and thinning processing on the original point cloud data.

[0069] The interpolation module 202 is used to perform interpolation processing on the point cloud data to be displayed acquired by the acquisition module 201 to obtain interpolated point cloud data.

[0070] The display module 203 is used to display the interpolated point cloud data obtained by the interpolation module 202 in real time in the digital twin scene.

[0071] Among them, the denoising process includes: Determine the neighborhood range of each point in the original point cloud data. The radius of the neighborhood is , is the average distance of the original point cloud data, The sum of the distances between all points in the original point cloud data and their nearest neighbor points / the total number of points in the original point cloud data, is the first multiple, .

[0072] Determine the denoised value of each point in the original point cloud data based on the neighborhood range. The denoised value of , For any point Point identifiers within the neighborhood of For point The value of For point The weight of .

[0073] in, .

[0074] in, For point To any point The contribution weight of , For any point The value of For any point The standard deviation of , For any point The total number of points in the neighborhood of For any point The mean of .

[0075] Among them, the thinning process includes: Determine the voxel grid edge length as ,in, is the average distance of the point cloud data after denoising, The sum of the distances between all points in the denoised point cloud data and their nearest neighbor points / the total number of points in the denoised point cloud data, is the second multiple, Determined according to the density of the point cloud data after denoising.

[0076] Determine the voxel index of each point in the denoised point cloud data. The voxel index of any point in the denoised point cloud data is , is the voxel index of any point on the horizontal axis in the point cloud data after denoising, is the rounded value of the quotient of the horizontal axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid. is the voxel index of any point on the vertical axis in the point cloud data after denoising, is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid. is the voxel index of any point in the point cloud data on the vertical axis after denoising, It is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid.

[0077] In the denoised point cloud data, voxels are formed with voxel indices as keys, wherein a voxel with any voxel index as a key includes all points with any voxel index.

[0078] The points in each voxel are thinned to one point.

[0079] Among them, the points in each voxel are thinned into one point, including: Thin out any point in a voxel into a single point .

[0080] in, is the key of any voxel, is the total number of points in any voxel, is the point identifier in any voxel, The midpoint of any voxel The value of .

[0081] The interpolation module 202 is used to determine the neighborhood range of each point in the point cloud data to be displayed. The radius of the neighborhood is , is the average distance of the point cloud data to be displayed, The sum of the distances between all points in the point cloud data to be displayed and their nearest neighbor points / the total number of points in the point cloud data to be displayed, is the first multiple, . Insert the predicted value at each insertion point in the neighborhood of each point in the point cloud data to be displayed. The predicted value of , Any point in the point cloud data to be displayed The identifier of the point in the neighborhood of For point The value of For point The normalized weight of .

[0082] in, .

[0083] in, For point The weight of , To insert point to point The distance is the third parameter, .

[0084] The device provided in this embodiment obtains point cloud data to be displayed; wherein the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data; the point cloud data to be displayed is interpolated to obtain interpolated point cloud data; and the interpolated point cloud data is displayed in real time in the digital twin scene. The point cloud data to be displayed is obtained by denoising and thinning the original point cloud data. This processing reduces the amount of transmission data of the point cloud data to be displayed and reduces bandwidth occupancy. After obtaining the point cloud data to be displayed, the point cloud data to be displayed is first interpolated to obtain interpolated point cloud data, and then the interpolated point cloud data is displayed in real time in the digital twin scene. This processing enriches the details of the point cloud data finally displayed, thereby maintaining a smoother playback frame rate, thereby providing a better viewing experience.

[0085] With the same inventive concept as the point cloud data display method based on digital twins, this embodiment provides an electronic device, such as Figure 3 As shown, it includes: a memory 301, a processor 302, and a computer program.

[0086] Among them, the computer program is stored in the memory 301 and is configured to be executed by the processor 302 to implement the above-mentioned point cloud data display method based on digital twins.

[0087] Specifically, Obtain the point cloud data to be displayed, wherein the point cloud data to be displayed is obtained by performing denoising and thinning processing on the original point cloud data.

[0088] Interpolation processing is performed on the point cloud data to be displayed to obtain interpolated point cloud data.

[0089] Display interpolated point cloud data in real time in the digital twin scene.

[0090] Optionally, denoising processing includes: Determine the neighborhood range of each point in the original point cloud data. The radius of the neighborhood is , is the average distance of the original point cloud data, The sum of the distances between all points in the original point cloud data and their nearest neighbor points / the total number of points in the original point cloud data, is the first multiple, .

[0091] Determine the denoised value of each point in the original point cloud data based on the neighborhood range. The denoised value of , For any point Point identifiers within the neighborhood of For point The value of For point The weight of .

[0092] Optionally, .

[0093] in, For point To any point The contribution weight of , For any point The value of For any point The standard deviation of , For any point The total number of points in the neighborhood of For any point The mean of .

[0094] Optionally, the thinning process includes: Determine the voxel grid edge length as ,in, is the average distance of the point cloud data after denoising, The sum of the distances between all points in the denoised point cloud data and their nearest neighbor points / the total number of points in the denoised point cloud data, is the second multiple, Determined according to the density of the point cloud data after denoising.

[0095] Determine the voxel index of each point in the denoised point cloud data. The voxel index of any point in the denoised point cloud data is , is the voxel index of any point on the horizontal axis in the point cloud data after denoising, is the rounded value of the quotient of the horizontal axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid. is the voxel index of any point on the vertical axis in the point cloud data after denoising, is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid. is the voxel index of any point in the point cloud data on the vertical axis after denoising, It is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid.

[0096] In the denoised point cloud data, voxels are formed with voxel indices as keys, wherein a voxel with any voxel index as a key includes all points with any voxel index.

[0097] The points in each voxel are thinned to one point.

[0098] Optionally, the points in each voxel are thinned to one point, including: Thin out any point in a voxel into a single point .

[0099] in, is the key of any voxel, is the total number of points in any voxel, is the point identifier in any voxel, The midpoint of any voxel The value of .

[0100] Optionally, interpolation processing is performed on the point cloud data to be displayed to obtain interpolated point cloud data, including: Determine the neighborhood range of each point in the point cloud data to be displayed. The radius of the neighborhood is , is the average distance of the point cloud data to be displayed, The sum of the distances between all points in the point cloud data to be displayed and their nearest neighbor points / the total number of points in the point cloud data to be displayed, is the first multiple, .

[0101] Insert the predicted value at each insertion point in the neighborhood of each point in the point cloud data to be displayed. The predicted value of , Any point in the point cloud data to be displayed The identifier of the point in the neighborhood of For point The value of For point The normalized weight of .

[0102] Optionally, .

[0103] in, For point The weight of , To insert point to point The distance is the third parameter, .

[0104] The electronic device provided by the present embodiment has a computer program executed by a processor to obtain point cloud data to be displayed; wherein the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data; the point cloud data to be displayed is interpolated to obtain interpolated point cloud data; and the interpolated point cloud data is displayed in real time in the digital twin scene. wherein the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data, and such processing reduces the amount of transmission data of the point cloud data to be displayed and reduces bandwidth occupancy. after obtaining the point cloud data to be displayed, the point cloud data to be displayed is first interpolated to obtain interpolated point cloud data, and then the interpolated point cloud data is displayed in real time in the digital twin scene, and such processing enriches the details of the point cloud data finally displayed, thereby maintaining a smoother playback frame rate, thereby providing a better viewing experience.

[0105] With the same inventive concept as the point cloud data display method based on digital twins, this embodiment provides a computer-readable storage medium, and a computer program is stored thereon. The computer program is executed by a processor to implement the above-mentioned point cloud data display method based on digital twins.

[0106] Specifically, Obtain the point cloud data to be displayed, wherein the point cloud data to be displayed is obtained by performing denoising and thinning processing on the original point cloud data.

[0107] Interpolation processing is performed on the point cloud data to be displayed to obtain interpolated point cloud data.

[0108] Display interpolated point cloud data in real time in the digital twin scene.

[0109] Optionally, denoising processing includes: Determine the neighborhood range of each point in the original point cloud data. The radius of the neighborhood is , is the average distance of the original point cloud data, The sum of the distances between all points in the original point cloud data and their nearest neighbor points / the total number of points in the original point cloud data, is the first multiple, .

[0110] Determine the denoised value of each point in the original point cloud data based on the neighborhood range. The denoised value of , For any point Point identifiers within the neighborhood of For point The value of For point The weight of .

[0111] Optionally, .

[0112] in, For point To any point The contribution weight of , For any point The value of For any point The standard deviation of , For any point The total number of points in the neighborhood of For any point The mean of .

[0113] Optionally, the thinning process includes: Determine the voxel grid edge length as ,in, is the average distance of the point cloud data after denoising, The sum of the distances between all points in the denoised point cloud data and their nearest neighbor points / the total number of points in the denoised point cloud data, is the second multiple, Determined according to the density of the point cloud data after denoising.

[0114] Determine the voxel index of each point in the denoised point cloud data. The voxel index of any point in the denoised point cloud data is , is the voxel index of any point on the horizontal axis in the point cloud data after denoising, is the rounded value of the quotient of the horizontal axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid. is the voxel index of any point on the vertical axis in the point cloud data after denoising, is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid. is the voxel index of any point in the point cloud data on the vertical axis after denoising, It is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after denoising and the side length of the voxel grid.

[0115] In the denoised point cloud data, voxels are formed with voxel indices as keys, wherein a voxel with any voxel index as a key includes all points with any voxel index.

[0116] The points in each voxel are thinned to one point.

[0117] Optionally, the points in each voxel are thinned to one point, including: Thin out any point in a voxel into a single point .

[0118] in, is the key of any voxel, is the total number of points in any voxel, is the point identifier in any voxel, The midpoint of any voxel The value of .

[0119] Optionally, interpolation processing is performed on the point cloud data to be displayed to obtain interpolated point cloud data, including: Determine the neighborhood range of each point in the point cloud data to be displayed. The radius of the neighborhood is , is the average distance of the point cloud data to be displayed, The sum of the distances between all points in the point cloud data to be displayed and their nearest neighbor points / the total number of points in the point cloud data to be displayed, is the first multiple, .

[0120] Insert the predicted value at each insertion point in the neighborhood of each point in the point cloud data to be displayed. The predicted value of , Any point in the point cloud data to be displayed The identifier of the point in the neighborhood of For point The value of For point The normalized weight of .

[0121] Optionally, .

[0122] in, For point The weight of , To insert point to point The distance is the third parameter, .

[0123] The computer-readable storage medium provided in this embodiment has a computer program executed by a processor to obtain point cloud data to be displayed; wherein the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data; the point cloud data to be displayed is interpolated to obtain interpolated point cloud data; and the interpolated point cloud data is displayed in real time in the digital twin scene. wherein the point cloud data to be displayed is obtained by denoising and thinning the original point cloud data, and such processing reduces the amount of transmission data of the point cloud data to be displayed and reduces bandwidth occupancy. after obtaining the point cloud data to be displayed, the point cloud data to be displayed is first interpolated to obtain interpolated point cloud data, and then the interpolated point cloud data is displayed in real time in the digital twin scene, and such processing enriches the details of the point cloud data finally displayed, thereby maintaining a smoother playback frame rate, thereby providing a better viewing experience.

[0124] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present application may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0125] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0126] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0128] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0129] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A point cloud data display method based on digital twins, characterized in that: The method comprises: Acquire point cloud data to be displayed; wherein the point cloud data to be displayed is obtained by performing denoising and thinning processing on the original point cloud data; Performing interpolation processing on the point cloud data to be displayed to obtain interpolated point cloud data; The interpolated point cloud data is displayed in real time in the digital twin scene.

2. The method according to claim 1, characterized in that The denoising process comprises: Determine the neighborhood range of each point in the original point cloud data; among them, any point in the original point cloud data The radius of the neighborhood is , is the average distance of the original point cloud data, The sum of the distances between all points in the original point cloud data and their nearest neighbor points / the total number of points in the original point cloud data, is the first multiple, ; Determine the denoised value of each point in the original point cloud data according to the neighborhood range; The denoised value of , For any point Point identifiers within the neighborhood of For point The value of For point The weight of .

3. The method according to claim 2, characterized in that ； in, For point To any point The contribution weight of , For any point The value of For any point The standard deviation of , For any point The total number of points in the neighborhood of For any point The mean of .

4. The method according to claim 1, characterized in that: The thinning process comprises: Determine the voxel grid edge length as ,in, is the average distance of the point cloud data after denoising, The sum of the distances between all points in the denoised point cloud data and their nearest neighbor points / the total number of points in the denoised point cloud data, is the second multiple, Determined according to the density of the point cloud data after denoising; Determine the voxel index of each point in the denoised point cloud data; where the voxel index of any point in the denoised point cloud data is , is the voxel index of any point in the point cloud data after the denoising process on the horizontal axis, is the rounded value of the quotient of the horizontal axis coordinate value of any point in the point cloud data after the denoising process and the side length of the voxel grid, is the voxel index of any point in the point cloud data after denoising on the vertical axis, is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after the denoising process and the side length of the voxel grid, is the voxel index of any point in the point cloud data after the denoising process on the vertical axis, is the rounded value of the quotient of the vertical axis coordinate value of any point in the point cloud data after the denoising process and the side length of the voxel grid; In the denoised point cloud data, voxels are formed with voxel indices as keys; wherein the voxels with any voxel index as a key include all points with any voxel index; The points in each voxel are thinned to one point.

5. The method according to claim 4, characterized in that The step of thinning the points in each voxel into one point includes: Thin out any point in a voxel into a single point ; in, is the key of any voxel, is the total number of points in any voxel, is the point identifier in any voxel, is the midpoint of any voxel The value of .

6. The method according to claim 1, characterized in that The interpolation processing is performed on the point cloud data to be displayed to obtain interpolated point cloud data, including: Determine the neighborhood range of each point in the point cloud data to be displayed; among them, any point in the point cloud data to be displayed The radius of the neighborhood is , is the average distance of the point cloud data to be displayed, The sum of the distances between all points in the point cloud data to be displayed and their nearest neighbor points / the total number of points in the point cloud data to be displayed, is the first multiple, ; Insert the predicted value at each insertion point in the neighborhood of each point in the point cloud data to be displayed; The predicted value of , Any point in the point cloud data to be displayed The identifier of the point in the neighborhood of , For point The value of For point The normalized weight of .

7. The method according to claim 6, characterized in that Said ; in, For point The weight of , To insert point to point The distance is the third parameter, .

8. A point cloud data display device based on digital twins, characterized in that: The device comprises: An acquisition module, used for acquiring point cloud data to be displayed; wherein the point cloud data to be displayed is obtained by performing denoising and thinning processing on the original point cloud data; An interpolation module, used for performing interpolation processing on the point cloud data to be displayed acquired by the acquisition module to obtain interpolated point cloud data; The display module is used to display the interpolated point cloud data obtained by the interpolation module in real time in the digital twin scene.

9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon; the computer program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images