3D Point Cloud Rendering Method, Device and Storage Medium Based on Enhanced Deformation Features

Abstract

The present invention discloses a three-dimensional point cloud rendering method, device and storage medium based on deformation feature enhancement, which relates to the technical field of point cloud rendering, and solves the technical problem that the rendering result cannot identify the weak deformation in the point cloud. The key technical solution is to smooth the input three-dimensional point cloud; extract enhanced normal vectors from the smoothed three-dimensional point cloud as surface deformation features; perform pseudo-color mapping on the deformation features to render a pseudo-color map. In view of the problem of poor visibility of surface weak deformation in the effect of the existing point cloud rendering method, the present application innovatively designs a three-dimensional point cloud rendering method with enhanced deformation features, effectively visualizing the surface weak deformation that cannot be clearly rendered by the existing rendering method. The degree of enhancement of the visual features of the weak deformation in the present application is better than that of the existing point cloud rendering method, and the gain for subsequent defect detection is also better than that of the existing point cloud rendering method.

Description

Technical Field

[0001] The present application relates to the technical field of point cloud rendering, and in particular, to a three-dimensional point cloud rendering method, device, and storage medium based on enhanced deformation features. Background Art

[0002] With the development of informatization, data in the three-dimensional point cloud format has been widely applied in many fields such as defect detection, virtual tourism, and digital cities. The rendering effect of three-dimensional point clouds has a crucial impact on the subsequent application effect. Three-dimensional point cloud rendering methods are often designed specifically for subsequent applications. According to different application requirements and the core technologies they adopt, point cloud rendering methods can be divided into two major categories: rendering methods based on level of detail (LOD) and rendering methods based on hole filling.

[0003] The main goal of the rendering method based on level of detail (LOD) is to reduce the computational complexity of rendering, thereby shortening the rendering time and achieving real-time rendering. Specifically, this type of method first dynamically divides the objects in the three-dimensional point cloud into multiple levels of detail according to the distance between the objects and the viewing point in the three-dimensional point cloud, the density relationship between coordinate points, or the semantic relationship between objects; then different rendering precisions are used for different levels of detail, that is, a higher rendering precision is used for the level of detail close to the viewing point, and a lower rendering precision is used for the level of detail far from the viewing point. The rendering method based on level of detail (LOD) can reduce the rendering calculation amount of the entire point cloud through the above operations, and achieve fast, even real-time point cloud rendering.

[0004] The main goal of the rendering method based on hole filling is to achieve a more realistic visual effect by filling the holes between coordinates in the point cloud. Specifically, this type of method first replaces the coordinate points with rectangular or triangular patches according to the direction of the coordinate points, and then dynamically expands and connects these patches according to the relationship between the neighborhoods of the coordinate points, so that adjacent patches are connected to each other. The rendering method based on hole filling can fill the holes between coordinate points through the above operations, thereby obtaining a more realistic visual effect compared to the original point cloud.

[0005] However, the rendering results obtained by the above rendering methods cannot identify the weak deformations in the point cloud. Summary of the Invention

[0006] The present application provides a three-dimensional point cloud rendering method, device, and storage medium based on enhanced deformation features, and its technical purpose is to enhance the visual features of the surface deformation of the point cloud, so that the rendering result can identify the weak deformations in the point cloud.

[0007] The above technical purpose of the present application is achieved through the following technical solutions:

[0008] A 3D point cloud rendering method based on enhanced deformation features, comprising:

[0009] S1: Smooth the input 3D point cloud to obtain a smoothed 3D point cloud;

[0010] S2: Extract deformation features from the smoothed 3D point cloud to obtain a deformation feature map;

[0011] S3: Perform pseudo-color mapping on the deformation feature map to obtain a pseudo-color image; wherein, the pseudo-color image is the output result obtained by rendering the 3D point cloud;

[0012] The step S1 includes:

[0013] S11: Calculate the support plane of the 3D point cloud, and at the same time perform coordinate rotation transformation on the 3D point cloud, and then make the support plane coincide with the XY plane of the coordinate axis to obtain a rotated 3D point cloud;

[0014] S12: Fit the main target in the rotated 3D point cloud through a quadratic surface to obtain a fitted surface, and eliminate the deformation of the coordinate axis in the Z direction caused by the fitted surface to obtain a smoothed 3D point cloud;

[0015] The step S2 includes:

[0016] S21: Enhance the smoothed 3D point cloud in the Z direction to obtain an enhanced 3D point cloud, and the enhancement coefficient is Z time ; Then extract the neighborhood point sets of all coordinate points in the enhanced 3D point cloud, and then estimate the local plane according to the neighborhood point sets to obtain a set of normal vectors of the 3D point cloud;

[0017] S22: Arrange the set of normal vectors into a matrix according to their corresponding coordinate point positions, and normalize the RGB channel values of each dimension of the matrix to between [0, 255] to obtain a deformation feature map; wherein, the RGB channel values respectively correspond to the XYZ coordinate values of the normal vectors of the coordinate points;

[0018] The step S3 includes:

[0019] S31: Reduce the dimension of the deformation feature map through the principal component analysis algorithm to obtain a single-channel deformation feature map;

[0020] S32: Increase the local contrast of the single-channel deformation feature map through the local histogram equalization algorithm, and then use the color mapping algorithm to map the single-channel deformation feature map with increased local contrast to the pseudo-color space to obtain a pseudo-color image.

[0021] Further, the input 3D point cloud includes the 3D point cloud obtained by rotating and scanning industrial products or agricultural and sideline products through a high-precision 3D laser sensor.

[0022] A 3D point cloud rendering device based on enhanced deformation features includes a processor and a memory; a program or instruction is stored in the memory, and the program or the instruction is loaded and executed by the processor to implement the 3D point cloud rendering method based on enhanced deformation features as described in any one of the above.

[0023] A computer-readable storage medium stores a program or instruction, and when the program or the instruction is executed by a processor, the 3D point cloud rendering method based on enhanced deformation features as described in any one of the above is implemented.

[0024] The beneficial effect of this application is that the pseudo-color image obtained in this application can effectively magnify the visual features of the deformation on the surface of the point cloud, so that surface weak deformation type defects can be accurately observed and marked. Compared with unstructured point cloud data, the pseudo-color image in the form of an image obtained in this application also has the advantages of being easy to observe and detect. Moreover, compared with the existing point cloud rendering methods, the gain of this application for post-defect detection is also better than the existing point cloud rendering methods.

[0025] The method described in this application can be used to render the 3D point cloud of industrial products or agricultural and sideline products, especially to render the 3D point cloud of industrial products or agricultural and sideline products containing surface weak deformation type defects, so as to magnify the deformation features of surface defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of the method described in this application;

[0027] Figure 2 is a schematic comparison diagram of the rendering results of the existing point cloud rendering software and this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The technical solution of this application will be described in detail below with reference to the accompanying drawings.

[0029] Example 1

[0030] As Figure 1 shown, assume that the original 3D point cloud data S obtained by scanning a workpiece containing surface weak deformation originDepth in one of the original 3D point clouds Then the specific steps of this application are as follows:

[0031] (1) Smooth the original 3D point cloud to obtain the smoothed point cloud data In the original three-dimensional point cloud, the surface of the point cloud still presents a curved shape, which is due to the inherent shape of the workpiece surface. The depth changes caused by the curved surface of the workpiece surface and the depth changes caused by the weak deformation of the workpiece surface share the color space, which will result in weak color changes in the weak deformation of the workpiece surface, making it difficult to label and detect. Therefore, first, the overall depth change in the depth map is weakened through surface fitting, and then the depth change range is reduced to enhance the recognizability of local weak deformations in the final rendered pseudo-color map. The least squares method is used for the original three-dimensional point cloud to perform quadratic surface fitting to obtain the surface equation z = F(x, y); for the original three-dimensional point cloud the Z-direction change caused by the surface z = F(x, y) is eliminated to obtain the flattened point cloud data The calculation formula for the flattened point cloud data is as follows:

[0032]

[0033] (2) Extract the deformation features from the flattened point cloud data to obtain the deformation feature map The deformation feature map can enhance the visual recognizability of the weak deformation of the workpiece surface in the point cloud and weaken the visual interference of the noise texture of the workpiece. The specific process is described as follows:

[0034] In the first step, extract the Z-direction enhanced normal vector as the deformation feature. First, use the enhancement coefficient Z time to multiply the Z component of the flattened point cloud data , and then estimate the normal vector of the neighborhood tangent plane. The normal vector direction correction includes calculating the tangent plane for each point according to the neighborhood, extracting the normal vector, and correcting the normal vector direction. Denote the process of extracting the normal vector as f normal : Then the calculation method for extracting the Z-direction enhanced normal vector feature can be expressed by the following formula:

[0035]

[0036] Among them, δ(x, y, z) represents the neighborhood point set of the point (x, y, z), and this point set is actually solved using KDTree; ⊙ represents the bitwise multiplication operation of vectors. The set form of the normal vector features extracted for each point in the flattened three-dimensional point cloud data is expressed as The normal vector set contains the normal vector of each point, and the normal vector can reflect the deformation information of each point.

[0037] In the second step, the normal vector set Arrange them in a matrix according to the positions of their corresponding coordinate points, and denote the normal vector of each coordinate point as Normalize the RGB channel values of each dimension to the range of [0, 255] to obtain the deformed feature map. The matrix dimension of the deformed feature map is (w'+2k, h'+2k, 3), which is the same as the matrix representation dimension of the general image format, and can be displayed and modified using an image viewing software.

[0038] For the deformed feature map Perform pseudo-color mapping to obtain the pseudo-color map Because the characteristics of weak defects can be further enhanced through contrast enhancement and color mapping, the deformed feature map is subjected to principal component analysis (PCA), contrast-limited adaptive histogram equalization (CLAHE), and color mapping (ColorMap) to obtain the pseudo-color map Although the deformed feature map already has a format similar to an image and can provide a good visualization effect for surface weak deformations, it is still difficult to identify smaller deformations. Therefore, it is necessary to further enhance the visual effect of weak defects.

[0039] (3) Deformed feature map The color of the deformed feature map is obtained by directly mapping the components of each dimension of the normal vector feature to the RGB space. Although the normal vector feature can effectively reduce visual interference caused by noise textures, etc., the color obtained directly from the numerical values of each dimension of the normal vector has low contrast, and it is still difficult for the human eye to identify smaller defects, so they cannot be marked. It is necessary to further enhance the contrast of the pseudo-color map. This application discovers that by combining a contrast enhancement algorithm and a color mapping algorithm, the visual effect of weak deformation type defects can be effectively enhanced. The specific steps are as follows:

[0040] First step, use the principal component analysis algorithm to reduce the dimension of the deformed feature map of the form (w'+2k, h'+2k, 3) to (w'+2k, h'+2k, 1), so as to retain the effect of the normal vector feature to the greatest extent.

[0041] Second step, use the contrast-limited adaptive histogram equalization algorithm to increase the local contrast of the deformed feature map and improve the visualization effect of local weak deformations. The block coefficient of the contrast-limited adaptive histogram equalization algorithm is 8, and the local contrast limit is 12.

[0042] Third step, use the color mapping algorithm to map the pseudo-color map to the pseudo-color space to obtain the pseudo-color image

[0043] The above are the detailed steps for obtaining a pseudo-color map from a single original point cloud through the rendering method proposed in this application. Performing the same operation on all the original point cloud data in parallel can obtain a large amount of pseudo-color map data.

[0044] Based on the original three-dimensional point cloud dataset obtained by three-dimensional sensor scanning of industrial products containing surface weak deformation defects, this application performs rendering with enhanced deformation features. Figure 2 It shows a comparison between the rendering results of the existing mainstream software CloudCompare and OpenCv software and the pseudo-color map results rendered by this application. Since in the rendering results of three-dimensional point cloud software, different viewing angles need to be adjusted to view the point cloud. From the top-down perspective, only the texture of the point cloud itself can be seen, and the surface shape changes of the point cloud cannot be identified. Figure 2 The comparison software shown above all searches for the best viewing angle manually. It can be seen from the experimental results that in the rendering results of the point cloud software, although the original shape of the point cloud can be seen more clearly, the weak deformation will be submerged or blocked by the texture and is difficult to identify, with an obscure visual effect. In the rendering effect of this application, the weak deformation can be clearly identified, and the visual effect is more fluent, effectively enhancing the surface weak deformation in the point cloud that was originally difficult to identify.

[0045] Embodiment 2

[0046] This application also provides a three-dimensional point cloud rendering device with enhanced deformation features, including a processor and a memory; programs or instructions are stored in the memory, and the programs or instructions are loaded and executed by the processor to implement the three-dimensional point cloud rendering method with enhanced deformation features in Embodiment 1.

[0047] Embodiment 3

[0048] This application also provides a computer-readable storage medium. This computer-readable storage medium can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions run on a computer, the computer is made to execute the three-dimensional point cloud rendering method with enhanced deformation features in Embodiment 1.

[0049] Those skilled in the art can clearly understand that the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0050] The three-dimensional point cloud rendering method, device, and storage medium for deformation feature enhancement provided by the present application can be implemented in many specific ways and means for realizing this technical solution. The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application. Each component not clearly defined in this embodiment can be implemented using existing technologies.

Claims

1. A 3D point cloud rendering method based on enhanced deformation features, characterized in that, it includes: S1: Smooth the input 3D point cloud to obtain a smoothed 3D point cloud; S2: Extract deformation features from the smoothed 3D point cloud to obtain a deformation feature map; S3: Perform pseudo-color mapping on the deformation feature map to obtain a pseudo-color image; wherein, the pseudo-color image is the output result obtained by rendering the 3D point cloud; wherein, the step S1 includes: S11: Calculate the support plane of the 3D point cloud, and at the same time perform coordinate rotation transformation on the 3D point cloud, and then make the support plane coincide with the XY plane of the coordinate axis to obtain a rotated 3D point cloud; S12: Fit the main target in the rotated 3D point cloud through a quadratic surface to obtain a fitted surface, and eliminate the deformation of the coordinate axis in the Z direction caused by the fitted surface to obtain a smoothed 3D point cloud; The step S2 includes: S21: Enhance the smoothed three-dimensional point cloud in the Z direction to obtain the enhanced three-dimensional point cloud, with the enhancement coefficient being Z time ; then extract the neighborhood point sets of all coordinate points in the enhanced three-dimensional point cloud, and then estimate the local plane based on the neighborhood point sets to obtain the normal vector set of the three-dimensional point cloud; S22: Arrange the normal vector set into a matrix according to its corresponding coordinate point positions, and normalize the RGB channel values of each dimension of the matrix to between [0, 255] to obtain a deformation feature map; wherein, the RGB channel values respectively correspond to the XYZ coordinate values of the normal vector of the coordinate point; The step S3 includes: S31: Reduce the dimension of the deformation feature map through the principal component analysis algorithm to obtain a single-channel deformation feature map; S32: Increase the local contrast of the single-channel deformation feature map through the local histogram equalization algorithm, and then use the color mapping algorithm to map the single-channel deformation feature map with increased local contrast to the pseudo-color space to obtain a pseudo-color image.

2. The 3D point cloud rendering method according to claim 1, characterized in that, the input 3D point cloud includes a 3D point cloud obtained by rotating and scanning industrial products or agricultural and sideline products through a high-precision 3D laser sensor.

3. A 3D point cloud rendering device based on enhanced deformation features, characterized in that, it includes a processor and a memory; a program or instruction is stored in the memory, and the program or the instruction is loaded and executed by the processor to implement the 3D point cloud rendering method based on enhanced deformation features according to any one of claims 1 to 2.

4. A computer-readable storage medium, on which a program or instruction is stored, and when the program or the instruction is executed by a processor, the 3D point cloud rendering method based on enhanced deformation features according to any one of claims 1 to 2 is implemented.

Images